No option is available (Split will make horizontal query windows).
Solution
Step 1: Close all docs
Step 2: choose Tabbed Document
Step 3: New Query Window 1
Step 4: New Query Window 2
Step 5: At this point, the menu option "New Vertical Tab Group" should be available now
Step 6: Enjoy it!