Enabling parallel testing in each scheme will reduce the execution time for each test run.
To enable this option:
-Select the scheme and -> Edit
-Go to Test -> Under Test XCUITests is available, tap on Options
-Enable Execute in parallel on simulator option

Not sure what to do...Tests work running on parallel but there are timing issues when running them this way. The test suite more affected is the XCUITest in iPad sim with around 15 tests failing:
https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/5c49b2b978dbac00017bea9a

My doubt is if we should try to fix them here, or land this change and work on the tests later on when we have more information about the intermittent failures, if they are always the same or they change, if it happens more in one sim or the other if it depends on BB workload...

No-Jun, would you mind letting me know your thinking about this? Thanks!

Btw, I did not add numbers here but the test execution time, if there are no retries, goes approximately from around (in regular execution) 110-120 min to 60-70 (in parallel) as per run seen so far.

Hmm, so for some strange reason, it cannot transition properly, after 10 seconds or so. Either there is a timing issue in our script, or the device is busy to execute the test command. To me it looks like it may not be the timing, because most failures here complain about Cannot get from BrowserTab to Tabtray. Test log shows:

5:31:41.623 XCUITests-Runner[6739:26781] -[XCTRunnerIDESession testCase:didFailWithDescription:inFile:atLine:]
05:31:41.624 xcodebuild[5568:21168] _XCT_testCaseDidFailForTestClass:XCUITests.DragAndDropTests method:testRearrangeTabsTabTray withMessage:Cannot get from BrowserTab to TabTray. See /tmp/sandbox/workspace/XCUITests/FxScreenGraph.swift:873 file:/tmp/sandbox/workspace/XCUITests/DragAndDropTests.swift line:105
05:31:41.623 XCUITests-Runner[6739:26781] -[DragAndDropTests testRearrangeTabsTabTray] started activity <XCActivityRecord: 0x600003880d70> 2019-01-24 13:31:41 +0000: Assertion Failure: DragAndDropTests.swift:105: Cannot get from BrowserTab to TabTray. See /tmp/sandbox/workspace/XCUITests/FxScreenGraph.swift:873
05:31:41.625 xcodebuild[5568:21168] -[XCUITests.DragAndDropTests testRearrangeTabsTabTray] will start activity CF72C1CF-2C4D-4D82-8415-1DE0C147B133:'Assertion Failure: DragAndDropTests.swift:105: Cannot get from BrowserTab to TabTray. See /tmp/sandbox/workspace/XCUITests/FxScreenGraph.swift:873'
05:31:41.624 XCUITests-Runner[6739:26781] Using modern screenshot path for rect {{inf, inf}, {0.0, 0.0}}
05:31:41.624 XCUITests-Runner[6739:26781] Request screenshot of screen with ID 1 (quality 1)
05:31:41.624 XCUITests-Runner[6739:26781] Creating future for 'Request screenshot' with timeout 30.00
t = 24.80s Assertion Failure: DragAndDropTests.swift:105: Cannot get from BrowserTab to TabTray. See /tmp/sandbox/workspace/XCUITests/FxScreenGraph.swift:873

Sounds as if there is no path in screengraph? which is very strange.

I think we should try to fix it here, rather than landing the change. Let's timebox the work to end of this week, to see whether we can find more clues.

We had that issue in the past with the screengraph when trying to go to tab tray by tapping on the tabs button, it was not waiting enough to see the button and tap on it to go to tab tray. It still happens not very often though when the test run slower than usual.

But, yeah, agree on trying to get a fix within this week. Thanks1

I have been investigating and trying to fix this. It is for sure an intermittent on BB side. Running the same test suite, same device in Bitrise, there are only 8 failures and none of them is due to the unavailability to route from BrowserTab to Tab Tray. Please see logs:
https://app.bitrise.io/build/f092e7812d92f99d Under App&Artifacts -> index.html

In fact, we have started to see this issue on BB master for iPad too:
https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/5c5105219932d3000192d898#tests

I have tried a workaround in this PR for this particular routing issue, and similar ones, which is to directly tap on the tabs button to go to tab tray instead of doing a navigator.goto(TabTray)
This way I'm seeing less frequently this issue on BB and still never on Bitrise.
Once I clean the PR will ask for r? again to enable the parallel testing with this solution if that looks go to you and starting from there fix the rest of intermittents that may appear

Adding ni FYi, no action needed but so that you read latest status on this and we can comment offline...Thanks!

To update the status of this bug: We have decided not to land the PR for enabling parallel testing until the environment is more stable. We will try again when there is a new release of Xcode if there are less crashes and intermittent failures due to timing.
This timing issues were making routing between screens and performing actions very flaky.