Some marionette manifests are not included by the master manifest
Categories
(Testing :: Marionette Client and Harness, task, P3)
Tracking
(firefox128 fixed)
Tracking | Status | |
---|---|---|
firefox128 | --- | fixed |
People
(Reporter: gbrown, Assigned: whimboo)
References
(Blocks 1 open bug)
Details
(Whiteboard: [webdriver:m11])
Attachments
(1 file)
In bug 1892038 we noticed that some marionette tests could not be run in CI because they were not included in testing/marionette/harness/marionette_harness/tests/unit-tests.toml. It is not clear to me if all marionette manifests should be included there or not, so just noting here for someone to review and consider.
Comment 1•3 months ago
|
||
Is this a correct assessment of the issue? It is surprising to learn that the file needs to be listed here. In other test types, listing it in the directory-specific toml file is enough for it to be picked up in CI.
I previously wondered about the test selection behavior (locally) at https://phabricator.services.mozilla.com/D196188#6507426
If it ultimately turns out that the test needs to be listed in unit-tests.toml, then that should be documents.
Assignee | ||
Comment 2•3 months ago
|
||
As we discussed we do not have the time to change the underlying packaging behavior for Marionette tests. As such we suggest that the unit-test.toml
file gets updated so that it contains all the missing references to tests that are not run yet in CI. As suggested we should also document that and maybe this could directly be done in the same file.
Assignee | ||
Comment 3•2 months ago
|
||
Updated•2 months ago
|
Updated•2 months ago
|
Pushed by hskupin@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/b609a929be9f [marionette] Fix master manifest for in-tree Marionette tests. r=webdriver-reviewers,application-update-reviewers,Sasha,nalexander
Comment 5•2 months ago
|
||
Backed out for causing marionette failures on test_exitcode.py.
Assignee | ||
Comment 6•2 months ago
|
||
That's interesting. This test was running before as well and did not fail. My patch only changed the order between the update and the XRE tests. And I have to admit that I missed to have a final check for the try build results. There it's failing as well but only for MacOS and no other platform. I'll take a look what's wrong here. Most likely that the update tests do not correctly clean-up.
Assignee | ||
Comment 7•2 months ago
|
||
I filed bug 1898477 to cover the specific test failure that happens when the update test is run before the xre test. For now I will keep the update test to run as last test to prevent that.
Pushed by hskupin@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/a9f0952d79a4 [marionette] Fix master manifest for in-tree Marionette tests. r=webdriver-reviewers,application-update-reviewers,Sasha,nalexander
Comment 9•2 months ago
|
||
bugherder |
Assignee | ||
Updated•2 months ago
|
Description
•