Per https://bugzilla.mozilla.org/show_bug.cgi?id=1759288#c209, there are frequent failures of the mochitest-remote tests on linux ASAN, where the workers drop off the net part way through the run. Looking at one that did run to completion (https://share.firefox.dev/3VCRxG1, from https://treeherder.mozilla.org/jobs?repo=autoland&revision=ff34de8a49b079ff98eb45e3243575e2a4d5646b&selectedTaskRun=XztS7P3kTKCc_UxO6IEjdg.3), it seems likely the worker is running OOM and sometimes crashing.
Can something be changed to use less memory here? If not, these tests should probably run on bigger instances.

Match the instance type used by the mochitest-devtools-chrome.

Interesting. As it looks like the messagehandler browser chrome tests specifically cause such an increased memory usage. Julian, is that only visible on Linux or do Windows ASAN builds show similar behavior?

Before we bump the instance type we probably should indeed check what's causing the high memory usage. Maybe running these tests locally with the gecko profiler active could give some insights.

https://profiler.firefox.com/from-url/https%3A%2F%2Ffirefox-ci-tc.services.mozilla.com%2Fapi%2Fqueue%2Fv1%2Ftask%2FeI77sO4MRTihi63mLLOZMQ%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_resource-usage.json/marker-chart/?globalTrackOrder=0&thread=0&timelineType=stack&v=10 is from https://treeherder.mozilla.org/jobs?repo=autoland&revision=ff34de8a49b079ff98eb45e3243575e2a4d5646b&searchStr=asan%2Cremote&selectedTaskRun=eI77sO4MRTihi63mLLOZMQ.0
Looks like it peaks at 7GB memory usage.

I guess worth noting the windows job is running with 16GB ram already.

I see. So then lets go ahead and update the instance type so it matches other browser chrome test suites.

https://hg.mozilla.org/mozilla-central/rev/fa79fcc44490