Random errors : cannot start Firefox/FirefoxHeadless

[cedx]

Firefox : v67.0.3
OS : Ubuntu 19.04 and Windows 10
Karma : v4.1.0
Firefox launcher : v1.1.0
Node.js : v12.4.0

I've some projects whose Karma tests work as expected when I'm using the Chrome launcher. But when I use the Firefox launcher, I get the following error:

0 06 2019 19:37:21.060:INFO [karma-server]: Karma v4.1.0 server started at http://0.0.0.0:9876/
20 06 2019 19:37:21.062:INFO [launcher]: Launching browsers Firefox with concurrency unlimited
20 06 2019 19:37:21.067:INFO [launcher]: Starting browser Firefox
20 06 2019 19:37:21.124:ERROR [launcher]: Cannot start Firefox
	
20 06 2019 19:37:21.124:ERROR [launcher]: Firefox stdout: 
20 06 2019 19:37:21.124:ERROR [launcher]: Firefox stderr: 
20 06 2019 19:37:21.127:INFO [launcher]: Trying to start Firefox again (1/2).
20 06 2019 19:37:21.181:ERROR [launcher]: Cannot start Firefox
	
20 06 2019 19:37:21.181:ERROR [launcher]: Firefox stdout: 
20 06 2019 19:37:21.181:ERROR [launcher]: Firefox stderr: 
20 06 2019 19:37:21.183:INFO [launcher]: Trying to start Firefox again (2/2).
20 06 2019 19:37:21.237:ERROR [launcher]: Cannot start Firefox
	
20 06 2019 19:37:21.238:ERROR [launcher]: Firefox stdout: 
20 06 2019 19:37:21.238:ERROR [launcher]: Firefox stderr: 
20 06 2019 19:37:21.239:ERROR [launcher]: Firefox failed 2 times (cannot start). Giving up.
[19:37:21] 'test:browser' errored after 585 ms
[19:37:21] Error: karma: 1
    at ChildProcess.<anonymous> (/repo/js/enum.js/gulpfile.esm.js:93:40)
    at ChildProcess.emit (events.js:200:13)
    at ChildProcess.EventEmitter.emit (domain.js:494:23)
    at maybeClose (internal/child_process.js:1021:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)

This error seems to be random: on some projects, the launcher works as expected (for example: https://github.com/cedx/lcov.js). On other projects, the launcher triggers the error (for example: https://github.com/cedx/enum.js).

For the projects that fail, it also seems to be random: the error is triggered on Ubuntu 19.04 and Windows 10, but not on TravisCI with Ubuntu 16.04 (for example: https://travis-ci.com/cedx/enum.js/jobs/209733946).

My Karma config (which is the same on the two mentioned projects):

module.exports = config => config.set({
  basePath: require('path').resolve(__dirname, '..'),
  browsers: ['FirefoxHeadless'],
  files: [
    {pattern: 'lib/**/*.js', type: 'module'},
    {pattern: 'test/**/*.js', type: 'module'}
  ],
  frameworks: ['mocha', 'chai'],
  reporters: ['progress'],
  singleRun: true
});

Using Firefox, instead of FirefoxHeadless, does not change anything (i.e. same error).

[birtles]

This, I suspect, is because Firefox uses a separate launcher process (as of Firefox 64?) which triggers a separate process to avoid DLL injection. The launcher process then exits. Apparently previously -no-remote would prevent this behavior but that changed recently as part of the profile-per-install work (or so I'm told).

As a result, karma launcher will see that the launcher process exited while it is still in its STATE_BEING_CAPTURED state and throw an error. So, I think karma-firefox-launcher needs to override the behavior in the base launcher (i.e. in _execCommand) to not error in this case. However, I'm not sure how it should then find the actual Firefox process to watch so it can clean up the temp directory when it is done.

[birtles]

Actually, it looks like we can fix this just by passing --wait-for-browser. Some details here.

[birtles]

--wait-for-browser works well for starting the browser, but unfortunately killing the launcher process doesn't kill the browser process so the browser remains running after the test has completed.

https://azimi.me/2014/12/31/kill-child_process-node-js.html describes how to kill a process group but it doesn't seem to work for me on Windows, unfortunately (or at least not under WSL).

[birtles]

Looks like for talos we watch to see the first child process generated by the firefox process and then record that as the browser process: https://hg.mozilla.org/mozilla-central/rev/8461e2f532ed

[larsthorup]

With firefox@68 on Windows 10, I see it fairly consistently fail the first time it tries to launch, and then succeed the second time:

First it tries

16 07 2019 11:18:09.605:ERROR [launcher]: Cannot start Firefox

Then it succeeds

16 07 2019 11:18:12.672:INFO [launcher]: Trying to start Firefox again (2/2)

It will also leave an error dialog open:

[birtles]

Right, that's because the browser process will run in the background and on the second time through it will be running when the launcher tries to connect.

Depending on the load on your machine it might connect on the third time or it might fail altogether. With -wait-for-browser it will succeed first time.

I put up PR #95 for this but there seems to be something wrong with the CI for this repository. Looking at other PRs too it looks like no one is maintaining this anymore so I'm not sure if the PR will get merged.

[cedx]

On my side, upgrading Firefox to version 68 fixed the issue on Windows 10 (no more project in error, at least the ones I've tested), but not on Ubuntu 19.10 (exactly the same errors). Using a custom launcher with the -wait-for-browser flag did not change anything.

What I find weird: it's always the same projects that fail, as if it was related to the code rather than Firefox itself. But again: everything is fine if I use Chrome instead of Firefox.

it looks like no one is maintaining this anymore

(off topic) Too bad that this project is a Google one... I also find it really sad that Chrome/Chromium has become the dominant browser (and it will not get better with the new MS Edge).

[ericcarino]

Ubuntu 18.04
Karma 4.2
karma-firefox-launcher 1.1.0
node.js 10.16
Firefox 68.0

We are seeing the same issue when running inside a Docker container. It was working fine until we upgraded to Firefox 68. Now, the issue is reproducible 100% of the time. We downgraded back to Firefox 67 as a workaround in the mean time.

Exact same environment, but not w/ in Docker, we are having no issues. Weird.

[sebavan]

We are having also the issue since Firefox 68 and can not run our tests anymore :-( We can repro on any env using FF 68 with https://github.com/BabylonJS/Babylon.js

[sebavan]

@birtles your fix #95 seems to do it for us \o/ thanks a lot. Do you know is there are any new relrease to npm planned soon ?

[birtles]

@sebavan Thanks for confirming. I was planning on digging into the Ubuntu issues in #93 before releasing (assuming I even have the access required to submit a new release) but haven't had a chance to set up a VM for it yet.

Let me have another look at Ubuntu later this week (I'm currently traveling) and if I can't fix it easily I'll try to release another version.

[birtles]

Let me have another look at Ubuntu later this week (I'm currently traveling) and if I can't fix it easily I'll try to release another version.

Sorry, this is taking longer than expected (having trouble with the VM).