We have been using puppeteer in one of our projects to write end-to-end tests. We run our tests in headful mode to see the browser in action.
If we start puppeteer tests and do nothing in our laptop (just watch the tests being executed) then all the tests will pass.
However if we are doing our regular work in our laptop while tests are running then tests would fail randomly. This was quite puzzling.
Debugging such flaky tests is hard. We first suspected that the test cases themselves needed more of implicit waits for element/text to be present/visible on the DOM.
After some debugging using puppeteer protocol logs, it seemed like the browser was performing certain actions very slowly or was waiting for the browser to be active ( in view ) before performing those actions.
Chrome starting with version 57 introduced throttling of background tabs for improving performance and battery life. We execute one test per browser meaning we didn't make use of multiple tabs. Also tests failed only when the user was performing some other activities while the tests were executing in other background windows. Pages were hidden only when user switched tabs or minimized the browser window containing the tab.
After observing closely we noticed that the pages were making requests to the server. The issue was the page was not painting if the page is not in view. We added flag --disable-background-timer-throttling but we did not notice any difference.
After doing some searches we noticed the flag --disable-renderer-backgrounding was being used in karma-launcher. The comment states that it is specifically required on macOS. Here is the code responsible for lowering the priority of the renderer when it is hidden.
But the new flag didn't help either.
While looking at all the available command line switches for chromium, we stumbled upon --disable-backgrounding-occluded-windows. Chromium also backgrounds the renderer while the window is not visible to the user. It seems from the comment that the flag kDisableBackgroundingOccludedWindowsForTesting is specifically added to avoid non-deterministic behavior during tests.
We have added following flags to chromium for running our integration suite and this solved our problem.
1const chromeArgs = [ 2 "--disable-background-timer-throttling", 3 "--disable-backgrounding-occluded-windows", 4 "--disable-renderer-backgrounding", 5];
References