gatsby-plugin-offline: SW isn't always updated on each build, resulting in inconsistent state

[TylerBarnes]

This is really hard to reproduce but I'm googling gatsby enough that I see this a couple times a week.
When opening a gatsbyjs.org page from google in a new tab, sometimes (rarely) the full page is blank and remains blank until a manual browser refresh.
I cleared my service worker and full site cache on the site last week thinking it was just a leftover early v2 bug but I'm still seeing it this week.

Screenshot:

[TylerBarnes]

Just realized I should've checked my console for errors to pass on 🤣 I'll check next time this happens

[oorestisime]

I sometimes get Failed to load ‘https://www.google-analytics.com/analytics.js’. A ServiceWorker passed a promise to FetchEvent.respondWith() that rejected with ‘TypeError: NetworkError when attempting to fetch resource.’. but after a few seconds everything is "refreshed" and work and a service worker is installed. do you have the same?

[ZLevine]

I've seen this several times as well over the past few months. I'll grab console output next time.

[btk]

Yes, I come across this weekly, on the gatsbyjs.org. Not only coming from Google, also while directly loading a page.

As all of us I couldn't think of looking at the console. I'm writing a comment here so I will update this with console errors when it happens again.

[nicklewisatx]

I'll leave the console open. Here's what I got:

My two cents. The page is way too old. Notice the response header for the doc is from Nov 18th. Notice the 404s at the bottom. Its as if the doc cache is stuck, but the webpack js files insist on re-validating. (if it was pure browser cache issues, I'd expect the opposite problem?)

[LekoArts]

Should be fixed with: #9907

[dustinhorton]

just experienced it @ https://www.gatsbyjs.org/starters?d=gatsby-plugin-create-client-paths

[LekoArts]

@davidbailey00 ^

[vtenfys]

Hmm, perhaps we aren't always triggering an SW update when a new version of the site is built. This could cause an inconsistent state where we have some resources cached for build A, and others which are being requested (with the URLs for build A) but we have build B actually running on the server (newer).

Unfortunately this is one of those problems which is tricky to debug without knowing exactly what's going wrong - I'll do my best to find the cause of this next week!

@dustinhorton, thanks for letting us know about this issue! Please could you also make another comment if you encounter the same problem again, while this bug is open? With these problems, sometimes they could be one-offs (i.e. the error is only apparent a single time for each user after upgrading a Gatsby component) - on the one hand this reduces the number of affected page loads, but on the other hand, as before, it's harder to debug :( Hopefully this is reproducible, even if we don't yet know the specific cause.

[janicduplessis]

I've also seen this issue on my site. From what I remember the service worker was trying to load a file that didn't exist anymore after deploying a site update. I ended up disabling the offline plugin and the problem stopped. Hopefully this can help but I don't have more specific info since I disabled the sw as a workaround for now.

[vtenfys]

@janicduplessis Roughly how long ago was this? I'm just wondering if this problem might actually be fixed by some recent changes to ensure-resources.js and perhaps we can close it after all.

[janicduplessis]

@davidbailey00 This was 9 days ago, with [email protected]. Let me know if you release a version with the fix I can deploy it and see if I can reproduce the bug still.

[vtenfys]

@janicduplessis Okay, I think if you update your packages and deploy then it might be fixed!

Or what might happen is that it will error once (since you're upgrading from a buggy version), but then if you make some minor change(s) and deploy again with the same version then it should be okay.

Could you try this and let me know what happens? Thanks!

[janicduplessis]

@davidbailey00 Cool, I just updated and deployed with the offline plugin. I'll see if I hit the white screen again while deploying further updates in the next few days.

[dustinhorton]

https://www.gatsbyjs.com/gatsby-days-preview-andrew/

[janicduplessis]

I can confirm I also hit the issue again on my site, seems like the same thing that @dustinhorton just linked.

I kept the tab open, let me know if you need more info about the error.

[YoshiWalsh]

I've found a way to reproduce this bug reliably on my site. If I use the DevTools (in Firefox or Chrome) and in settings select "Disable Javascript", my site will always appear as a blank page. (Service Workers continue running even when JS is disabled, which is interesting)

It has taken me a long time to find the cause of this issue, because in Firefox (my primary browser) there's no indication on the Network tab that a response was provided by a Service Worker. I've been trying to work out why my server was providing different content to browsers vs to cURL.

Anyway, my understanding is that with JS disabled, the plain-HTML version of the page should still be served by the Service Worker, but instead it seems to be serving a blank page. I think this is most likely the root cause of the issue, and that the different error messages above are just different things that can cause the script-based population of the page to fail.

[vtenfys]

@YMIndustries thanks for the information! Unfortunately what you've described isn't a reproduction of the original problem, despite the same effect - the problem is that sometimes during upgrades, there's a mismatch between file versions which results in 404s. Disabling JS unfortunately can't help us figure out exactly when and how these problems are occurring, and the offline shell is intentionally blank since JS is expected to work.

Having said that, we've been considering the advantages and disadvantages of caching HTML rather than just resources for each page, regarding your comments about the blank pages appearing with JS disabled - the main disadvantage is that Gatsby doesn't normally download an HTML file for internal navigation, so by doing this we would increase bandwidth on each navigation. However, there are also some advantages, namely that the complexity of the offline plugin could be reduced significantly, and I believe it would eliminate these kinds of problems with mismatching filenames/versions.

[YoshiWalsh]

Same symptoms, different problem I guess. Is it worth me creating another issue to track the "blank page when JS is disabled" problem?

[vtenfys]

I'm not sure if it's worth creating another issue, because if users have disabled JS in their browser then the SW won't install, so they will see the HTML content correctly.

The difference in your case was that JS was disabled after the SW was already installed, rather than being disabled beforehand, but we can expect this to be reasonably unlikely in the real world. For this reason I think we shouldn't make an effort to handle it - and, if an end user is savvy enough to disable JS, then they can probably figure out that it's the reason for the site to stop working.

Thanks for checking though, and thanks again for the first comment with your initial findings!

[YoshiWalsh]

Ah, the reason I was going to make a separate issue for it is because I didn't realise that it was intended behaviour for the HTML result of sw.js to not include the page contents. Thanks for your replies.

[koreyboone]

Seems gatsby-plugin-offline has skipWaiting enabled by default.

Note: If your web app lazy-loads resources that are uniquely versioned with, e.g., hashes in their URLs, it's recommended that you avoid using skip waiting. Enabling it could lead to failures when lazily-loading URLs that were previously precached and were purged during an updated service worker's activation.

From: https://developers.google.com/web/tools/workbox/modules/workbox-sw

https://stackoverflow.com/questions/51715127/what-are-the-downsides-to-using-skipwaiting-and-clientsclaim-with-workbox

This stackoverflow thread suggests forcibly reloading if lazy-loading failures are encountered. If we are ok with that UX then enabling skipWaiting may be ok.

[vtenfys]

@koreyboone Thanks for pointing this out! @pieh and I talked about this briefly last week and we're considering changing this option as well as clientsClaim due to the reasons you've mentioned - of course we'll have to look into potential impact of changing this.

[dustinhorton]

i continue to experience this

[vtenfys]

@dustinhorton Was this showing repeatedly or just once? Old SW issues can still appear after we've updated the site, since the old SW is still installed in your browser - however, if this keeps happening then please open a new issue.

[dustinhorton]

@davidbailey00 Just to clarify—this is the expected tradeoff, and is functioning as expected if I refresh and then get the page?

[vtenfys]

It's not an expected trade-off in the sense that it should happen with each update - what I meant was that old bugs can still appear once the site has updated, due to browsers having the previous SW installed, but future updates should be okay assuming the bug was fixed because by that point, browsers have installed the latest SW.

[riddla]

@davidbailey00, I‘m curious as I faced this issue a lot the last days: would it make sense for the Gatsby site to run a while with „gatsby-plugin-remove-serviceworker“ in place? To make sure the leftovers from the now fixed bug really get removed for every (former) visitor?

[vtenfys]

@riddla In most cases, the issue should fix itself after a single reload, although it wouldn't do any harm to run the remove plugin temporarily so yes, this could work. I'm currently working on another PR (#11965) which changes how the offline plugin works so stay tuned :)