[v2] [gatsby-plugin-offline] Service Workers throw errors when offline

[fabe]

It seems that service workers are broken on v2. They are successfully installed, but return errors when the user is offline:

Cannot read property 'pageContext' of undefined

This also leads to this Lighthouse warning:

Service worker does not successfully serve the manifest's start_url.

Could potentially be related to the pageContext rename?

Steps to reproduce

Example repo: https://github.com/fabe/test-gatsby-v2-offline
Production build: https://peaceful-wozniak-ecfc89.netlify.com

Reload the site while offline (after installing the service worker).

Environment

  System:
    OS: macOS High Sierra 10.13.5
    CPU: x64 Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 8.11.3 - /usr/local/bin/node
    Yarn: 1.7.0 - /usr/local/bin/yarn
    npm: 5.6.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 67.0.3396.99
    Safari: 11.1.1
  npmPackages:
    gatsby: next => 2.0.0-beta.12
    gatsby-plugin-offline: ^2.0.0-beta.2 => 2.0.0-beta.2
  npmGlobalPackages:
    gatsby-cli: 1.1.58

[kkemple]

hey @fabe thanks for submitting this! I'm actually working on this problem right now!

[josepjaume]

I'm using 2.0.0-beta.65 (I just migrated from 1.0) and this is also happening to me. Any clue on what's going on?

Here's a screen capture:

Thanks!

[octalmage]

Same issue with: https://jason.stallin.gs

[octalmage]

The issue is that the static json data paths aren't getting stored in the service worker. Still not sure why, but that json path contains the pageContext.

[LekoArts]

@kkemple Wanna have a look at this?

[KyleAMathews]

@octalmage these were fixed in #7092

[vtenfys]

@KyleAMathews I'm also still having this issue with JSON data, with gatsby-plugin-offline version 2.0.0-beta.9 which contains the fixes in #7092, so I don't think it's actually fixed...

[vtenfys]

Just checked and this is indeed still an issue - I forked @fabe's original repo and updated the packages. Link: https://github.com/davidbailey00/test-gatsby-v2-offline / deployed at: https://davidbailey00-test-gatsby-v2-offline.netlify.com/

[vtenfys]

I realised that I only updated package-lock.json and not yarn.lock, so Netlify was still deploying with the outdated packages.

However - this is still an issue on multi-page sites, as they may load additional json data when switching pages, which is not cached by default. For example, the Gatsby 2 tutorial site does not work reliably offline - go to https://wizardly-babbage-0faa32.netlify.com/ and browse a few pages, then simulate offline network and reload. Upon visiting some pages, the same error "Cannot read property 'pageContext' of undefined" still sometimes occurs.

I think a fix for this would simply be to add json to the array of default runtime cached extensions. @KyleAMathews please let me know if that sounds ok and I can easily submit a pull request, since it's only a one-line change.

[vtenfys]

Update: I tried locally modifying gatsby-node.js inside my node_modules, including css and json in the runtime-cached extension list.

However, this still didn't solve the problem as I still couldn't reliably load other pages on the site, even after visiting them when online - error 0 was shown in the console when trying to load these files offline. (I checked the generated sw.js to make sure it had updated the changes.)

Either I'm misunderstanding how runtime caching works, or it doesn't seem to be working properly, since it isn't caching the json files even when json is added to the urlPattern regex.

---

Edit: even on the official Gatsby v2 website this is broken. Open up the dev tools and simulate offline mode, then click around some pages. On some pages the whole screen will turn white and a bunch of Cannot read property 'pageContext' of undefined errors will be shown in the console, as with the initial issue.

For me the problem is triggered when going to the tutorial and clicking part two, even though I've been to this page before. On the v1 website, there are no such problems. I hope this issue can be reopened since it's clear that there are still some major problems.

Additionally, for both the v1 and v2 websites, when offline, only the homepage can be opened directly without showing a "No Internet" error, but other cached pages can be navigated to. It seems odd not to cache all pages so that they can be opened directly, especially when their contents are all cached anyway. Shall I open another bug report for this?

[vtenfys]

I've started work on a proper fix for this

[vtenfys]

I'm almost ready with the fix but I've found another problem which might have been caused by my changes, so I'm just checking on that before sending a PR.

[vtenfys]

I've determined the other problem is not caused by my changes, since it affects pages using the current version. I'll submit a PR shortly along with a new bug report.

[vtenfys]

On second thoughts I've decided to hold off on submitting the PR, since my changes don't actually fix the root problem. I had refactored gatsby-node.js, among various other files, to cache all files in the public directory matching a regex, resulting in the entire site being cached when the worker is installed.

However, the root problem was only masked - given that for a large website it's not feasible to statically-cache all contents, the bug would still occur, assuming the options were changed to skip caching certain files, as soon as anyone visited a non-cached page.

This is proving quite difficult to debug, since gatsby-plugin-offline isn't designed to work when running gatsby develop, and the compiled site doesn't show full debugging information. As a result, it's slightly harder to tell why certain problems are occurring, or exactly which file is causing each problem. Also, a different error is shown in Firefox in the same situation, as well as various other behaviour differences between the two browsers with other problems I've found (related to the offline handling).

Now that I've familiarised myself somewhat with the codebase I want to have another crack at this tomorrow, although I feel like it might require some more complex changes, possibly to different plugins which gatsby-plugin-offline depends on, and/or the core Gatsby code.

N.b. my original changes, which I was going to submit as a PR, can be seen at https://github.com/davidbailey00/gatsby/tree/fix-issue-6212

[vtenfys]

I'm making progress on a better fix, although it's taking me a long time due to various interdependent bugs which I'm also trying to fix. I'll post an update later when I've made more progress

[josepjaume]

Thanks, @davidbailey00! Much appreciated!

[vtenfys]

Most of the fixes are now done - I'm just trying to find a way to hard refresh when a page isn't found. Otherwise, either we get a blank page from the service worker, rather than a 404 / network error, or we get an infinite loop of blank pages if we do a normal refresh. Unfortunately, sw-precache seems to ignore search queries when matching against navigateFallBackWhitelist, so I can't simply append ?no-cache to the URL upon a page-not-found error (I've tried doing this).

[vtenfys]

I've found a way to patch sw.js which allows me to do what I'd described in my last comment. Still some issues which I need to fix before PR'ing but lots of progress has been made.

[vtenfys]

Everything fixed except I've uncovered what appears to be bug in React itself, in which className isn't applying the class correctly to the outermost DOM element - but only when loaded from the service worker (i.e. the offline shell) and only in Chrome (not in Firefox, haven't tested Edge etc). Trying to find a workaround for this.

[KyleAMathews]

@davidbailey00 what are you working on exactly? The original issue described here was fixed last week with #7066 The 404 issue was fixed in #7268

It's often fastest to put up small PRs that solve slices of the overall problem. Would love to look at what you have!

Going to close this issue now as it's fixed but please open up a new issue describing what you're trying to solve!

[KyleAMathews]

Also, yeah, service workers are only for the built site so you need to debug it (unfortunately) by running gatsby build then gatsby serve to view the results.

[rcaracaus]

@KyleAMathews this was happening periodically for us on a client site we launched recently, we just updated gatsby yesterday to 2.0.15 and we are hoping that fixed it, still watching our log... but I went to Gatsbyjs.com and it happened there first time I ever went to site, maybe you need to update that site too to use latest v2.. or maybe it isn't fixed.

[DSchau]

@rcaracaus would you be able to open up the inspector and grab the Gatsby version for me?

document.head.querySelector('meta[name="generator"]').getAttribute('content')

We made a bunch of improvements to gatsby-plugin-offline (thanks @davidbailey00!), so it's possible you're on a cached version of the site that doesn't contain those improvements, which is unfortunate. Service workers are challenging 😱

[rcaracaus]

"Gatsby 2.0.5" -- when I just did that on Gatsbyjs.com -- but I did hard refresh the site when I got the error and it went away... so that might mean if I had visited the site in the past I might have had a different version.

Also probably a dumb question but (I am still learning serviceworker and PWA's).. on our site we don't specify gatsby-plugin-offline in our config file, should we be (assuming it is disabled)? Could not using it have caused the the pageContext issue for us? With the recent updates, if we have gatsby-plugin-offline disabled will we even benefit from the changes? Do Gatsby v2 sensible defaults recommend to have that plugin enabled?

[rcaracaus]

@DSchau ^

[kkemple]

Also probably a dumb question but (I am still learning serviceworker and PWA's).. on our site we don't specify gatsby-plugin-offline in our config file, should we be (assuming it is disabled)? Could not using it have caused the the pageContext issue for us?

@rcaracaus not having the gatsby-plugin-offline plugin installed and in config will definitely make that error occur. I would recommend installing and adding to config and seeing if that resolves your issue.

[mgifford]

Ok, so if I set up a demo site on my local.

Add the manifest & offline:
npm install --save gatsby-plugin-manifest
npm install --save gatsby-plugin-offline

Then edit gatsby-config.js

Uncomment 'gatsby-plugin-offline',

Then run:
npm run develop

I should be able to evaluate http://localhost:8000/ with Google's Lighthouse and have 100% for my Progressive Web App settings, right?

I'm seeing some failures here that I'm thinking could be fixed out of the box. Maybe I'm wrong about this though. I haven't played with this at all much.

[fabe]

@mgifford To test Lighthouse scores/service workers, run npm run build && npm run serve or test the deployed version. The develop version of your site will not have all the optimizations.

[mgifford]

Brilliant.. with that the only thing that doesn't work is "Does not redirect HTTP traffic to HTTPS" which really is an easy enough thing to do in Apache anyways.