Use Service Worker for LiveDev server

[humphd]

In order to fix https://github.com/mozilla/thimble.mozilla.org/issues/1322, Bramble needs to add support for Service Workers, such that the LiveDev preview can intercept network requests for files and satisfy them with contents from the user's Filer filesystem.

This work should be done in stages, especially since we won't be able to totally replace what we have now, unless/until all our supported browsers implement Service Workers (http://caniuse.com/#feat=serviceworkers). I would suggest we do the following:

• Write a parallel version of the current rewrite-based LiveDev server based on SW, and get it working in general. Leave this code off by default, and enable it via an extension you opt into (for testing).
• Refactor the rewrite code, which is currently spread across the source tree, and properly separate the SW vs. rewrite servers
• Change the startup logic to favour the SW over the rewrite server if SW is supported.

The way our current setup works is as follows. At startup, we register our custom nohost servers here: https://github.com/mozilla/brackets/tree/master/src/extensions/default/bramble/nohost. These servers are basically tools for rewriting URLs and Paths, such that you can redirect the preview from loading a file directly. It's used, for example, to send instrumented versions of HTML and CSS files vs. what's on disk (i.e., we give the current contents of the editor vs. what's saved, and inject special JS scripts for highlighting and dynamic communication with the editor via postMessage).

Our method of serving content from the filesystem is to use pre-generated, cached, Blob URLs (see https://github.com/mozilla/brackets/blob/master/src/filesystem/impls/filer/BlobUtils.js). We also recursively rewrite HTML and CSS files in order to swap regular URLs for cached Blob URLs to things like images, scripts, stylesheets, etc. that are in the filesystem (see https://github.com/mozilla/brackets/blob/master/src/filesystem/impls/filer/lib/handlers.js). Basically, you can think of what we do like a build system, where we generate static sites Just In Time, and serve those vs. letting the network request things.

Doing this with a SW is going to involve us examining URLs, deciding if we have a file in the filesystem (or LiveDoc HTML/CSS open in the editor), and then getting that content (fs.readFile() via https://github.com/mozilla/brackets/blob/master/src/filesystem/impls/filer/BracketsFiler.js) and serving it with the proper MIME type (see https://github.com/mozilla/brackets/blob/master/src/filesystem/impls/filer/lib/content.js).

I'm not entirely sure how the communication should be structured here. As I see it, we'll have:

• a SW running that's watching for GET requests that match some pattern, and requests file contents when needed via postMessage
• an event handler in the editor code (main thread) that can service a request for a file's contents, figure out whether or not we a) have the file; b) if we have it open in an editor (and need to use an instrumented version vs. raw); c) figure out the right MIME type; d) send all this back to the SW. Probably this can all run in our custom LiveDev server?

As I said above, we can prototype this SW approach in parallel to the current rewrite/cache code--we just won't use the Blob URLs. Later we can refactor things to only rewrite/cache if we aren't running the SW server.

[humphd]

FYI, I've just landed another SW related patch to do precache for all static assets (let's us run Bramble offline): #547. This will be testable on https://bramble.mofostaging.net in a bit.

[humphd]

I've done some more reading on this, and I actually think we can do this using window.caches to essentially replicate how our BlogUrl cache works now, but put entire cached responses into the cache on the editor side, and serve it from the SW side.

[humphd]

I tried an experiment today. I wrote a small web app that creates an image in the window, and then tries to load it from a ServiceWorker via shared CacheStorage. Here is what it looks like:

1. index.html

<!DOCTYPE html>
<title>Service worker demo</title>

<h1>cache-image.png</h1>
<!-- This png is going to get built with code, come from cache storage -->
<img src="/cache-image.png" width=16 height=16>

<script src="app.js"></script>

2. app.js

var CACHE = "caches-experiment";

// register service worker
if ('serviceWorker' in navigator) {
  navigator.serviceWorker.register('/sw.js', { scope: '/' })
}

function buildResponse() {
  // From http://stackoverflow.com/questions/14967647/ (continues on next line)
  // encode-decode-image-with-base64-breaks-image (2013-04-21)
  function fixBinary (bin) {
    var length = bin.length;
    var buf = new ArrayBuffer(length);
    var arr = new Uint8Array(buf);
    for (var i = 0; i < length; i++) {
      arr[i] = bin.charCodeAt(i);
    }
    return buf;
  }

  // 16x16 Mario png
  var marioBase64 = 
    "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAB1klEQVR42n2TzytEURTHv3e8N1joRhZG" + 
    "zJsoCjsLhcw0jClKWbHwY2GnLGUlIfIP2IjyY2djZTHSMJNQSilFNkz24z0/Ms2MrnvfvMu8mcfZvPvu" + 
    "Pfdzz/mecwgKLNYKb0cFEgXbRvwV2s2HuWazCbzKA5LvNecDXayBjv9NL7tEpSNgbYzQ5kZmAlSXgsGG" + 
    "XmS+MjhKxDHgC+quyaPKQtoPYMQPOh5U9H6tBxF+Icy/aolqAqLP5wjWd5r/Ip3YXVILrF4ZRYAxDhCO" + 
    "J/yCwiMI+/xgjOEzmzIhAio04GeGayIXjQ0wGoAuQ5cmIjh8jNo0GF78QwNhpyvV1O9tdxSSR6PLl51F" + 
    "nIK3uQ4JJQME4sCxCIRxQbMwPNSjqaobsfskm9l4Ky6jvCzWEnDKU1ayQPe5BbN64vYJ2vwO7CIeLIi3" + 
    "ciYAoby0M4oNYBrXgdgAbC/MhGCRhyhCZwrcEz1Ib3KKO7f+2I4iFvoVmIxHigGiZHhPIb0bL1bQApFS" + 
    "9U/AC0ulSXrrhMotka/lQy0Ic08FDeIiAmDvA2HX01W05TopS2j2/H4T6FBVbj4YgV5+AecyLk+Ctvms" + 
    "QWK8WZZ+Hdf7QGu7fobMuZHyq1DoJLvUqQrfM966EU/qYGwAAAAASUVORK5CYII=";

  var marioBinary = fixBinary(atob(marioBase64));
  var marioBlob = new Blob([marioBinary], {type: "image/png"});
  
  var init = {
    "status": 200,
    "statusText": "FromCaches"
  };
  return new Response(marioBlob, init);
}

function buildRequest() {
  var headers = new Headers();
  headers.append("Content-Type", "image/png");

  var init = {
    method: "GET",
    headers: headers
  }

  return new Request('/cache-image.png', init);
}

function writeCache() {
  caches.open(CACHE).then(function(cache) {
    var request = buildRequest();

    cache.match(request).then(function(response) {
      if (response) {
        console.log(' Found response in cache:', response);
      } else {
        console.log(' No response in cache, adding');
        cache.put(request, buildResponse());
      }
    }).catch(function(error) {
      // Handles exceptions that arise from match() or fetch().
      console.error('  Error in fetch handler:', error);
      throw error;
    });
  })
}

window.onload = writeCache;

3. sw.js

var CACHE = "caches-experiment";

this.addEventListener('fetch', function(event) {
  event.respondWith(
    caches.match(event.request)
    .then(function(response) {
      // Either we have a cached response, or we need to go to the network
      return response || fetch(event.request);
    })
    .catch(function(error) {
	console.error("Error: ", error);
    })
  );
});

And here is what it looks like when it runs:

I think it's possible for us to rework our Blob URL cache code to use cache storage and a simple service worker!

[humphd]

Essentially, what this means is that a page can create its own requests/responses, which is exactly what we need. Here's a more technical diagram of how it works:

[ryanwarsaw]

[humphd]

I was able to successfully implement a prototype of what I described above, and it's working:

I ran into an unexpected issue, where I was serving the live dev preview in the iframe as a Blob URL, and apparently browsers don't yet do the right thing with respect to inheriting the service worker controller of the parent in all iframe situations. It meant that my cached resources were loading as 404s, when they are really there, and load fine if I just enter the URL.

Now that I know this can work, I'll start to slowly rework our code to integrate this secondary path for the live dev server. I will have to keep what we have now for browsers without SW support, so it won't be as easy as doing what I want. However, we're close!

[humphd]

This is fixed.