Service worker with navigation preload causes blocking of navigation requests

[steffenweber]

Prerequisites

• I verified that this is not a filter issue
  • Filter issues MUST be reported at filter issue tracker
• This is not a support issue or a question
  • Support issues and questions are handled at /r/uBlockOrigin
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
  • Your issue may already be reported.
• I tried to reproduce the issue when...
  • uBlock Origin is the only extension
  • uBlock Origin with default lists/settings
  • using a new, unmodified browser profile
• I am running the latest version of uBlock Origin
• I checked the documentation to understand that the issue I report is not a normal behavior

Description

If a website has a service worker that uses Navigation Preload then uBlock Origin thinks that such navigation preload requests are subresource requests and blocks them if they match a filter. I'm pretty sure that this is the equivalent of the uMatrix issue #156.

A specific URL where the issue occurs

https://www.computerbase.de/forum/threads/google-analytics-und-dsgvo.1874439/

Steps to Reproduce

1. Open https://www.computerbase.de/ to initialize the service worker
2. Install uBlock Origin 1.19.6 or 1.19.7rc0 (don't do this before step 1, otherwise the service worker initialization currently fails, I'll fix that in a few days)
3. Open https://www.computerbase.de/forum/threads/google-analytics-und-dsgvo.1874439/

Expected behavior:

The page should load fine.

Actual behavior:

The page load is blocked because uBlock origin thinks that this navigation is a subresource request and because an EasyPrivacy filter coincidentally matches the "google-analytics" part of the URL.

Screencast

https://www.youtube.com/watch?v=pdDTjVT9y20

Your environment

• uBlock Origin version: 1.19.6 or 1.19.7rc0
• Browser Name and version: Chrome 74, 75, or 76
• Operating System and version: Gentoo Linux

[uBlock-user]

because an EasyPrivacy filter coincidentally matches the "google-analytics" part of the URL.

That's the only reason. I cannot reproduce this issue.

[uBlock-user]

FYI, that filter doesn't exist in Easy Privacy, after installing uBlock Origin, force update your filters to get rid of outdated third-party filters which come with the extension package.

[steffenweber]

@uBlock-user Could it be that you cannot reproduce this issue because you've disabled navigation preload? To quote you from uMatrix issue #156: "As per chrome://serviceworker-internals, navigation preload enabled is set to false, this is why it seems I can't reproduce." @gorhill was then able to reproduce the issue.

The filter rule cannot be the sole issue because the problem goes away if you skip step 1 (such that there is no service worker and no navigation preload).

Furthermore, I'm not the only one experiencing this issue. It was originally reported by one of our users (https://www.computerbase.de/forum/threads/login-probleme-offline-meldungen-seit-14-mai-2019.1872153/post-22701821) and I successfully reproduced it (using a fresh Chrome profile) to submit this bug report.

[uBlock-user]

@steffenweber Did you force update your filters yet ?

Could it be that you cannot reproduce this issue because you've disabled navigation preload?

It's disabled by default, see https://github.com/gorhill/uBlock/wiki/Dashboard:-Settings#disable-pre-fetching

It was originally reported by one of our users (https://www.computerbase.de/forum/threads/login-probleme-offline-meldungen-seit-14-mai-2019.1872153/post-22701821)

Nowhere there I see the user reporting it for uBlock Origin.

But I have now recognized the culprit: uBlock Origin seems to be, if I disable the adblocker, the page opens properly.

Disabling the adblocker is a terrible way to recognise uBO as the culprit, that doesn't prove anything at all. Investigation starts with the logger.

[steffenweber]

It's disabled by default, see https://github.com/gorhill/uBlock/wiki/Dashboard:-Settings#disable-pre-fetching

Afaik, prefetching/preloading is about <link rel="preload" …> (and maybe <link rel="next" …>) but not about service worker navigation preload (although both have "preload" in their name). uBlock doesn't seem to disable service worker navigation preload because as soon as you visit computerbase.de, the chrome://serviceworker-internals/ line Navigation preload enabled switches from false to true for me. Furthermore, I'm not using any non-default uBlock settings but a clean Chrome profile as you can see in the screencast I provided with my initial bug report.

Nowhere there I see the user reporting it for uBlock Origin.

As you've meanwhile noticed, our user has identified uBlock Origin as the problem in his second post.

Disabling the adblocker is a terrible way to recognise uBO as the culprit, that doesn't prove anything at all. Investigation starts with the logger.

I agree that it would have been useful if he had provided the logs himself but I think it doesn't change much if I can provide the logs that you are asking for.

Here is another screencast which shows that force updating the filters doesn't change anything: https://www.youtube.com/watch?v=9t2D0CEs-2Y

Please try to reproduce the issue by following exactly what I do in the video.

[uBlock-user]

I can reproduce your issue with the below STR.

1. Disable uBlock Origin and unregister all service workers from chrome://serviceworker-internals/

2. Browse to https://www.computerbase.de/

3. Now in the same tab browse to https://chrome.google.com/webstore/detail/ublock-origin/cjpalhdlnbpafiamejdnhcphjbkeiagm?hl=en

4. Webstore will notify you to enable uBlock Origin since we disabled it in step 1. Enable it. Then open the extension popup and open the logger, set it to All.

5. Now in the same webstore tab, go to https://www.computerbase.de/forum/threads/google-analytics-und-dsgvo.1874439/

6. Notice the offline page appearing and now the logger tab which will report /google-analytics-$other in Easy Privacy filter blocking a network request of type other originating from the page chrome.google.com.

[uBlock-user]

Now add /google-analytics-$other,badfilter in My Filters and repeat all the steps mentioned in #608 (comment) and the page opens successfully when the filter is not applied - https://i.gyazo.com/f6858d3541450afcc6c48b6c45614a91.png

[uBlock-user]

Here is another screencast that which shows that force updating the filters doesn't change anything: https://www.youtube.com/watch?v=9t2D0CEs-2Y

You're correct, that will not help. Add /google-analytics-$other,badfilter to get rid of that filter. and the page will open, which suggests which I thought previously, it's a filterlist issue.

[uBlock-user]

Easy Privacy has this filter /google-analytics-$~image which uBO interprets as /google-analytics-$other, because ~image means block everything except images and other satisfies that condition.

[steffenweber]

Thank you for the explanation why /google-analytics-$~image triggers this issue.

However, similar to uMatrix Issue #156, the root cause is that the navigation preload request has the type other (instead of main_frame). This is a Chrome bug that will eventually be fixed (see Chrome Issue #965960) but maybe uBlock could apply the same workaround that @gorhill added to uMatrix 1.3.17rc2 until then?

[uBlock-user]

the root cause is that the navigation preload request has the type other (instead of main_frame).

That's a whole another issue and not relevant to this bug because here we have a terrible filter added in Easy Privacy by Easylist team. Secondly, this issue is not similar to uBlockOrigin/uMatrix-issues#156 except for the fact you're seeing the Offline page which is really mis-leading in this case.

but maybe uBlock could apply the same workaround that @gorhill added to uMatrix 1.3.17rc2 until then?

There is no preload navigation being blocked here, this a site breakage issue caused by a badfilter which is not surprising given it happens every now and then -- https://github.com/uBlockOrigin/uBlock-issues/issues?q=is%3Aissue+is%3Aclosed+label%3Afilterlist

You should appeal to Easylist team and have /google-analytics-$~image removed or modified enough to not cause such breakage again.

[steffenweber]

I'm sorry, I meant to say uMatrix Issue #155 (not 156).

uMatrix Issue #155 is very similar to this issue here because both bugs are triggered by the type other instead of main_frame. Neither this issue nor the uMatrix issue would exist if Chrome supplied the correct type main_frame to extensions. uMatrix got a workaround and my hope is that uBlock can apply a similar workaround.

[uBlock-user]

Neither this issue nor the uMatrix issue would exist

This issue will happen regardless, because main_frame = document and $document != $image which will satisfy the condition required for /google-analytics-$~image to be applied.

I meant to say uMatrix Issue #155

uBlock doesn't see/deal with cookies at all, this is why I suggest you not compare uMatrix with uBO, they're similar, not the same.

uMatrix got a workaround and my hope is that uBlock can apply a similar workaround.

Why would there be a workaround when there's no issue with uBO itself ? This is something that doesn't make sense to me. You keep insisting that the issue is with uBO, while I proved here that there is no issue. Site breakage is not a uBlock issue and never has been.

[steffenweber]

Why doesn't this issue exist if there is no service worker (i.e. skip step 1 in the steps to reproduce)? My guess is that uBlock special-cases main_frame / doc requests.

The issue occurs as soon as there is a service worker with navigation preload (which causes the request type to be other).

Edit: If you add a filter computerbase.de$~image then you can still load computerbase.de (without any CSS, JS). Only sub-resource requests are blocked. So $~image filters don't seem to ever block user navigations. Unless you have a service worker with navigation preload which causes navigations to lose their special-casing (I guess because type then is other instead of main_frame).

[gorhill]

From uBO's perspective, it's first a filter issue. Once the filter issue is solved, there is a chance it become a uBO issue because of uBlockOrigin/uMatrix-issues#156, for example if someone blocks all 3rd-party in dynamic filtering pane. Just as with uBlockOrigin/uMatrix-issues#156, in the end it's a browser issue, because the context given to uBO by the browser is incorrect, chrome.google.com instead of www.computerbase.de.

So the filter issue can be fixed, but further issues arising from uBlockOrigin/uMatrix-issues#156 can't be fixed.

[uBlock-user]

The issue occurs as soon as there is a service worker with navigation preload

because of a bad filter match.

Why doesn't this issue exist if there is no service worker (i.e. skip step 1 in the steps to reproduce)?

if it was a uBO issue it would still happen without service worker. Service worker only triggers the Chromium issue, nothing else.

In the end, If uBO's doing anything, it's only triggering an underlying browser issue at best, that doesn't make it a uBO issue automatically.

If you add a filter computerbase.de$~image then you can still load computerbase.de

The site is broken same as before, ignore the Offline msg which is misleading as I told you before, it's site breakage,

[gwarser]