http://kinotochka.net websockets

[TheFinalCut83]

URL(s) where the issue occurs

http://kinotochka.net/3552-cherepashki-nindzya-2-2016.html

Describe the issue

Exactly the same situation as in #120.

Screenshot(s)

Link with ads in red frame

Versions

• Browser/version: Cyberfox 48.0.2, Firefox 49.0b7
• uBlock Origin version: 1.9.4

Settings

default settings

Filter lists:
uBlock filters
uBlock filters – Badware risks‎
uBlock filters – Privacy‎
uBlock filters – Unbreak‎
Adblock Warning Removal List
Anti-Adblock Killer | Reek
EasyList
EasyPrivacy‎
Malvertising filter list by Disconnect‎
RUS: Adguard Russian Filter
RUS: BitBlock List
RUS: RU AdList

Notes

Candidate for inclusion in *$websocket,domain=extratorrent.cc|free-torrents.org|natureworldnews.com|opensubtitles.org|parentherald.com|pornhub.com|redtube.com|redtube.com.br|tomshardware.com|tube8.com|tube8.es|tube8.fr|xtube.com|youporn.com|youporngay.com

But this is becoming a frightening trend - 2nd website with missing websockets for a 2 days and the number of websockets missed that way could spontaneously increase.
RU AdList is already got fix for subject (in the same way as for free-torrents.org 2 days ago).
Cyberfox with only ABP - all is fine because of |ws://*.kinotochka.net^
Cyberfox with only Adguard addon - all is fine because of |ws://*.kinotochka.net^
uBO really needs ABP syntax for websockets (because uBO just can't deal with ABP websocket syntax at the moment) until it will become a huge problem for all uBO users under Gecko-browsers.

[gorhill]

Unable to reproduce. The websocket request is being blocked by RUAdlist's .kinotochka.net^$~object-subrequest,domain=kinotochka.net in uBO 1.9.4:

[gorhill]

Your RUAdlist was updated before you submitted this issue: https://hg.adblockplus.org/ruadlist/rev/14d75c6c8b7e. Be sure you use the most up to date lists before submitting an issue.

However, I still need to add *$websocket,domain=kinotochka.net not for your specific issue, but for Chromium-based browsers.

[TheFinalCut83]

I cleared all the caches right before the investigation of this issue. Unfortunately, filter lists in uBO don't update instantly. Still, conclusion of "Notes" section remains actual - similar issues in the near future can occur more often (this naturally coincided with the update of ABP in Chrome with websockets support). And can i clarify - are you always investigating issues in Chromium?

[dimisa-RUAdList]

I deliberately introduced a rule in membership due to the fact that using a standard ABP-syntax uBO not block anything. TheFinalCut83 was initially aware of this problem and this commit. I debited from it. Soon, these resources will be very much. We are talking about support for ABP-syntax.

I will not enter in the future, these patches.

[gorhill]

conclusion of "Notes" section remains actual

Except that it is incorrect to claim that ABP blocked the websocket connection because of |ws://*.kinotochka.net^: that filter does not work -- maybe it used to but it does not work with the current state of the site. It's quite probably why RUAdList was updated with the new filter.

So you tested with ABP and Adguard with an up to date RUAdList. You tested with uBO without an up to date RUAdList.

[gorhill]

I deliberately introduced a rule in membership due to the fact that using a standard ABP-syntax uBO not block anything.

Blocking the fetching of a resource should always be the first choice rather than hiding it after the fact. Are you making the point that using [-abp-properties="..."] was your solution of choice rather than outright blocking the network request fetching the resource?

I do not consider all of ABP's choice regarding filter syntax to be religious doctrine. I stick to pragmatism.

[lainverse]

Gorhill, I think you misunderstood a few things.

dimisa-RUAdList is maintainer of RU Adlist. Same as me. He added that |ws://*.kinotochka.net^ rule and it works just fine with Adblock Plus.
Here is an example: http://pasteboard.co/f24zUJHJf.png

He had to replace it with .kinotochka.net^$~object-subrequest,domain=kinotochka.net later since first one, even though it's a completely valid ABP filter didn't work in uBO.

In fact he has to resort to |ws://*.kinotochka.net^ syntax because ||kinotochka.net^$other blocks access to site in uBO for some reason even though it has to be $document instead of $other. No?

Also, he didn't mentioned [-abp-properties="..."] anywhere here even once. We do need it in some cases, but this is not the one.
Offtopic: we need it because we can't use :has() or other your's extensions since they treated as valid CSS in ABP right now and inserted into a page in one block with the rest of the filters. This breaks all ABP's hiding filters on a page. Well, it's ABP's problem as well, I know, but that's how it is.

Right now your extension breaking compatibility and introducing new problems:

1. It blocks access to site if something blocked with $other there. It shouldn't. If you want to treat $document without @@ before filter this way it's fine - it's a new functionality in comparison with ABP, but you absolutely must not make $other work like this. Yes, I do understand it should be websocket type in ABP, but it is other there right now and most likely it will remain like this for a while. It's not the reason to treat other like this, though.
2. It doesn't work with valid ABP filter |ws://*.kinotochka.net^ even though this filter works fine in ABP as I shown above.

[gorhill]

@lainverse

I looked a bit more afterward. Here is what is happening.

I had assumed @dimisa-RUAdList was referring to [-abp-properties="..."] when he said "about support for ABP-syntax" as this has been a litigious point with him in the past. Now you are right there is no [-abp-properties="..."] involved.

The issue which affected uBO on FF was not because of "[e]xactly the same situation as in #120" as stated by @TheFinalCut83.

The issue was the site adding display: block !important on the element targeted by the cosmetic filter kinotochka.net###\36 00. uBO does not make use of user stylesheets on Firefox (I may start to look into this), so it is more sensitive to sites adding inline display style to target elements. ABP suffers exactly the same problem on Chromium-based browser, so this is not a ABP-syntax support issue.

The issue does not occur on Chromium-based browsers because of the filter |ws://*.kinotochka.net^, which I gather was created to address the issue strictly for the display: block !important issue on Chromium-based browsers: without the websocket filter, ABP suffers exactly the same fate as uBO on Firefox.

The websocket request is not blocked on Firefox+ABP (an upgradeable http: request is used for the websocket connection -- it matches no filter), the click bait ads are just hidden by the filter kinotochka.net###\36 00. Easy to verify with @@||kinotochka.net^$element-hide. See:

^{This network request does not appear in ABP's blockable item list}

The best solution for all browsers of course is to block websocket requests for that site, so that the click-bait ads are not downloaded in the first place, and not inserted into the DOM. This can be done for Firefox as well with:

||etg.kinotochka.net^

This would work for both Firefox and Chromium, and for both uBO and ABP. (Or do they change the etg part regularly and it's why the * in |ws://*.kinotochka.net^?)

[gorhill]

In fact he has to resort to |ws://*.kinotochka.net^ syntax because ||kinotochka.net^$other blocks access to site in uBO for some reason even though it has to be $document instead of $other. No?

other should not trigger strict blocking, I consider this a bug. I was not aware of this and even surprised, thanks for bringing this to my attention. I suspect it's an old code path I forgot to remove after I started to use document instead of other.

[dimisa-RUAdList]

I spent 8 months on the adaptation RU AdList under the uBO. Rule .kinotochka.net^$~object-subrequest,domain=kinotochka.net not the only one added specifically because of uBO, such rules hundreds, closer to thousands. I need not so much - to work the syntax of ABP. This applies to ws://, wss://(for Firefox) and abp-properties. And when I found that option other locks the resource, I was very surprised. We can cooperate, if no, then let it be your decision.

[gorhill]

when I found that option other locks the resource, I was very surprised.

Me too, this is a bug which is fixed in 1.9.5b1. When I moved to use document instead of other for strict blocking, I needed a transition period before I stop using other to ensure strict blocking still worked for those with older filter lists. I just forgot to come back to finish the fix after the transition period. I will try to publish the fix asap given that now other is used a lot more than it used to.

Also, I added code to internally add websocket filter option every time other is encountered. It's unfortunate that ABP sees websocket requests as other, I think this is going to be an issue sooner than later, given that there are other useful things that also go in the other basket.

This applies to ws://, wss:// (for Firefox)

Just to be sure this is understood, the filter |ws://*.kinotochka.net^ does nothing on Firefox, websocket connections are not blocked for that site on Firefox (Nightly) with ABP (dev build). Also, I was mistaken when I said ||etg.kinotochka.net^ would block websocket connections for Firefox/ABP, for some reasons it does not work.

This needs to be investigated by ABP devs. The only thing preventing the click-bait ads from being visible with Firefox/ABP is element hiding, not websocket blocking (use @@||kinotochka.net^$elemhide and see that the click bait ads are there). I checked in the past that Firefox/ABP websocket connections could be blocked fine, so something is going on in this page preventing this from happening. ||etg.kinotochka.net^ does block websocket connections on Firefox with uBO 1.9.5b1.

[lainverse]

I've reported lack of 'websocket' type into ABP's issue tracker: https://issues.adblockplus.org/ticket/4387
Feel free to add more details there.

[lainverse]

Hi Gorhill,

Are you using mutli-threaded version of Firefox? I'm using 50.0a2 and it looks like ABP doesn't see any WS connections there at all!
However, it does see them in the normal Fx version. Here is an example from free-torrents.org:
https://drive.google.com/open?id=0BzbKPhiCU6YUcjBKRXRkVXVyWHp2TlZtZ29sQmlOSU83WDc0
Here is the same site in Fx 50.0a2:
https://drive.google.com/file/d/0BzbKPhiCU6YURWhsdDZSdDhHQmRVTXVsRENtU0pyZ1B2OUdn/view?usp=sharing
As you can see they appear as upgraded there and doesn't appear in blockable items at all.

So, Dimisa tested |ws://*.kinotochka.net^ filter with normal non-multithreaded Fx and it worked there.

[gorhill]

Yes, I was using Nightly -- I didn't realize FF stable would be different.

[gorhill]

Dev build 1.9.5b2 is on AMO, it solves many of the issues raised here:

• other will no longer trigger strict blocking
• when encountering filter option other, uBO will internally also apply (or unapply for negated other) the filter to websocket requests
• cosmetic filtering make use of user style sheet (this means this ad is now always properly hidden with Firefox).

With 1.9.5b2, there should be no special filters needed, except for the other known differences in which case people can open issues here when a uBO-specific filters is needed.

I tested the page in OP's comment with Firefox stable + 1.9.5b2 with default settings minus uBlock filters + RUAdList, and all seems ok -- except that the video does not play, but there appear to be a blocking filter in RUAdList preventing the video from loading when using non-Flash player: .kinotochka.net^$~object-subrequest,domain=kinotochka.net (probably should be .kinotochka.net^$~media,~object-subrequest,domain=kinotochka.net?). If I bypass the block filter for media resources from cdn6.kinotochka.net, the video played fine.

[dimisa-RUAdList]

Ok, I added an exception media to the rule: https://hg.adblockplus.org/ruadlist/rev/476dd6c1f3fc

By the way, this option media does not detect all the browsers. In any case, this is a temporary solution. When can we expect ws://, wss:// (for Firefox)?

[mapx-]

you have to add also "other" for chromium browsers
.kinotochka.net^$~media,~other,~object-subrequest,domain=kinotochka.net

[dimisa-RUAdList]

What for? You can more details?

[mapx-]

$media for chrome browsers is seen like $other
for example see easylist examples:
@@||vidible.tv/stage/$media,object,other

[dimisa-RUAdList]

How does this relate to the kinotochka.net?

[mapx-]

You excluded $media, right ? to be able to see the video in firefox browser.
The same you have to do for chromium browsers ==> using $other keyword.

[dimisa-RUAdList]

By default, it is not necessary. It is necessary only if the hand-off Flash plugin (chrome://plugins/).

[gorhill]

When can we expect ws://, wss:// (for Firefox)?

uBO can handle ws:///wss://, it does for Chrome with companion extension.

On Firefox, uBO is only given something like http://etg.kinotochka.net:8040/304/websocket for that page.

If I bypass the filter .kinotochka.net^$~object-subrequest,domain=kinotochka.net to not block the above websocket request, the Network tab in the dev console does not show anything else, there is no ws://... in it.

I tried with Firefox/ABP, and there was a ws://... in its blockable items list, but I do not see either a ws://... in the browser's Network tab.

So it seems a lot of the confusion here stems from this -- I see no ws://... in the browser's Network pane.

Now I am wondering, is ABP internally remapping the URL http://etg.kinotochka.net:8040/304 into ws://etg.kinotochka.net:8040/304?

I will investigate.

[mapx-]

in firefox 49, ABP, I can see:
ws://etg.kinotochka.net:8040/304

in firefox developer edition 50.0a2 (2016-08-31) not anymore as Lain said above.

W.Palant filed an issue on bugzilla:
https://bugzilla.mozilla.org/show_bug.cgi?id=1299766

[dimisa-RUAdList]

Ok,
https://hg.adblockplus.org/ruadlist/rev/eb9d1a250c3b#l1.13

[gorhill]

I think I see what is going on.

A shouldLoad observer sees a websocket request as ws://....

A HTTP obsever sees a websocket request as http://....

uBO works mostly as HTTP observer level, while ABP works mostly as shouldLoad level.

I will try to see what can be done on uBO's side so that filter list maintainers do not have to worry about these implementation-specific details.

[mapx-]

@gorhill, did you read #123 (comment)
?

[gorhill]

@mapx- Ok, so that helps making the decision of where I should add a special handling path for websockets in FF/uBO: this will be in its HTTP handler, so that it won't be affected by that issue.

[gorhill]

1.9.5b3 is on AMO: http(s):-based websocket requests are mapped to ws(s):-based ones.

Workaround filters such as .kinotochka.net^$~object-subrequest,domain=kinotochka.net should no longer be needed. For the current case there is the uBO-specific filter in uBlock filters for those without the fixes.