Ukrainian filter #2692
Comments
|
When I import AdGuard Ukrainian and search for A long time ago it was decided by AdGuard and RuAd List that uBO should stick to only one RU list because mixing both was causing issues. I am assuming this is still the case, so having potentially both AdGuard Ukraine used along RuAd List would cause issues as per past decision. Maybe @dimisa-RUAdList can provide more insights. I need more input before making a decision. |
|
The goal of making the Russian and Ukrainian filters independent of each other seems to me a non-trivial task. There will be a large number of intersections and repeating rules. For several centuries Russia and Ukraine were one country. During this time, a huge number of cultural, family and business ties have been formed. At the moment, only a little less than half of my relatives are residents and citizens of Ukraine, while the other part and I myself are residents and citizens of Russia. And my case is by no means unique. To make the current interpenetration insignificant, the efforts of more than one generation will be required. And I hope that never happens. As I can see from Adblock Plus reports statistics, Ukrainians visit Russian sites, and Russians visit Ukrainian sites. Ukrainian resources continue to cooperate with Russian advertising networks, despite the current conflict. I may be wrong, but I believe the guys from AdGuard created a project of a separate Ukrainian filter in order to get the possibility of more flexible manipulation in the conditions of MV3 restrictions. In any case, we do not plan to debug for possible conflicts, and do not even discuss it. |
Indeed, quite interesting. I suppose it's not surprising, seeing as AdGuard is a russian company. Perhaps, we'll see a proper Ukrainian filter some day only when it will be done by Ukrainians for Ukrainians... |
https://github.com/ukrainianfilters/lists You can join in and make your contribution. Maybe in a few years something will come of it. The main problem is feedback. In RU AdList, I can report through Adblock Plus. |
This is clearly a political and pro-russian, imperial position. Not a technical issue that could be resolved. Our freedom and independence is very important to us.
|
|
Do not bring politics or geopolitics here, that is not how decision are made here. More of this and I will lock the discussion.
You have this option: use
It was created only ~4 months ago, we need a list to have been maintained for a much longer time to qualify as a candidate for stock lists. The bottom line is that volunteers here are the ones dealing with filter issues, and some AdGuard lists have interfered negatively in the past when enabled along RUAdList. If I were to add AdGuard Ukraine as stock list, it would be enabled by default for Whether we should re-consider adding AdGuard Ukraine to stock lists, I will leave the decision to filter list maintainers, I am not involved enough with filter lists issues to decide this on my own. |
|
Hello, can you help resolve this issue? I still see a lot of Screen
|
|
The Ukrainian Filters collection has been created and maintained since 2023. The Ukrainian Security Filter was the first to be created, and work on the rest of the filters began in February 2024. So I don't quite understand where the information that “It was created only ~4 months ago” came from. @gorhill, what prevents from adding our filters, which are created by a team that lives in Ukraine and understands what the ukrnet is, unlike the developers of the above filters? |
https://github.com/ukrainianfilters/lists/graphs/commit-activity
|
|
There is also the old repository name https://github.com/serhiyguryev/ukrainian-filters/graphs/commit-activity with 6 months of developement:
which in total gives 10 months from the both repositories. |
@gorhill, I have provided web links that prove that the filter collection started in 2023. These links are also available on the project page, so it's easy to check. What you are showing is the project being moved to a separate repository. |
Sure, you clarified that after I posted my findings.
The screenshot I posted was to clarify where the information came from when I wrote my comment. Now regarding https://raw.githubusercontent.com/ukrainianfilters/lists/main/ads/ads.txt:
If I decide to include the Ukraine list as a stock list, I will remove |
I don't know where does 107 come from, as it shows 429: screenshoot.
By the way, instances counted in uBO - Asset viewer are lines/filters, and not occurences (domains), and since a line/filter can contain unknown/random number of
But another factor is how many domains are alive and how many are dead/parked in both filter lists. Someone would have to check it out. By using this tool for example: https://github.com/funilrys/PyFunceble (but it can only find dead domains, and not parked domains (this feature is still not implemented)). |
@gorhill If you translated the first 4 paragraphs of the description on the project page, you would not only learn about the date the filter collection was created (your initial findings are not relevant), but you would also learn that we are not chasing quantity, but at the same time we make sure that our filters work as efficiently as possible on popular Ukrainian sites. And we are doing a good job with this task (not to mention that we are the only team that develops 4 types of filters for Ukrainian users: Ad Filter, Annoyance Filter, Privacy Filter, Security Filter). As a user, I don't care how many domains RUAdList has, it's important for me not to see ads. 1,721 domains? And what's the point if I see ads on popular Ukrainian websites? And I'm not talking here about a bunch of outdated filters that RUAdList has, because hardly anyone conducts regular audits of such a large list. As a user from Ukraine, I don't understand why RUAdList contains rules that purposefully block banners of Ukrainian charities that help the Armed Forces of Ukraine (in particular, the Come Back Alive Charity Foundation).
I also don't understand why users from Poland have a choice between two filters, while users from Ukraine, with only two filters made specifically for Ukraine, don't have the opportunity to choose the one that suits them. As a filter developer, I don't see any clear and objective criteria that would guarantee that a filter will be added to the list of regional filters. If the criterion is to remove ads from Ukrainian websites, our list meets it. If the criterion is no conflicts with other filters set by default, our list meets it. However, I don't want to escalate the negative further, because this thread is created to find an effective solution to a specific problem. So let me offer my vision. For the version based on Manifest V2, we can do the following. Take the list of filters developed for uBlock Origin from FilterLists as a basis and use it as a data source so that users can find all available filters for a particular country directly in the uBO interface in the search field (this option should replace the existing “curated” list of regional filters). If this is the first installation of uBO, launch the Regional Filters Wizard, which allows users to select the required regional filters. Thus, users will be able to choose what they need directly in the uBO interface. Our team is ready to create a prototype of this solution and submit it for discussion (we need to work on the interface part and on compiling the necessary data from FilterLists). Unfortunately, this option won't work for Manifest V3, because, as far as I understand, lists need to be added manually when developing the extension itself. So we need to come up with something else. |
Duplicate of #2717 (comment). After the first install, uBO enables the appropriate regional list based on the browser language.
Can you at least address the unoptimized filters like those mentioned above by gorhill for now? |
Lines/filters, not domains. Domains count is 2827, like already mentioned in the table.
These are two different POL lists. One is for ads, the another for anti-adblocks. In the past, both of them were in the same file, as a single POL list, in uBlock too, but later, due to lawsuits from website owners, against the owner of the POL list, due to the removal of anti-adblock mechanisms on web sites, the owner was forced to split the list into two separate and independent lists and to move the anti-adblock list into another / separate repository under the care of another person (who is Ukrainian by the way), that's why there are 2 separate POL filter lists. But worth to mention, that there also exist more pairs of regional filters in uBO, the following threads contain informations:
Custom filters are possible in MV3 (for example AdGuard), but it's out of uBOL philosophy scope: |
|
@serhiyguryev Which exact URL should be used for a UKR list?
I try to resist as much as possible adding multiple lists for each region, but this has proven very difficult in some cases, so there are exceptions to this, and I still wish I could avoid this. |
If I understand correctly, you are referring to our collection of filters? If so, the URLs for the filters are as follows. GitHub repository: Ukrainian Ad Filter: Ukrainian Annoyance Filter: Ukrainian Privacy Filter: Ukrainian Security Filter: |
|
As my many years of experience tell me, compiling and maintaining filter lists is not as simple a task as it may seem at first glance. There are global dependencies that cannot be ignored. The basis for wide coverage and correct operation is primarily feedback, which depends, among other things, on the distribution of the filter. If the applied rules have a specific syntax characteristic of a certain extension, this will make it impossible or ineffective to use it in extensions that support only the classic syntax. If we take the AdGuard Ukrainian filter as an example, we will find more than 8 thousand rules in it. This is a very fresh filter, it was created by Ukrainians for Ukrainians in March 2023, and feedback from AdGuard users was used in its compilation. Interpenetration with the AdGuard Russian filter is about 70% and is explained by the fact that the specifics of geolocation for the Runet and Ukrnet involve the use of the same masks, ad servers and layouts. And don't forget that in addition to ua domains, we have many resources located on com, net, org, info, online, etc. The RU AdList filter exists in two versions, with classic syntax and one specific to uBO. Thanks to this, I receive feedback from a large number of users of various blockers, both large ones, like Adblock Plus, AdBlock, uBO, and smaller ones, including those built into browsers. Only from Adblock Plus users I receive from 10 to 15 thousand reports monthly, and in percentage terms, feedback from Ukrainians currently amounts to about 17%. Not all of them are useful, but in the end this is the case when quantity turns into quality. Therefore, the coverage of the RU AdList filter is larger, and if you try to select a separate Ukrainian list from it, the number of necessary rules will most likely exceed 11 thousand. Thus, we see how two different teams (AdGuard and RU AdList) with many years of experience and mass feedback use, or intend to use, much more than ~800 rules from Ukrainian Ad Filter. Therefore, it seems doubtful that such a filter will be more effective for users. As for blocking advertising of third-party funds, it is carried out in accordance with EasyList Policy. There is no difference between a charitable foundation or a support fund if the foundation's banner is placed on a third-party site. All such banners have been blocked by the RU AdList filter for the past 9 years and this will not change. |
@MasterKia On the one hand, as a developer, I understand this approach. However, I use exclusively English-language interfaces (operating system, browser, and other software), so this approach won't work in such cases. |
|
I will add only one list, so if you want to can create a I looked at |
||disqus.com^$third-partyYeah, this will break all disqus comments in non-Ukrainian sites too. We are the ones who will receive the reported issues from users. And when regional filter list authors have different policies, we will have to override the filters ourselves, causing more work for other volunteers. |
Then why does this list contain 1700+ rules for .ru / .рф domains, 60+ for .by domains, 120+ for .kz domains?
For Russians, there may be no difference, but for our team there is. Therefore, when creating rules, we make special (rules) exceptions for banner campaigns of Ukrainian charitable foundations. Moreover, in the case I mentioned, we are talking about the targeted hiding of banners of Ukrainian charitable foundations.
Then why don't yours and AdGuard's filters remove ads on popular Ukrainian websites (I've given examples)? |
No problem, we can remove this rule so that it does not conflict with other filters |
|
Lots of unnecessary wildcards in ://*.*/bi/*1704*$document://*.*/common/php/send-order.php?partnerid=*&lendid=*&offer=*$document,xmlhttprequest
://*.*/user.php?mod=do&act=register&fromto=*$document,xmlhttprequest
://*.*/user.php?mod=do&act=login&fromto=*$document,xmlhttprequest
://*.*/user.php?mod=do&act=login$document,xmlhttprequest
://*.*/user.php?mod=do&act=register$document,xmlhttprequest
://*.*/ok.php$document,domain=shop|store
://*loto*.*/comission.php$document
://*loto*.*/gametip*.php$document
://*loto*.*/connect.php$document
://*loto*.*/operator.php$document
://*loto*.*/payout.php$document
://*loto*.*/pay.php?price=*&s=*$document
://*loto*.*/pay/?am=*&who=*&sub=*$document
://*.*/*0registrations.html$document
://*.*/*0checkingbase.html$document
://*.*/price.php?price=*&ss=1$document
://*.*/pay/?aa=*&who=1&sub=*$document
://*.*/gwapi.php?a=pay&p=*&aa=*$document |
Through commenting systems built into Ukrainian popular websites, links to phishing sites that steal personal data and access to online bank account management systems are being massively distributed. If this is the only criticism of this filter, we can move these rules to a more specialized filter - Ukrainian Security Filter |
The motivation for creating such rules is disclosed in the filter description. After Russia's full-scale invasion of Ukraine, the number of phishing campaigns has grown exponentially. Many of these campaigns are organized and managed from the territory of Russia. And this is a huge problem that can be compared to an epidemic but in the digital dimension, as Ukrainians lose their last money by falling for phishing websites. Every day, dozens of new domain names are registered, which will be used in new phishing campaigns and online fraud. In such circumstances, blocking malicious web resources by domain name alone is not effective. That is why the Ukrainian Security Filter contains three groups of filtering rules:
What you have written about is pattern-based filtering rules that proactively block the phishing patterns we have detected (at the level of domain names, at the level of html page code, and at the level of network requests). If you can come up with a more sophisticated approach that will help us fight dozens of new phishing campaigns every day and hundreds of new domain names used in them, I'd love to read about it. |
|
If you use more than Just omit the |
I mean, it's just an example. My point is, we are very hesitant to consider regional lists that could potentially block connections used by other international sites. We already have EasyList and EasyPrivacy for these types of filters. How the lists are written is up to lists' authors, but we need to assure our own policies to balance usability and performance for users in general. Otherwise, it's hard for us to consider using that list, unless there are no other choices. |
|
If I'm not wrong, more than 1 wildcard is already making the filters to become regex. |
There are many cases where this is impossible due to the specific patterns used to form phishing domain names. So we make everything as optimal as possible. |
|
Right, these filters: To: And any filters in the form: Can be better rewritten as: And filters such as: Can be written as: Filters such as: Can just written as: Because the No filters should cause widespread breakage outside their region, so having a filter such as But there are other concerns, it's not just the quantity in RUAdList, it's also the quality of the filters, and additionally there are a lot of filters in RUAdList which uses advanced filtering capabilities such as scriptlet injections, procedural cosmetic filtering, etc. Bottomline is that RUAdList is very mature filter list from maintainers with a lot of experience in crafting optimal filters -- this is what we expect to make it as a stock list in uBO. |
Just a single wildcard which can't be ignored will cause a regex to be used internally. Though it's not bad if a good token can be extracted. |
|
@gorhill Okay, does this particular list meet the necessary requirements? Ukrainian Ad Filter: If so, can we add only this filter to start with? |
Related discussion: uBlockOrigin/uBlock-issues#2692
|
Added to dev version of stock lists for the time being. |
I thought you would add it to regional filters and not replace RU AdList. It will not be possible to completely replace it so quickly, and after removing the Ukraine region, in stable version of uBO/uBOL users will lose support from RU AdList. It is better not to hurry. |
I do not plan to change stable releases for a while, and anyway people can still enable RUAdList, it's not being removed from stock lists. We have this statement in our README:
To be honest I hate to deal with such issue of people arguing endlessly that their list should be a stock lists, we have gone through this many times, hence why the request to not submit lists. In the end I made the change here against what I think is best given the tone of the thread, so I gave in because I don't have the time and motivation to discuss this forever. If in the end users have issues with the UKR list, best is to discuss it with the issue tracker of the list maintainers, not here. |
|
@gorhill we have taken the previous comments into account and created a combined filter list (ads + privacy + security) - Ukrainian Filters (uBlock Origin Combined List) that we will continue to work with. If you have no objections to this list, please add it instead of ads.txt: Thank you in advance |
As per feedback: uBlockOrigin/uBlock-issues#2692 (comment)
Prerequisites
I tried to reproduce the issue when...
Description
So, to be short, now we have Ukrainian specific filter, NOT a bloatpile that is ru adlist. The article: https://adguard.com/en/blog/ukrainian-filter.html
Raw link for uBlock filter: https://filters.adtidy.org/extension/ublock/filters/23.txt
Also there's Ukrainian Malicious URL Blocklist: https://github.com/braveinnovators/url-blocklist which could certainly be useful for a lot of people at these trying times, perhaps it could be added under a dropdown like polish filters.
URL(s) where the issue occurs.
Screenshot(s)
No response
Notes
No response
Configuration
The text was updated successfully, but these errors were encountered: