diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index 339cd43d82578..c4d49ea84c756 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -1,25 +1,26 @@ -> Filter issues **MUST NOT** be reported here. Read first: + ### Describe the issue ### One or more specific URLs where the issue occurs -> [URL(s) for issue on a specific site are **mandatory**] + ### Screenshot in which the issue can be seen -> [Screenshot(s) for visual issues are **mandatory**] + ### Steps for anyone to reproduce the issue -1. Be as detailed as possible: -1. Because we are not looking over your shoulder and -1. Keep in mind: Nobody can read your mind. + ### Your settings -> [If you fail to provide this info, I will mark the issue as invalid. Lists all settings which differs from default settings] + - MacOS version: - Safari version: @@ -27,6 +28,6 @@ ##### Your filter lists -> [Example: "Default filter lists + FRA", or "Default filter lists minus uBlock -- Badware risks"] + ##### Your custom filters (if any) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9ba475634144b..0fb8fcd58fe3f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,83 +1,20 @@ # Submitting issues -From now on, I will be quite less inclined to deal with issues for which little to no investigation to find the **actual cause** of a purported issue was made by the reporter. Positive contributions are those which are reported with efforts to find the actual cause of an issue, or at the very least efforts were made to narrow it as much as possible. This project is a hobby, do not expect "customer support"-style interaction. Requiring people to investigate as much as possible before opening an issue will more than likely avoid burdening the project with [invalid issues](https://github.com/gorhill/uBlock/issues?q=is%3Aissue+is%3Aclosed+label%3Ainvalid) or [issues unrelated](https://github.com/gorhill/uBlock/issues?q=is%3Aissue+is%3Aclosed+label%3Aexternal) to uBO. +Despite all the guidelines and warnings I added over the years about how to use it, the GitHub issue tracker always end up being used as a [help desk](https://en.wikipedia.org/wiki/Help_desk) rather than a place to report actual, real and proven issues. -For **support/discussions/help**, there is [/r/uBlockOrigin](https://www.reddit.com/r/uBlockOrigin/) on Reddit -- this is where I see the most activity for people helping each other regarding uBlock Origin. +Typically help desk are best handled by enthusiast volunteers -- so hopefully this is what will happen. -For **filter-related issues**, report on the respective filter list support site, or at [uBlockOrigin/uAssets](https://github.com/uBlockOrigin/uAssets/issues). Use [the logger](https://github.com/gorhill/uBlock/wiki/The-logger) to diagnose/confirm filter-related issues. If something does not work properly with uBO enabled, the **first step** is to rule out filter-related issues. +Given this, from now on I will consider GitHub issue tracker as a help desk one for uBlock Origin. I unsubscribed from being notified from all issues being opened, I will visit once in a while as time and mood allow. -Ignorance of the above rules is no excuse: **Opening an issue for purpose of support or discussion, or opening a filter-related issue will result in the user being immediately blocked.** Given the [amount of invalid issues being opened](https://github.com/gorhill/uBlock/issues?q=is%3Aissue+label%3Ainvalid+is%3Aclosed), I have no choice but to resort to such a drastic measure. You will still be able to open filter list issues at [uBlockOrigin/uAssets](https://github.com/uBlockOrigin/uAssets/issues). +When I will be visiting the "help desk", I will focus **only** on issues which I have good reasons to believe to be valid issues with uBlock Origin **itself**. -**The issue tracker is for provable issues only:** You will have to make the case that the issue is really with uBlock Origin and not something else on your side. To make a case means to provide detailed steps so that anybody can reproduce the issue. Be sure to rule out that the issue is not caused by something specific on your side. Specifically, _speculated_ performance issues will be marked as invalid and closed if they do not come with **actual profiling data + analysis** supporting the claim. +Valid issues typically comes with minimal but detailed steps to reproduce (i.e. no guesswork whatsoever required for others to reproduce), and especially genuine efforts to investigate the cause of an issue **as far as possible** using all the browser tools (uBO is HTML/CSS/JS, the browser has all the tools to investigate all of this). These will have my attention, and I will address them. -**Any issue opened without effort to provide the required details for me (or anybody else) to reproduce the problem will be closed as _invalid_.** If you provide more details thereafter for me to reproduce the issue, I will reopen it if I can confirm there is indeed an issue with uBlock Origin. Example of detailed steps: - -> 1. browser version/ublock version -> 1. these settings, these filter lists, these custom filters. -> 1. do this. -> 1. open this exact URL. -> 1. do this. -> 1. observe this. -> 1. click this. -> 1. observe such and such issue -> 1. see screenshot -> 1. etc. - -The most important part when opening an issue: **detailed steps**. - -**Important:** I simply do not have the time to take care of filter-related issues, you will have to find help elsewhere for this. The mere need to have to respond to filter-related issues can quickly become a burden. Consider that writing code/doc occupies all my free time. Surely there are other people out there ready to help with filter-related issues, it does not have to be me. - -*** - -### Before you submit - -1. Submit **bugs/issues only**. - - Bugs occur, I will fix them. -1. _One specific_ issue per submission. -1. The logger is the tool of choice to use to help diagnose issues. -1. Do **NOT**: - - Submit pull requests. - - Submit design ideas. - - Submit feature requests. - - Submit "revolutionary ideas". - - Post comments like "+1" or "me too!" without providing new relevant info on the issue. - - Use issues as replacement for threads on a bulletin board. - - Any such issue will be closed without comment. - - Ask me to publish the latest version to AMO/Chrome store: In all likelihood it is already published, but pending review, something which is out of my control. -1. Make sure your issue [hasn't already been fixed in a recent release](https://github.com/gorhill/uBlock/releases). -1. Verify that the issue does **not** occur with uBlock disabled. -1. **Verify that the issue is not related to a 3rd-party filter lists.** - - Issues with 3rd-party filter lists are the responsibility of their respective maintainers. -1. Verify that the issue is not caused by another extension. -1. Do not submit issues which can be reproduced **only** on Chrome Canary or Firefox Nightly: these are not stable browser versions and in all likelihood, whatever issue is not within uBO. - - Report **only** if you can reproduce in an official stable release, or a beta release. +In case I am really needed for a given issue, you can mention me using `@gorhill` so that GitHub will notify me. *** -### What to include - -To help diagnose and fix the bug/issue, please always include the following in your report: - -* A clear list of steps to reproduce the problem - * **ALWAYS INCLUDE A SPECIFIC URL WHERE THE ISSUE OCCURS**, _even_ if "it happens everywhere". -* Symptoms of the issue - * Describe what you observe and consider broken behavior; this is what we'll be looking for after executing the steps - * Example: video doesn't start playing, page layout broken -* Include whatever relevant the logger reports. -* A screenshot or transcription of **any of uBlock's preferences that differ from the defaults** - * This includes a whitelisted website, enabled/disabled filter list, anything - * Please do include everything different from the defaults whether or not it seems relevant to your issue -* The version of uBlock Origin you're having the issue with; you can find this in [uBlock's popup UI](https://github.com/gorhill/uBlock/wiki/Quick-guide:-popup-user-interface) - * Example: `uBlock 1.10.0` -* The Safari version you're using - * Examples: `Safari 10.0.1`, `Safari TP 18` -* The MacOS version you're using - * Examples: `MacOS Sierra 10.12.1`, `OSX 10.10.5` -* A list of other extensions you have installed - * Tip: try disabling them and see if your issue still occurs - -Otherwise, we've noticed that a lot of **your** time (and the developers') gets thrown away on exchanging back and forth to get this information. - -*** +Good readings: -**Good read:** [How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html). +- [How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html) +- [Why I didn’t fix your bug](http://magnusmanske.de/wordpress/?p=518) diff --git a/assets/assets.json b/assets/assets.json index edbba5e5a1803..2a1e2633a844f 100644 --- a/assets/assets.json +++ b/assets/assets.json @@ -424,7 +424,7 @@ "off": true, "title": "IRN: Adblock-Iran", "lang": "fa", - "contentURL": "https://raw.githubusercontent.com/farrokhi/adblock-iran/master/filter.txt", + "contentURL": "https://cdn.rawgit.com/farrokhi/adblock-iran/master/filter.txt", "supportURL": "https://github.com/farrokhi/adblock-iran" }, "ISL-0": { @@ -526,6 +526,15 @@ "contentURL": "https://easylist-downloads.adblockplus.org/easylistdutch.txt", "supportURL": "https://forums.lanik.us/viewforum.php?f=100" }, + "NOR-0": { + "content": "filters", + "group": "regions", + "off": true, + "title": "NOR: Dandelion Sprouts norske filtre", + "lang": "nb", + "contentURL": "https://raw.githubusercontent.com/DandelionSprout/adfilt/master/ExperimentalNorwegianList.txt", + "supportURL": "https://github.com/DandelionSprout/adfilt" + }, "POL-0": { "content": "filters", "group": "regions", @@ -533,16 +542,20 @@ "title": "POL: polskie filtry do Adblocka i uBlocka", "lang": "pl", "contentURL": "https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-adblock-filters/adblock.txt", - "supportURL": "https://github.com/MajkiIT/polish-ads-filter/issues" + "supportURL": "https://github.com/MajkiIT/polish-ads-filter/issues", + "instructionURL": "https://github.com/MajkiIT/polish-ads-filter#polish-filters-for-adblock-ublock-origin--adguard" }, - "POL-1": { + "ROU-1": { "content": "filters", "group": "regions", "off": true, - "title": "POL: polskie filtry do uBlocka uzupelnienie", - "lang": "pl", - "contentURL": "https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-adblock-filters/adblock_ublock.txt", - "supportURL": "https://github.com/MajkiIT/polish-ads-filter/issues" + "title": "ROU: Romanian Ad (ROad) Block List Light", + "lang": "ro", + "contentURL": [ + "https://road.adblock.ro/lista.txt", + "https://raw.githubusercontent.com/tcptomato/ROad-Block/master/road-block-filters-light.txt" + ], + "supportURL": "https://github.com/tcptomato/ROad-Block" }, "RUS-0": { "content": "filters", diff --git a/dist/description/description-fa.txt b/dist/description/description-fa.txt index b204bd8b4c5d0..1fba6ef815c7b 100644 --- a/dist/description/description-fa.txt +++ b/dist/description/description-fa.txt @@ -2,11 +2,11 @@ بررسی تصویری از کارایی این محصول: https://github.com/gorhill/uBlock/wiki/uBlock-vs.-ABP:-efficiency-compared -کاربرد: دکمه ی پاور بزرگ در پنجره برای فعال یا غیر فعال کردن uBlock برای صفحه ی جاری است. فقط برای همین سایت اعمال میشود، دکمه ی پاوری برای تمام سایت ها نیست. +روش استفاده: دکمۀ قدرت بزرگ در پنجرۀ بالاپَر برای فعال یا غیرفعال کردن دائمی یوبلاک برای وب‌سایت فعلی می‌باشد. این فقط برای همین سایت اعمال میشود، این دکمه ی قدرتی برای تمام سایت ها نیست. *** -انعطاف پذیری آن بیشتر از "ad blocker" است: همچنین می تواند فیلتر ها را از هاست میزبان، بخواند و بسازد. +انعطاف پذیری آن بیشتر از "ad blocker" است: این یکی همچنین می تواند فیلتر‌هایی را از فایل‌های هاست‌های میزبان، خوانده و بسازد. بیرون از جعبه، این لیست فیلترها بارگذاری و اجرا میشوند: diff --git a/dist/description/description-ka.txt b/dist/description/description-ka.txt index 43af9b585c586..15d5189a15549 100644 --- a/dist/description/description-ka.txt +++ b/dist/description/description-ka.txt @@ -1,49 +1,49 @@ -An efficient blocker: easy on memory and CPU footprint, and yet can load and enforce thousands more filters than other popular blockers out there. +რეკლამების შედეგიანი შემზღუდავი: მეხსიერებისა და პროცესორის შემსუბუქებული მოხმარება, რეკლამების სხვა შემზღუდავებთან შედარებით, ათასობით მეტი ფილტრის გამოყენების პირობებშიც კი. -Illustrated overview of its efficiency: https://github.com/gorhill/uBlock/wiki/uBlock-vs.-ABP:-efficiency-compared +შედეგიანობის მიმოხილვა იხილეთ ბმულზე: https://github.com/gorhill/uBlock/wiki/uBlock-vs.-ABP:-efficiency-compared -Usage: The big power button in the popup is to permanently disable/enable uBlock for the current web site. It applies to the current web site only, it is not a global power button. +გამოყენება: ჩართვის დიდი ღილაკით, ჩამოშლილ მენიუში, შესაძლებელია uBlock-ის ჩართვა/გამორთვა მიმდინარე ვებსაიტზე. ეს ღილაკი მოქმედებს მხოლოდ არსებულ საიტზე და არ გამოიყენება ზოგადად ჩართვა/გამორთვისთვის. *** -Flexible, it's more than an "ad blocker": it can also read and create filters from hosts files. +მეტად მოქნილი, ეს არაა უბრალოდ „რეკლამების შემზღუდავი“: ასევე შესაძლებელია hosts ფაილის წაკითხვა და ფილტრების შექმნა. -Out of the box, these lists of filters are loaded and enforced: +გარდა ამისა, ნაგულისხმევად ჩართულია და გამოიყენება შემდეგი გასაფილტრი სიები: - EasyList -- Peter Lowe’s Ad server list +- Peter Lowe-ის სარეკლამო სერვერების სია - EasyPrivacy -- Malware domains +- მავნე დომენები -More lists are available for you to select if you wish: +ასევე, ხელმისაწვდომია სიები სურვილისამებრ შესარჩევად: -- Fanboy’s Enhanced Tracking List -- Dan Pollock’s hosts file -- hpHosts’s Ad and tracking servers +- Fanboy-ის გაუმჯობესებული სია თვალყურისმდევნებლების +- Dan Pollock-ის hosts ფაილი +- hpHosts-ის სარეკლამო და თვალყურისმდევნელი სერვერები - MVPS HOSTS - Spam404 -- And many others +- და კიდევ ბევრი -Of course, the more filters enabled, the higher the memory footprint. Yet, even after adding Fanboy's two extra lists, hpHosts’s Ad and tracking servers, uBlock still has a lower memory footprint than other very popular blockers out there. +რასაკვირველია, რაც უფრო მეტი ფილტრია ჩართული, მეხსიერების გამოყენება იზრდება. თუმცა, Fanboy-ის გაფართოებული სიების, hpHosts-ის სარეკლამო და თვალყურისმდევნელი სერვერების დამატების შემთხვევაშიც კი, uBlock მაინც ნაკლებ მეხსიერებას იყენებს, ვიდრე ყველა სხვა ცნობილი შემზღუდავი პროგრამები. -Also, be aware that selecting some of these extra lists may lead to higher likelihood of web site breakage -- especially those lists which are normally used as hosts file. +ამასთან, გაითვალისწინეთ, რომ ზოგიერთი დამატებითი სიის შერჩევის შედეგად, შესაძლოა ვებსაიტები არ გამოჩნდეს გამართულად -- განსაკუთრებით იმ სიების შემთხვევაში, რომელიც ჩვეულებრივ, hosts ფაილად გამოიყენება. *** -Without the preset lists of filters, this extension is nothing. So if ever you really do want to contribute something, think about the people working hard to maintain the filter lists you are using, which were made available to use by all for free. +წინასწარ შედგენილ სიებს, მნიშვნელოვანი ადგილი უჭირავს ამ გაფართოების შედეგიან მუშაობაში. ასე რომ, თუ ოდესმე გადაწყვეტთ ვინმესთვის შემოწირულობის გაღებას, იფიქრეთ იმ ადამიანებზე, რომლებიც თავდაუზოგავად შრომობენ იმ გასაფილტრი სიების მუდმივ განახლებაზე, რომლითაც სარგებლობთ და რომელიც ხელმისაწვდომია ყველასთვის უფასოდ. *** -Free. -Open source with public license (GPLv3) -For users by users. +უფასო. +ღია წყაროს მქონე საჯარო ლიცენზიით (GPLv3) +მომხმარებლების მიერ, მომხმარებლებისთვის. -Contributors @ Github: https://github.com/gorhill/uBlock/graphs/contributors -Contributors @ Crowdin: https://crowdin.net/project/ublock +წვლილის შემტანები @ Github: https://github.com/gorhill/uBlock/graphs/contributors +წვლილის შემტანები @ Crowdin: https://crowdin.net/project/ublock *** It's quite an early version, keep this in mind when you review. -Project change log: +ცვლილებების ჩამონათვალი: https://github.com/gorhill/uBlock/releases diff --git a/dist/description/description-kk.txt b/dist/description/description-kk.txt new file mode 100644 index 0000000000000..0407a694837a8 --- /dev/null +++ b/dist/description/description-kk.txt @@ -0,0 +1,49 @@ +An efficient blocker: easy on memory and CPU footprint, and yet can load and enforce thousands more filters than other popular blockers out there. + +Illustrated overview of its efficiency: https://github.com/gorhill/uBlock/wiki/uBlock-vs.-ABP:-efficiency-compared + +Usage: The big power button in the popup is to permanently disable/enable uBlock for the current web site. It applies to the current web site only, it is not a global power button. + +*** + +Flexible, it's more than an "ad blocker": it can also read and create filters from hosts files. + +Out of the box, these lists of filters are loaded and enforced: + +- EasyList +- Peter Lowe’s Ad server list +- EasyPrivacy +- Malware domains + +More lists are available for you to select if you wish: + +- Fanboy’s Enhanced Tracking List +- Dan Pollock’s hosts file +- hpHosts’s Ad and tracking servers +- MVPS HOSTS +- Spam404 +- And many others + +Of course, the more filters enabled, the higher the memory footprint. Yet, even after adding Fanboy's two extra lists, hpHosts’s Ad and tracking servers, uBlock still has a lower memory footprint than other very popular blockers out there. + +Also, be aware that selecting some of these extra lists may lead to higher likelihood of web site breakage -- especially those lists which are normally used as hosts file. + +*** + +Without the preset lists of filters, this extension is nothing. So if ever you really do want to contribute something, think about the people working hard to maintain the filter lists you are using, which were made available to use by all for free. + +*** + +Еркін. +Open source with public license (GPLv3) +For users by users. + +Contributors @ Github: https://github.com/gorhill/uBlock/graphs/contributors +Contributors @ Crowdin: https://crowdin.net/project/ublock + +*** + +It's quite an early version, keep this in mind when you review. + +Project change log: +https://github.com/gorhill/uBlock/releases diff --git a/dist/description/description-no.txt b/dist/description/description-nb.txt similarity index 100% rename from dist/description/description-no.txt rename to dist/description/description-nb.txt diff --git a/dist/firefox/publish-signed-beta.py b/dist/firefox/publish-signed-beta.py new file mode 100755 index 0000000000000..4d84b3bd8543a --- /dev/null +++ b/dist/firefox/publish-signed-beta.py @@ -0,0 +1,274 @@ +#!/usr/bin/env python3 + +import datetime +import json +import jwt +import os +import re +import requests +import shutil +import subprocess +import sys +import tempfile +import time +import zipfile + +from distutils.version import LooseVersion +from string import Template + +# - Download target (raw) uBlock0.webext.xpi from GitHub +# - This is referred to as "raw" package +# - This will fail if not a dev build +# - Modify raw package to make it self-hosted +# - This is referred to as "unsigned" package +# - Ask AMO to sign uBlock0.webext.xpi +# - Generate JWT to be used for communication with server +# - Upload unsigned package to AMO +# - Wait for a valid download URL for signed package +# - Download signed package as uBlock0.webext.signed.xpi +# - This is referred to as "signed" package +# - Upload uBlock0.webext.signed.xpi to GitHub +# - Remove uBlock0.webext.xpi from GitHub +# - Modify updates.json to point to new version +# - Commit changes to repo + +# Find path to project root +projdir = os.path.split(os.path.abspath(__file__))[0] +while not os.path.isdir(os.path.join(projdir, '.git')): + projdir = os.path.normpath(os.path.join(projdir, '..')) +# Check that found project root is valid +version_filepath = os.path.join(projdir, 'dist', 'version') +if not os.path.isfile(version_filepath): + print('Version file not found.') + exit(1) + +extension_id = 'uBlock0@raymondhill.net' +tmpdir = tempfile.TemporaryDirectory() +raw_xpi_filename = 'uBlock0.webext.xpi' +raw_xpi_filepath = os.path.join(tmpdir.name, raw_xpi_filename) +unsigned_xpi_filepath = os.path.join(tmpdir.name, 'uBlock0.webext.unsigned.xpi') +signed_xpi_filename = 'uBlock0.webext.signed.xpi' +signed_xpi_filepath = os.path.join(tmpdir.name, signed_xpi_filename) +github_owner = 'gorhill' +github_repo = 'uBlock' + +# We need a version string to work with +if len(sys.argv) >= 2 and sys.argv[1]: + version = sys.argv[1] +else: + version = input('Github release version: ') +version.strip() +if not re.search('^\d+\.\d+\.\d+(b|rc)\d+$', version): + print('Error: Invalid version string.') + exit(1) + +# GitHub API token +# TODO: support as environment variable? (see os.environ) +github_token = input("Github token: ").strip() +if len(github_token) == 0: + print('Error: invalid GitHub token') + exit(1) +github_auth = 'token ' + github_token + +# +# Get metadata from GitHub about the release +# + +# https://developer.github.com/v3/repos/releases/#get-a-single-release +print('Downloading release info from GitHub...') +release_info_url = 'https://api.github.com/repos/{0}/{1}/releases/tags/{2}'.format(github_owner, github_repo, version) +headers = { 'Authorization': github_auth, } +response = requests.get(release_info_url, headers=headers) +if response.status_code != 200: + print('Error: Release not found: {0}'.format(response.status_code)) + exit(1) +release_info = response.json() + +# +# Extract URL to raw package from metadata +# + +# Find url for uBlock0.webext.xpi +raw_xpi_url = '' +for asset in release_info['assets']: + if asset['name'] == signed_xpi_filename: + print('Error: Found existing signed self-hosted package.') + exit(1) + if asset['name'] == raw_xpi_filename: + raw_xpi_url = asset['url'] +if len(raw_xpi_url) == 0: + print('Error: Release asset URL not found') + exit(1) + +# +# Download raw package from GitHub +# + +# https://developer.github.com/v3/repos/releases/#get-a-single-release-asset +print('Downloading raw xpi package from GitHub...') +headers = { + 'Authorization': github_auth, + 'Accept': 'application/octet-stream', +} +response = requests.get(raw_xpi_url, headers=headers) +# Redirections are transparently handled: +# http://docs.python-requests.org/en/master/user/quickstart/#redirection-and-history +if response.status_code != 200: + print('Error: Downloading raw package failed -- server error {0}'.format(response.status_code)) + exit(1) +with open(raw_xpi_filepath, 'wb') as f: + f.write(response.content) +print('Downloaded raw package saved as {0}'.format(raw_xpi_filepath)) + +# +# Convert the package to a self-hosted one: add `update_url` to the manifest +# + +print('Converting raw xpi package into self-hosted xpi package...') +with zipfile.ZipFile(raw_xpi_filepath, 'r') as zipin: + with zipfile.ZipFile(unsigned_xpi_filepath, 'w') as zipout: + for item in zipin.infolist(): + data = zipin.read(item.filename) + if item.filename == 'manifest.json': + manifest = json.loads(bytes.decode(data)) + manifest['applications']['gecko']['update_url'] = 'https://raw.githubusercontent.com/{0}/{1}/master/dist/firefox/updates.json'.format(github_owner, github_repo) + data = json.dumps(manifest, indent=2, separators=(',', ': '), sort_keys=True).encode() + zipout.writestr(item, data) + +# +# Ask AMO to sign the self-hosted package +# - https://developer.mozilla.org/en-US/Add-ons/Distribution#Distributing_your_add-on +# - https://pyjwt.readthedocs.io/en/latest/usage.html +# - https://addons-server.readthedocs.io/en/latest/topics/api/auth.html +# - https://addons-server.readthedocs.io/en/latest/topics/api/signing.html +# + +print('Ask AMO to sign self-hosted xpi package...') +with open(unsigned_xpi_filepath, 'rb') as f: + # TODO: support use of env variables for key/secret? + amo_api_key = input("AMO API key: ").strip() + amo_secret = input("AMO API secret: ").strip() + amo_nonce = os.urandom(8).hex() + jwt_payload = { + 'iss': amo_api_key, + 'jti': amo_nonce, + 'iat': datetime.datetime.utcnow(), + 'exp': datetime.datetime.utcnow() + datetime.timedelta(seconds=180), + } + jwt_auth = 'JWT ' + jwt.encode(jwt_payload, amo_secret).decode() + headers = { 'Authorization': jwt_auth, } + data = { 'channel': 'unlisted' } + files = { 'upload': f, } + signing_url = 'https://addons.mozilla.org/api/v3/addons/{0}/versions/{1}/'.format(extension_id, version) + print('Submitting package to be signed...') + response = requests.put(signing_url, headers=headers, data=data, files=files) + if response.status_code != 202: + print('Error: Creating new version failed -- server error {0}'.format(response.status_code)) + print(response.text) + exit(1) + print('Request for signing self-hosted xpi package succeeded.') + signing_request_response = response.json(); + f.close() + print('Waiting for AMO to process the request to sign the self-hosted xpi package...') + # Wait for signed package to be ready + signing_check_url = signing_request_response['url'] + # TODO: use real time instead + countdown = 180 / 5 + while True: + sys.stdout.write('.') + sys.stdout.flush() + time.sleep(5) + countdown -= 1 + if countdown <= 0: + print('Error: AMO signing timed out') + exit(1) + response = requests.get(signing_check_url, headers=headers) + if response.status_code != 200: + print('Error: AMO signing failed -- server error {0}'.format(response.status_code)) + exit(1) + signing_check_response = response.json() + if not signing_check_response['processed']: + continue + if not signing_check_response['valid']: + print('Error: AMO validation failed') + exit(1) + if not signing_check_response['files'] or len(signing_check_response['files']) == 0: + continue + if not signing_check_response['files'][0]['signed']: + print('Error: AMO signing failed') + exit(1) + print('\r') + print('Self-hosted xpi package successfully signed.') + download_url = signing_check_response['files'][0]['download_url'] + print('Downloading signed self-hosted xpi package from {0}...'.format(download_url)) + response = requests.get(download_url, headers=headers) + if response.status_code != 200: + print('Error: Download signed package failed -- server error {0}'.format(response.status_code)) + exit(1) + with open(signed_xpi_filepath, 'wb') as f: + f.write(response.content) + f.close() + print('Signed self-hosted xpi package downloaded.') + break + +# +# Upload signed package to GitHub +# + +# https://developer.github.com/v3/repos/releases/#upload-a-release-asset +print('Uploading signed self-hosted xpi package to GitHub...') +with open(signed_xpi_filepath, 'rb') as f: + url = release_info['upload_url'].replace('{?name,label}', '?name=' + signed_xpi_filename) + headers = { + 'Authorization': github_auth, + 'Content-Type': 'application/zip', + } + response = requests.post(url, headers=headers, data=f.read()) + if response.status_code != 201: + print('Error: Upload signed package failed -- server error: {0}'.format(response.status_code)) + exit(1) + +# +# Remove raw package from GitHub +# + +# https://developer.github.com/v3/repos/releases/#delete-a-release-asset +print('Remove raw xpi package from GitHub...') +headers = { 'Authorization': github_auth, } +response = requests.delete(raw_xpi_url, headers=headers) +if response.status_code != 204: + print('Error: Deletion of raw package failed -- server error: {0}'.format(response.status_code)) + +# +# Update updates.json to point to new package -- but only if just-signed +# package is higher version than current one. +# + +print('Update GitHub to point to newly signed self-hosted xpi package...') +updates_json_filepath = os.path.join(projdir, 'dist', 'firefox', 'updates.json') +with open(updates_json_filepath) as f: + updates_json = json.load(f) + f.close() + previous_version = updates_json['addons'][extension_id]['updates'][0]['version'] + if LooseVersion(version) > LooseVersion(previous_version): + with open(os.path.join(projdir, 'platform', 'webext', 'updates.template.json')) as f: + template_json = Template(f.read()) + f.close() + updates_json = template_json.substitute(version=version) + with open(updates_json_filepath, 'w') as f: + f.write(updates_json) + f.close() + # Automatically git add/commit if needed. + # - Stage the changed file + r = subprocess.run(['git', 'status', '-s', updates_json_filepath], stdout=subprocess.PIPE) + rout = bytes.decode(r.stdout).strip() + if len(rout) >= 2 and rout[1] == 'M': + subprocess.run(['git', 'add', updates_json_filepath]) + # - Commit the staged file + r = subprocess.run(['git', 'status', '-s', updates_json_filepath], stdout=subprocess.PIPE) + rout = bytes.decode(r.stdout).strip() + if len(rout) >= 2 and rout[0] == 'M': + subprocess.run(['git', 'commit', '-m', 'make Firefox dev build auto-update', updates_json_filepath]) + subprocess.run(['git', 'push', 'origin', 'master']) + +print('All done.') diff --git a/dist/firefox/updates.json b/dist/firefox/updates.json new file mode 100644 index 0000000000000..8e98c2eecca4a --- /dev/null +++ b/dist/firefox/updates.json @@ -0,0 +1,14 @@ +{ + "addons": { + "uBlock0@raymondhill.net": { + "updates": [ + { + "version": "1.15.23b0", + "applications": { "gecko": { "strict_min_version": "52" } }, + "update_info_url": "https://github.com/gorhill/uBlock/releases/tag/1.15.23b0", + "update_link": "https://github.com/gorhill/uBlock/releases/download/1.15.23b0/uBlock0.webext.signed.xpi" + } + ] + } + } +} diff --git a/dist/version b/dist/version new file mode 100644 index 0000000000000..d2d226120f41d --- /dev/null +++ b/dist/version @@ -0,0 +1 @@ +1.15.24 diff --git a/doc/benchmarks/cpu-usage-overall-chart-20141226.png b/doc/benchmarks/cpu-usage-overall-chart-20141226.png index 148a663fa152c..79e7f279f9236 100644 Binary files a/doc/benchmarks/cpu-usage-overall-chart-20141226.png and b/doc/benchmarks/cpu-usage-overall-chart-20141226.png differ diff --git a/platform/chromium/manifest.json b/platform/chromium/manifest.json index 8c75debe78d3a..6cfbfcf94901a 100644 --- a/platform/chromium/manifest.json +++ b/platform/chromium/manifest.json @@ -2,7 +2,7 @@ "manifest_version": 2, "name": "uBlock Origin", - "version": "1.14.24", + "version": "1.15.11.0", "commands": { "launch-element-zapper": { @@ -70,5 +70,8 @@ "short_name": "uBlock₀", "storage": { "managed_schema": "managed_storage.json" - } + }, + "web_accessible_resources": [ + "/web_accessible_resources/*" + ] } diff --git a/platform/chromium/vapi-background.js b/platform/chromium/vapi-background.js index 21c3dbcfb1004..c0f07427f8493 100644 --- a/platform/chromium/vapi-background.js +++ b/platform/chromium/vapi-background.js @@ -265,18 +265,29 @@ vAPI.tabs = {}; /******************************************************************************/ +// https://github.com/gorhill/uBlock/issues/3546 +// Added a new flavor of behind-the-scene tab id: vAPI.anyTabId. +// vAPI.anyTabId will be used for network requests which can be filtered, +// because they comes with enough contextual information. It's just not +// possible to pinpoint exactly from which tab it comes from. For example, +// with Firefox/webext, the `documentUrl` property is available for every +// network requests. + vAPI.isBehindTheSceneTabId = function(tabId) { - return tabId.toString() === '-1'; + if ( typeof tabId === 'string' ) { debugger; } + return tabId < 0; }; -vAPI.noTabId = '-1'; +vAPI.unsetTabId = 0; +vAPI.noTabId = -1; // definitely not any existing tab +vAPI.anyTabId = -2; // one of the existing tab /******************************************************************************/ +// To remove when tabId-as-integer has been tested enough. + var toChromiumTabId = function(tabId) { - if ( typeof tabId === 'string' ) { - tabId = parseInt(tabId, 10); - } + if ( typeof tabId === 'string' ) { debugger; } if ( typeof tabId !== 'number' || isNaN(tabId) || tabId === -1 ) { return 0; } @@ -329,8 +340,8 @@ vAPI.tabs.registerListeners = function() { } if ( typeof vAPI.tabs.onPopupCreated === 'function' ) { vAPI.tabs.onPopupCreated( - details.tabId.toString(), - details.sourceTabId.toString() + details.tabId, + details.sourceTabId ); } }; @@ -364,7 +375,7 @@ vAPI.tabs.registerListeners = function() { if ( changeInfo.url ) { changeInfo.url = sanitizeURL(changeInfo.url); } - onUpdatedClient(tabId.toString(), changeInfo, tab); + onUpdatedClient(tabId, changeInfo, tab); }; chrome.webNavigation.onBeforeNavigate.addListener(onBeforeNavigate); @@ -542,9 +553,7 @@ vAPI.tabs.open = function(details) { vAPI.tabs.replace = function(tabId, url) { tabId = toChromiumTabId(tabId); - if ( tabId === 0 ) { - return; - } + if ( tabId === 0 ) { return; } var targetURL = url; @@ -565,9 +574,7 @@ vAPI.tabs.replace = function(tabId, url) { vAPI.tabs.remove = function(tabId) { tabId = toChromiumTabId(tabId); - if ( tabId === 0 ) { - return; - } + if ( tabId === 0 ) { return; } var onTabRemoved = function() { // https://code.google.com/p/chromium/issues/detail?id=410868#c8 @@ -605,9 +612,7 @@ vAPI.tabs.reload = function(tabId, bypassCache) { vAPI.tabs.select = function(tabId) { tabId = toChromiumTabId(tabId); - if ( tabId === 0 ) { - return; - } + if ( tabId === 0 ) { return; } chrome.tabs.update(tabId, { active: true }, function(tab) { if ( chrome.runtime.lastError ) { @@ -789,7 +794,7 @@ vAPI.messaging.onPortMessage = (function() { case 'connectionRefused': toPort = messaging.ports.get(msg.fromToken); if ( toPort !== undefined ) { - msg.tabId = tabId && tabId.toString(); + msg.tabId = tabId; toPort.postMessage(request); } else { msg.what = 'connectionBroken'; @@ -797,7 +802,7 @@ vAPI.messaging.onPortMessage = (function() { } break; case 'connectionRequested': - msg.tabId = tabId && tabId.toString(); + msg.tabId = tabId; for ( toPort of messaging.ports.values() ) { toPort.postMessage(request); } @@ -809,7 +814,7 @@ vAPI.messaging.onPortMessage = (function() { port.name === msg.fromToken ? msg.toToken : msg.fromToken ); if ( toPort !== undefined ) { - msg.tabId = tabId && tabId.toString(); + msg.tabId = tabId; toPort.postMessage(request); } else { msg.what = 'connectionBroken'; @@ -944,6 +949,38 @@ vAPI.messaging.broadcast = function(message) { /******************************************************************************/ /******************************************************************************/ +// https://github.com/gorhill/uBlock/issues/3474 +// https://github.com/gorhill/uBlock/issues/2823 +// - foil ability of web pages to identify uBO through +// its web accessible resources. +// https://github.com/gorhill/uBlock/issues/3497 +// - prevent web pages from interfering with uBO's element picker + +(function() { + vAPI.warSecret = + Math.floor(Math.random() * 982451653 + 982451653).toString(36) + + Math.floor(Math.random() * 982451653 + 982451653).toString(36); + + var key = 'secret=' + vAPI.warSecret; + var root = vAPI.getURL('/'); + var guard = function(details) { + if ( details.url.indexOf(key) === -1 ) { + return { redirectUrl: root }; + } + }; + + chrome.webRequest.onBeforeRequest.addListener( + guard, + { + urls: [ root + 'web_accessible_resources/*' ] + }, + [ 'blocking' ] + ); +})(); + +/******************************************************************************/ +/******************************************************************************/ + // https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/contextMenus#Browser_compatibility // Firefox for Android does no support browser.contextMenus. diff --git a/platform/chromium/vapi-webrequest.js b/platform/chromium/vapi-webrequest.js index 49a87e3e24411..d57f2e2fec6ea 100644 --- a/platform/chromium/vapi-webrequest.js +++ b/platform/chromium/vapi-webrequest.js @@ -1,7 +1,7 @@ /******************************************************************************* uBlock Origin - a browser extension to block requests. - Copyright (C) 2017 Raymond Hill + Copyright (C) 2017-2018 Raymond Hill This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -105,7 +105,15 @@ vAPI.net.registerListeners = function() { }; var normalizeRequestDetails = function(details) { - details.tabId = details.tabId.toString(); + // Chromium 63+ supports the `initiator` property, which contains + // the URL of the origin from which the network request was made. + if ( + details.tabId === vAPI.noTabId && + typeof details.initiator === 'string' + ) { + details.tabId = vAPI.anyTabId; + details.documentUrl = details.initiator; + } var type = details.type; @@ -161,54 +169,11 @@ vAPI.net.registerListeners = function() { } }; - // https://bugs.chromium.org/p/chromium/issues/detail?id=129353 - // https://github.com/gorhill/uBlock/issues/1497 - // Expose websocket-based network requests to uBO's filtering engine, - // logger, etc. - // Counterpart of following block of code is found in "vapi-client.js" -- - // search for "https://github.com/gorhill/uBlock/issues/1497". - // - // Once uBO 1.11.1 and uBO-Extra 2.12 are widespread, the image-based - // handling code can be removed. - var onBeforeWebsocketRequest = function(details) { - if ( (details.type !== 'image') && - (details.method !== 'HEAD' || details.type !== 'xmlhttprequest') - ) { - return; - } - var requestURL = details.url, - matches = /[?&]u(?:rl)?=([^&]+)/.exec(requestURL); - if ( matches === null ) { return; } - details.type = 'websocket'; - details.url = decodeURIComponent(matches[1]); - var r = onBeforeRequestClient(details); - if ( r && r.cancel ) { return r; } - // Redirect to the provided URL, or a 1x1 data: URI if none provided. - matches = /[?&]r=([^&]+)/.exec(requestURL); - return { - redirectUrl: matches !== null ? - decodeURIComponent(matches[1]) : - 'data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==' - }; - }; - var onBeforeRequestClient = this.onBeforeRequest.callback; - var onBeforeRequest = validTypes.websocket - // modern Chromium/WebExtensions: type 'websocket' is supported - ? function(details) { - normalizeRequestDetails(details); - return onBeforeRequestClient(details); - } - // legacy Chromium - : function(details) { - // https://github.com/gorhill/uBlock/issues/1497 - if ( details.url.endsWith('ubofix=f41665f3028c7fd10eecf573336216d3') ) { - var r = onBeforeWebsocketRequest(details); - if ( r !== undefined ) { return r; } - } - normalizeRequestDetails(details); - return onBeforeRequestClient(details); - }; + var onBeforeRequest = function(details) { + normalizeRequestDetails(details); + return onBeforeRequestClient(details); + }; // This is needed for Chromium 49-55. var onBeforeSendHeaders = validTypes.csp_report @@ -228,44 +193,16 @@ vAPI.net.registerListeners = function() { var onHeadersReceivedClient = this.onHeadersReceived.callback, onHeadersReceivedClientTypes = this.onHeadersReceived.types.slice(0), onHeadersReceivedTypes = denormalizeTypes(onHeadersReceivedClientTypes); - var onHeadersReceived = validTypes.font - // modern Chromium/WebExtensions: type 'font' is supported - ? function(details) { - normalizeRequestDetails(details); - if ( - onHeadersReceivedClientTypes.length !== 0 && - onHeadersReceivedClientTypes.indexOf(details.type) === -1 - ) { - return; - } - return onHeadersReceivedClient(details); + var onHeadersReceived = function(details) { + normalizeRequestDetails(details); + if ( + onHeadersReceivedClientTypes.length !== 0 && + onHeadersReceivedClientTypes.indexOf(details.type) === -1 + ) { + return; } - // legacy Chromium - : function(details) { - normalizeRequestDetails(details); - // Hack to work around Chromium API limitations, where requests of - // type `font` are returned as `other`. For example, our normalization - // fail at transposing `other` into `font` for URLs which are outside - // what is expected. At least when headers are received we can check - // for content type `font/*`. Blocking at onHeadersReceived time is - // less worse than not blocking at all. Also, due to Chromium bug, - // `other` always becomes `object` when it can't be normalized into - // something else. Test case for "unfriendly" font URLs: - // https://www.google.com/fonts - if ( details.type === 'font' ) { - var r = onBeforeRequestClient(details); - if ( typeof r === 'object' && r.cancel === true ) { - return { cancel: true }; - } - } - if ( - onHeadersReceivedClientTypes.length !== 0 && - onHeadersReceivedClientTypes.indexOf(details.type) === -1 - ) { - return; - } - return onHeadersReceivedClient(details); - }; + return onHeadersReceivedClient(details); + }; var urls, types; diff --git a/platform/firefox/install.rdf b/platform/firefox/install.rdf index e478a268a1b68..e9b39263a357f 100644 --- a/platform/firefox/install.rdf +++ b/platform/firefox/install.rdf @@ -47,7 +47,7 @@ {{8de7fcbb-c55c-4fbe-bfc5-fc555c87dbc4}} - 26.0 + 27.0 27.* diff --git a/platform/firefox/vapi-background.js b/platform/firefox/vapi-background.js index c3579826e72df..39b46065d2827 100644 --- a/platform/firefox/vapi-background.js +++ b/platform/firefox/vapi-background.js @@ -1,7 +1,7 @@ /******************************************************************************* uBlock Origin - a browser extension to block requests. - Copyright (C) 2014-2107 The uBlock Origin authors + Copyright (C) 2014-2018 The uBlock Origin authors This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -833,10 +833,11 @@ var getOwnerWindow = function(target) { /******************************************************************************/ vAPI.isBehindTheSceneTabId = function(tabId) { - return tabId.toString() === '-1'; + return tabId < 0; }; -vAPI.noTabId = '-1'; +vAPI.noTabId = -1; +vAPI.anyTabId = -2; /******************************************************************************/ @@ -1283,7 +1284,7 @@ var tabWatcher = (function() { } var tabId = browserToTabIdMap.get(browser); if ( tabId === undefined ) { - tabId = '' + tabIdGenerator++; + tabId = tabIdGenerator++; browserToTabIdMap.set(browser, tabId); tabIdToBrowserMap.set(tabId, Cu.getWeakReference(browser)); } @@ -1316,7 +1317,7 @@ var tabWatcher = (function() { var removeBrowserEntry = function(tabId, browser) { if ( tabId && tabId !== vAPI.noTabId ) { vAPI.tabs.onClosed(tabId); - delete vAPI.toolbarButton.tabs[tabId]; + vAPI.toolbarButton.tabs.delete(tabId); tabIdToBrowserMap.delete(tabId); } if ( browser ) { @@ -1539,7 +1540,7 @@ vAPI.setIcon = function(tabId, iconStatus, badge) { if ( tabId === undefined ) { tabId = curTabId; } else if ( badge !== undefined ) { - tb.tabs[tabId] = { badge: badge, img: iconStatus === 'on' }; + tb.tabs.set(tabId, { badge: badge, img: iconStatus === 'on' }); } if ( curTabId && tabId === curTabId ) { @@ -2028,7 +2029,7 @@ var httpObserver = { }, // https://github.com/gorhill/uBlock/issues/959 - syntheticPendingRequest: { frameId: 0, parentFrameId: -1, tabId: '', rawtype: 1 }, + syntheticPendingRequest: { frameId: 0, parentFrameId: -1, tabId: 0, rawtype: 1 }, handleRequest: function(channel, URI, details) { var type = this.typeMap[details.rawtype] || 'other'; @@ -2437,7 +2438,7 @@ vAPI.toolbarButton = { viewId: location.host + '-panel', label: vAPI.app.name, tooltiptext: vAPI.app.name, - tabs: {/*tabId: {badge: 0, img: boolean}*/}, + tabs: new Map(/* tabId: { badge: 0, img: boolean } */), init: null, codePath: '' }; @@ -2473,8 +2474,8 @@ vAPI.toolbarButton = { if ( tabId === undefined ) { return label; } - var tabDetails = this.tabs[tabId]; - if ( !tabDetails ) { + var tabDetails = this.tabs.get(tabId); + if ( tabDetails === undefined ) { return label; } if ( !tabDetails.img ) { @@ -2563,7 +2564,7 @@ vAPI.toolbarButton = { return; } - var icon = this.tabs[tabId]; + var icon = this.tabs.get(tabId); button.setAttribute('badge', icon && icon.badge || ''); button.classList.toggle('off', !icon || !icon.img); @@ -3502,9 +3503,8 @@ vAPI.onLoadAllCompleted = function() { // TODO: vAPI shouldn't know about uBlock. Just like in uMatrix, uBlock // should collect on its side all the opened tabs whenever it is ready. var µb = µBlock; - var tabId; for ( var browser of tabWatcher.browsers() ) { - tabId = tabWatcher.tabIdFromTarget(browser); + var tabId = tabWatcher.tabIdFromTarget(browser); µb.tabContextManager.commit(tabId, browser.currentURI.asciiSpec); µb.bindTabToPageStats(tabId); } diff --git a/platform/safari/Info.plist b/platform/safari/Info.plist index 069d8bd839c80..b6a76e7c0908f 100644 --- a/platform/safari/Info.plist +++ b/platform/safari/Info.plist @@ -3,9 +3,11 @@ Author - Chris Aljoudi/Raymond Hill + Raymond Hill/Ellis Tsung Builder Version 534.57.2 + DeveloperIdentifier + 3NU33NW2M3 CFBundleDisplayName {name} CFBundleIdentifier diff --git a/platform/safari/client-injected.js b/platform/safari/client-injected.js index 99b84f0ddcdf4..b1db19b4ce018 100644 --- a/platform/safari/client-injected.js +++ b/platform/safari/client-injected.js @@ -75,9 +75,9 @@ if ( window.Worker instanceof Function ) { return new RealWorker(window.URL.createObjectURL(new Blob([';'], {type:'text/javascript'}))); }; return new RealWorker(url); - }; + }.bind(window); WrappedWorker.prototype = RealWorker.prototype; - window.Worker = WrappedWorker.bind(window); + window.Worker = WrappedWorker; }; // __MSG_historyScript__ diff --git a/platform/safari/vapi-background.js b/platform/safari/vapi-background.js index ed798baaafce2..45fe78faa189c 100644 --- a/platform/safari/vapi-background.js +++ b/platform/safari/vapi-background.js @@ -81,155 +81,6 @@ safari.extension.settings.addEventListener('change', function(e) { } }, false); -/******************************************************************************/ - -initStorageLib(); // Initialize storage library - -/******************************************************************************/ - -var storageQuota = 104857600; // copied from Info.plist -localforage.config({ - name: 'ublock', - size: storageQuota, - storeName: 'keyvaluepairs' -}); - -vAPI.cacheStorage = { - QUOTA_BYTES: storageQuota, // copied from Info.plist - - get: function(keys, callback) { - if ( typeof callback !== 'function' ) { - return; - } - - var result = {}; - - if ( keys === null ) { - localforage.iterate(function(value, key) { - if ( typeof value === 'string' ) { - result[key] = JSON.parse(value); - } - }, function() { - callback(result); - }); - } else if ( typeof keys === 'string' ) { - localforage.getItem(keys, function(err, value) { - if ( typeof value === 'string' ) { - result[keys] = JSON.parse(value); - } - callback(result); - }); - } else if ( Array.isArray(keys) ) { - var toSatisfy = keys.length, n = toSatisfy; - if ( n === 0 ) { - callback(result); - return; - } - var key; - for ( var i = 0; i < n; i++ ) { - key = keys[i]; - localforage.getItem(key, (function(key) { - return function(err, value) { - toSatisfy--; - if ( typeof value === 'string' ) { - result[key] = JSON.parse(value); - } - if ( toSatisfy === 0 ) { - callback(result); - } - } - })(key)); - } - } else if ( typeof keys === 'object' ) { - for ( var key in keys ) { - if ( !keys.hasOwnProperty(key) ) { - continue; - } - result[key] = keys[key]; - } - localforage.iterate(function(value, key) { - if ( !keys.hasOwnProperty(key) ) return; - if ( typeof value === 'string' ) { - result[key] = JSON.parse(value); - } - }, function() { - callback(result); - }); - } - }, - - set: function(details, callback) { - var key, toSatisfy = 0; - for ( key in details ) { - if ( !details.hasOwnProperty(key) ) { - continue; - } - toSatisfy++; - } - if ( toSatisfy === 0 ) { - // Nothing to set - callback && callback(); - return; - } - var callbackCaller = function() { - if ( --toSatisfy === 0 ) { - callback && callback(); - } - }; - for ( key in details ) { - if ( !details.hasOwnProperty(key) ) { - continue; - } - localforage.setItem(key, JSON.stringify(details[key]), callbackCaller); - } - }, - - remove: function(keys) { - if ( typeof keys === 'string' ) { - keys = [keys]; - } - - for ( var i = 0, n = keys.length; i < n; i++ ) { - localforage.removeItem(keys[i]); - } - }, - - clear: function(callback) { - localforage.clear(function() { - typeof callback === 'function' && callback(); - }); - }, - - getBytesInUse: function(keys, callback) { - if ( typeof callback !== 'function' ) { - return; - } - var size = 0; - if ( Array.isArray(keys) ) { - var toSatisfy = keys.length, n = toSatisfy; - if ( n === 0 ) { - callback(0); - return; - } - var callbackCaller = function(err, value) { - size += (value || '').length; - if ( --toSatisfy === 0 ) { - callback(size); - } - }; - for ( var i = 0; i < n; i++ ) { - localforage.getItem(keys[i], callbackCaller); - } - } else { - localforage.iterate(function(value, key) { - size += (value || '').length; - }, function() { - callback(size); - }); - } - } -}; - vAPI.storage = { _storage: safari.extension.settings, get: function(keys, callback) { @@ -327,6 +178,7 @@ vAPI.tabs.registerListeners = function() { if ( !vAPI.tabs.popupCandidate || !e.target || e.url === 'about:blank' ) { return; } + var targetUrl = e.url, targetTabId = vAPI.tabs.getTabId(e.target).toString(), openerTabId = vAPI.tabs.popupCandidate; @@ -538,6 +390,7 @@ vAPI.tabs.injectScript = function(tabId, details, callback) { } if ( details.file ) { + details.file = vAPI.getURL(details.file) var xhr = new XMLHttpRequest(); xhr.open('GET', details.file, true); xhr.addEventListener('readystatechange', function() { @@ -650,6 +503,7 @@ safari.application.addEventListener('deactivate', function(event) { if ( !(event.target instanceof SafariBrowserTab) ) { return; } + // when a tab is deactivated... var tabId = vAPI.tabs.getTabId(event.target), state = iconStateForTabId[tabId]; @@ -986,106 +840,6 @@ vAPI.messaging.broadcast = function(message) { /******************************************************************************/ -vAPI.net = {}; - -/******************************************************************************/ - -// Fast `contains` - -Array.prototype.contains = function(a) { - var b = this.length; - while ( b-- ) { - if ( this[b] === a ) { - return true; - } - } - return false; -}; - -/******************************************************************************/ - -vAPI.net.registerListeners = function() { - var µb = µBlock; - - // Until Safari has more specific events, those are instead handled - // in the onBeforeRequestAdapter; clean them up so they're garbage-collected - vAPI.net.onBeforeSendHeaders = null; - - var onBeforeRequest = vAPI.net.onBeforeRequest, - onBeforeRequestClient = onBeforeRequest.callback, - onHeadersReceivedClient = vAPI.net.onHeadersReceived.callback; - - // https://github.com/el1t/uBlock-Safari/issues/32 - // Ignore directives - var shouldBlockResponseHeader = { - script: /script-src/, - worker: /child-src/ - }; - - var onBeforeRequestAdapter = function(e) { - if ( e.name !== 'canLoad' ) { - return; - } - e.stopPropagation && e.stopPropagation(); - switch ( e.message.type ) { - case 'main_frame': - vAPI.tabs.onNavigation({ - url: e.message.url, - frameId: 0, - tabId: vAPI.tabs.getTabId(e.target).toString() - }); - e.message.hostname = µb.URI.hostnameFromURI(e.message.url); - e.message.tabId = vAPI.tabs.getTabId(e.target); - e.message.responseHeaders = []; - onBeforeRequestClient(e.message); - var blockVerdict = onHeadersReceivedClient(e.message); - blockVerdict = blockVerdict && blockVerdict.responseHeaders && blockVerdict.responseHeaders[0] && - shouldBlockResponseHeader.script.test(blockVerdict.responseHeaders[0].value); - e.message = { - shouldBlock: blockVerdict === true - }; - return; - case 'popup': - var openerTabId = vAPI.tabs.getTabId(e.target).toString(); - var shouldBlock = !!vAPI.tabs.onPopupUpdated('preempt', openerTabId, e.message.url); - if ( !shouldBlock ) { - vAPI.tabs.popupCandidate = openerTabId; - } - e.message = { - shouldBlock: shouldBlock - }; - break; - case 'popstate': - // No return value/message - vAPI.tabs.onUpdated(vAPI.tabs.getTabId(e.target), { - url: e.message.url - }, { - url: e.message.url - }); - break; - case 'worker': - e.message.type = 'sub_frame'; - e.message.hostname = µb.URI.hostnameFromURI(e.message.url); - e.message.tabId = vAPI.tabs.getTabId(e.target); - e.message.responseHeaders = []; - var blockVerdict = onHeadersReceivedClient(e.message); - blockVerdict = blockVerdict && blockVerdict.responseHeaders && blockVerdict.responseHeaders[0] && - shouldBlockResponseHeader.worker.test(blockVerdict.responseHeaders[0].value); - e.message = { - shouldBlock: blockVerdict === true - } - return; - default: - e.message.hostname = µb.URI.hostnameFromURI(e.message.url); - e.message.tabId = vAPI.tabs.getTabId(e.target); - var blockVerdict = onBeforeRequestClient(e.message) || {}; - blockVerdict.shouldBlock = blockVerdict.cancel === true || blockVerdict.redirectUrl !== undefined; - e.message = blockVerdict; - return; - } - }; - safari.application.addEventListener('message', onBeforeRequestAdapter, true); -}; /******************************************************************************/ @@ -1214,17 +968,4 @@ vAPI.adminStorage = { } }; -/******************************************************************************/ -/******************************************************************************/ - -function initStorageLib() { - /*! - localForage -- Offline Storage, Improved - Version 1.4.3 - https://localforage.github.io/localForage - (c) 2013-2016 Mozilla, Apache License 2.0 - */ - !function(a){if("object"==typeof exports&&"undefined"!=typeof module)module.exports=a();else if("function"==typeof define&&define.amd)define([],a);else{var b;b="undefined"!=typeof window?window:"undefined"!=typeof global?global:"undefined"!=typeof self?self:this,b.localforage=a()}}(function(){return function a(b,c,d){function e(g,h){if(!c[g]){if(!b[g]){var i="function"==typeof require&&require;if(!h&&i)return i(g,!0);if(f)return f(g,!0);var j=new Error("Cannot find module '"+g+"'");throw j.code="MODULE_NOT_FOUND",j}var k=c[g]={exports:{}};b[g][0].call(k.exports,function(a){var c=b[g][1][a];return e(c?c:a)},k,k.exports,a,b,c,d)}return c[g].exports}for(var f="function"==typeof require&&require,g=0;g=43)}}).catch(function(){return!1})}function n(a){return"boolean"==typeof ga?ia.resolve(ga):m(a).then(function(a){return ga=a})}function o(a){var b=ha[a.name],c={};c.promise=new ia(function(a){c.resolve=a}),b.deferredOperations.push(c),b.dbReady?b.dbReady=b.dbReady.then(function(){return c.promise}):b.dbReady=c.promise}function p(a){var b=ha[a.name],c=b.deferredOperations.pop();c&&c.resolve()}function q(a,b){return new ia(function(c,d){if(a.db){if(!b)return c(a.db);o(a),a.db.close()}var e=[a.name];b&&e.push(a.version);var f=fa.open.apply(fa,e);b&&(f.onupgradeneeded=function(b){var c=f.result;try{c.createObjectStore(a.storeName),b.oldVersion<=1&&c.createObjectStore(ja)}catch(c){if("ConstraintError"!==c.name)throw c;console.warn('The database "'+a.name+'" has been upgraded from version '+b.oldVersion+" to version "+b.newVersion+', but the storage "'+a.storeName+'" already exists.')}}),f.onerror=function(){d(f.error)},f.onsuccess=function(){c(f.result),p(a)}})}function r(a){return q(a,!1)}function s(a){return q(a,!0)}function t(a,b){if(!a.db)return!0;var c=!a.db.objectStoreNames.contains(a.storeName),d=a.versiona.db.version;if(d&&(a.version!==b&&console.warn('The database "'+a.name+"\" can't be downgraded from version "+a.db.version+" to version "+a.version+"."),a.version=a.db.version),e||c){if(c){var f=a.db.version+1;f>a.version&&(a.version=f)}return!0}return!1}function u(a){return new ia(function(b,c){var d=new FileReader;d.onerror=c,d.onloadend=function(c){var d=btoa(c.target.result||"");b({__local_forage_encoded_blob:!0,data:d,type:a.type})},d.readAsBinaryString(a)})}function v(a){var b=l(atob(a.data));return i([b],{type:a.type})}function w(a){return a&&a.__local_forage_encoded_blob}function x(a){var b=this,c=b._initReady().then(function(){var a=ha[b._dbInfo.name];if(a&&a.dbReady)return a.dbReady});return k(c,a,a),c}function y(a){function b(){return ia.resolve()}var c=this,d={db:null};if(a)for(var e in a)d[e]=a[e];ha||(ha={});var f=ha[d.name];f||(f={forages:[],db:null,dbReady:null,deferredOperations:[]},ha[d.name]=f),f.forages.push(c),c._initReady||(c._initReady=c.ready,c.ready=x);for(var g=[],h=0;h>4,k[i++]=(15&d)<<4|e>>2,k[i++]=(3&e)<<6|63&f;return j}function I(a){var b,c=new Uint8Array(a),d="";for(b=0;b>2],d+=ma[(3&c[b])<<4|c[b+1]>>4],d+=ma[(15&c[b+1])<<2|c[b+2]>>6],d+=ma[63&c[b+2]];return c.length%3===2?d=d.substring(0,d.length-1)+"=":c.length%3===1&&(d=d.substring(0,d.length-2)+"=="),d}function J(a,b){var c="";if(a&&(c=Da.call(a)),a&&("[object ArrayBuffer]"===c||a.buffer&&"[object ArrayBuffer]"===Da.call(a.buffer))){var d,e=pa;a instanceof ArrayBuffer?(d=a,e+=ra):(d=a.buffer,"[object Int8Array]"===c?e+=ta:"[object Uint8Array]"===c?e+=ua:"[object Uint8ClampedArray]"===c?e+=va:"[object Int16Array]"===c?e+=wa:"[object Uint16Array]"===c?e+=ya:"[object Int32Array]"===c?e+=xa:"[object Uint32Array]"===c?e+=za:"[object Float32Array]"===c?e+=Aa:"[object Float64Array]"===c?e+=Ba:b(new Error("Failed to get type for BinaryArray"))),b(e+I(d))}else if("[object Blob]"===c){var f=new FileReader;f.onload=function(){var c=na+a.type+"~"+I(this.result);b(pa+sa+c)},f.readAsArrayBuffer(a)}else try{b(JSON.stringify(a))}catch(c){console.error("Couldn't convert value into a JSON string: ",a),b(null,c)}}function K(a){if(a.substring(0,qa)!==pa)return JSON.parse(a);var b,c=a.substring(Ca),d=a.substring(qa,Ca);if(d===sa&&oa.test(c)){var e=c.match(oa);b=e[1],c=c.substring(e[0].length)}var f=H(c);switch(d){case ra:return f;case sa:return i([f],{type:b});case ta:return new Int8Array(f);case ua:return new Uint8Array(f);case va:return new Uint8ClampedArray(f);case wa:return new Int16Array(f);case ya:return new Uint16Array(f);case xa:return new Int32Array(f);case za:return new Uint32Array(f);case Aa:return new Float32Array(f);case Ba:return new Float64Array(f);default:throw new Error("Unkown type: "+d)}}function L(a){var b=this,c={db:null};if(a)for(var d in a)c[d]="string"!=typeof a[d]?a[d].toString():a[d];var e=new ia(function(a,d){try{c.db=openDatabase(c.name,String(c.version),c.description,c.size)}catch(a){return d(a)}c.db.transaction(function(e){e.executeSql("CREATE TABLE IF NOT EXISTS "+c.storeName+" (id INTEGER PRIMARY KEY, key unique, value)",[],function(){b._dbInfo=c,a()},function(a,b){d(b)})})});return c.serializer=Ea,e}function M(a,b){var c=this;"string"!=typeof a&&(console.warn(a+" used as a key, but it is not a string."),a=String(a));var d=new ia(function(b,d){c.ready().then(function(){var e=c._dbInfo;e.db.transaction(function(c){c.executeSql("SELECT * FROM "+e.storeName+" WHERE key = ? LIMIT 1",[a],function(a,c){var d=c.rows.length?c.rows.item(0).value:null;d&&(d=e.serializer.deserialize(d)),b(d)},function(a,b){d(b)})})}).catch(d)});return j(d,b),d}function N(a,b){var c=this,d=new ia(function(b,d){c.ready().then(function(){var e=c._dbInfo;e.db.transaction(function(c){c.executeSql("SELECT * FROM "+e.storeName,[],function(c,d){for(var f=d.rows,g=f.length,h=0;h=0;c--){var d=localStorage.key(c);0===d.indexOf(a)&&localStorage.removeItem(d)}});return j(c,a),c}function W(a,b){var c=this;"string"!=typeof a&&(console.warn(a+" used as a key, but it is not a string."),a=String(a));var d=c.ready().then(function(){var b=c._dbInfo,d=localStorage.getItem(b.keyPrefix+a);return d&&(d=b.serializer.deserialize(d)),d});return j(d,b),d}function X(a,b){var c=this,d=c.ready().then(function(){for(var b=c._dbInfo,d=b.keyPrefix,e=d.length,f=localStorage.length,g=1,h=0;h=43)}}).catch(function(){return!1})}function n(a){return"boolean"==typeof ga?ia.resolve(ga):m(a).then(function(a){return ga=a})}function o(a){var b=ha[a.name],c={};c.promise=new ia(function(a){c.resolve=a}),b.deferredOperations.push(c),b.dbReady?b.dbReady=b.dbReady.then(function(){return c.promise}):b.dbReady=c.promise}function p(a){var b=ha[a.name],c=b.deferredOperations.pop();c&&c.resolve()}function q(a,b){return new ia(function(c,d){if(a.db){if(!b)return c(a.db);o(a),a.db.close()}var e=[a.name];b&&e.push(a.version);var f=fa.open.apply(fa,e);b&&(f.onupgradeneeded=function(b){var c=f.result;try{c.createObjectStore(a.storeName),b.oldVersion<=1&&c.createObjectStore(ja)}catch(c){if("ConstraintError"!==c.name)throw c;console.warn('The database "'+a.name+'" has been upgraded from version '+b.oldVersion+" to version "+b.newVersion+', but the storage "'+a.storeName+'" already exists.')}}),f.onerror=function(){d(f.error)},f.onsuccess=function(){c(f.result),p(a)}})}function r(a){return q(a,!1)}function s(a){return q(a,!0)}function t(a,b){if(!a.db)return!0;var c=!a.db.objectStoreNames.contains(a.storeName),d=a.versiona.db.version;if(d&&(a.version!==b&&console.warn('The database "'+a.name+"\" can't be downgraded from version "+a.db.version+" to version "+a.version+"."),a.version=a.db.version),e||c){if(c){var f=a.db.version+1;f>a.version&&(a.version=f)}return!0}return!1}function u(a){return new ia(function(b,c){var d=new FileReader;d.onerror=c,d.onloadend=function(c){var d=btoa(c.target.result||"");b({__local_forage_encoded_blob:!0,data:d,type:a.type})},d.readAsBinaryString(a)})}function v(a){var b=l(atob(a.data));return i([b],{type:a.type})}function w(a){return a&&a.__local_forage_encoded_blob}function x(a){var b=this,c=b._initReady().then(function(){var a=ha[b._dbInfo.name];if(a&&a.dbReady)return a.dbReady});return k(c,a,a),c}function y(a){function b(){return ia.resolve()}var c=this,d={db:null};if(a)for(var e in a)d[e]=a[e];ha||(ha={});var f=ha[d.name];f||(f={forages:[],db:null,dbReady:null,deferredOperations:[]},ha[d.name]=f),f.forages.push(c),c._initReady||(c._initReady=c.ready,c.ready=x);for(var g=[],h=0;h>4,k[i++]=(15&d)<<4|e>>2,k[i++]=(3&e)<<6|63&f;return j}function I(a){var b,c=new Uint8Array(a),d="";for(b=0;b>2],d+=ma[(3&c[b])<<4|c[b+1]>>4],d+=ma[(15&c[b+1])<<2|c[b+2]>>6],d+=ma[63&c[b+2]];return c.length%3===2?d=d.substring(0,d.length-1)+"=":c.length%3===1&&(d=d.substring(0,d.length-2)+"=="),d}function J(a,b){var c="";if(a&&(c=Da.call(a)),a&&("[object ArrayBuffer]"===c||a.buffer&&"[object ArrayBuffer]"===Da.call(a.buffer))){var d,e=pa;a instanceof ArrayBuffer?(d=a,e+=ra):(d=a.buffer,"[object Int8Array]"===c?e+=ta:"[object Uint8Array]"===c?e+=ua:"[object Uint8ClampedArray]"===c?e+=va:"[object Int16Array]"===c?e+=wa:"[object Uint16Array]"===c?e+=ya:"[object Int32Array]"===c?e+=xa:"[object Uint32Array]"===c?e+=za:"[object Float32Array]"===c?e+=Aa:"[object Float64Array]"===c?e+=Ba:b(new Error("Failed to get type for BinaryArray"))),b(e+I(d))}else if("[object Blob]"===c){var f=new FileReader;f.onload=function(){var c=na+a.type+"~"+I(this.result);b(pa+sa+c)},f.readAsArrayBuffer(a)}else try{b(JSON.stringify(a))}catch(c){console.error("Couldn't convert value into a JSON string: ",a),b(null,c)}}function K(a){if(a.substring(0,qa)!==pa)return JSON.parse(a);var b,c=a.substring(Ca),d=a.substring(qa,Ca);if(d===sa&&oa.test(c)){var e=c.match(oa);b=e[1],c=c.substring(e[0].length)}var f=H(c);switch(d){case ra:return f;case sa:return i([f],{type:b});case ta:return new Int8Array(f);case ua:return new Uint8Array(f);case va:return new Uint8ClampedArray(f);case wa:return new Int16Array(f);case ya:return new Uint16Array(f);case xa:return new Int32Array(f);case za:return new Uint32Array(f);case Aa:return new Float32Array(f);case Ba:return new Float64Array(f);default:throw new Error("Unkown type: "+d)}}function L(a){var b=this,c={db:null};if(a)for(var d in a)c[d]="string"!=typeof a[d]?a[d].toString():a[d];var e=new ia(function(a,d){try{c.db=openDatabase(c.name,String(c.version),c.description,c.size)}catch(a){return d(a)}c.db.transaction(function(e){e.executeSql("CREATE TABLE IF NOT EXISTS "+c.storeName+" (id INTEGER PRIMARY KEY, key unique, value)",[],function(){b._dbInfo=c,a()},function(a,b){d(b)})})});return c.serializer=Ea,e}function M(a,b){var c=this;"string"!=typeof a&&(console.warn(a+" used as a key, but it is not a string."),a=String(a));var d=new ia(function(b,d){c.ready().then(function(){var e=c._dbInfo;e.db.transaction(function(c){c.executeSql("SELECT * FROM "+e.storeName+" WHERE key = ? LIMIT 1",[a],function(a,c){var d=c.rows.length?c.rows.item(0).value:null;d&&(d=e.serializer.deserialize(d)),b(d)},function(a,b){d(b)})})}).catch(d)});return j(d,b),d}function N(a,b){var c=this,d=new ia(function(b,d){c.ready().then(function(){var e=c._dbInfo;e.db.transaction(function(c){c.executeSql("SELECT * FROM "+e.storeName,[],function(c,d){for(var f=d.rows,g=f.length,h=0;h=0;c--){var d=localStorage.key(c);0===d.indexOf(a)&&localStorage.removeItem(d)}});return j(c,a),c}function W(a,b){var c=this;"string"!=typeof a&&(console.warn(a+" used as a key, but it is not a string."),a=String(a));var d=c.ready().then(function(){var b=c._dbInfo,d=localStorage.getItem(b.keyPrefix+a);return d&&(d=b.serializer.deserialize(d)),d});return j(d,b),d}function X(a,b){var c=this,d=c.ready().then(function(){for(var b=c._dbInfo,d=b.keyPrefix,e=d.length,f=localStorage.length,g=1,h=0;h>> 3; + let bufOut = new Uint8Array(sizeOut); + let iOut = 0; + let n = sizeIn & ~3; + while ( iIn < n ) { + let b0 = dec[bufIn.charCodeAt(iIn++)]; + let b1 = dec[bufIn.charCodeAt(iIn++)]; + let b2 = dec[bufIn.charCodeAt(iIn++)]; + let b3 = dec[bufIn.charCodeAt(iIn++)]; + bufOut[iOut++] = (b0 << 2) & 0xFC | (b1 >>> 4); + bufOut[iOut++] = (b1 << 4) & 0xF0 | (b2 >>> 2); + bufOut[iOut++] = (b2 << 6) & 0xC0 | b3; + } + if ( iIn !== sizeIn ) { + let b0 = dec[bufIn.charCodeAt(iIn++)]; + let b1 = dec[bufIn.charCodeAt(iIn++)]; + bufOut[iOut++] = (b0 << 2) & 0xFC | (b1 >>> 4); + if ( iIn !== sizeIn ) { + let b2 = dec[bufIn.charCodeAt(iIn++)]; + bufOut[iOut++] = (b1 << 4) & 0xF0 | (b2 >>> 2); + } + } + this.write(bufOut); + pr.disconnect(this); + }, + disconnect: function(f) { + let pr = pseudoRedirector; + pr.filters.delete(f); + f.disconnect(); + } + }; + + let onBeforeRequestClient = this.onBeforeRequest.callback; + let onBeforeRequest = function(details) { normalizeRequestDetails(details); return onBeforeRequestClient(details); }; @@ -144,7 +206,7 @@ vAPI.net.registerListeners = function() { let urls = this.onBeforeRequest.urls || ['']; let types = this.onBeforeRequest.types || undefined; if ( - (validTypes.websocket) && + (validTypes.has('websocket')) && (types === undefined || types.indexOf('websocket') !== -1) && (urls.indexOf('') === -1) ) { @@ -174,10 +236,10 @@ vAPI.net.registerListeners = function() { ); } - var onHeadersReceivedClient = this.onHeadersReceived.callback, + let onHeadersReceivedClient = this.onHeadersReceived.callback, onHeadersReceivedClientTypes = this.onHeadersReceived.types.slice(0), onHeadersReceivedTypes = denormalizeTypes(onHeadersReceivedClientTypes); - var onHeadersReceived = function(details) { + let onHeadersReceived = function(details) { normalizeRequestDetails(details); if ( onHeadersReceivedClientTypes.length !== 0 && diff --git a/src/1p-filters.html b/src/1p-filters.html index f2da57cb919db..9c482cf4477bb 100644 --- a/src/1p-filters.html +++ b/src/1p-filters.html @@ -4,27 +4,45 @@ uBlock — Your filters - - - - + + + + + + + + + -
- -

-

-   - -

-

-

-   - - -

+
+
+ +

+

+ + +    + + +

+
+
+ + + + + + + + + + + diff --git a/src/3p-filters.html b/src/3p-filters.html index efc2a8cfde87e..1416251f080ab 100644 --- a/src/3p-filters.html +++ b/src/3p-filters.html @@ -12,29 +12,29 @@ -
- -
-
    -
  • - - -
  •   -
  • -
    -
  • -
    -
-

-
    -
    - -
    -

    - - - +

    +
    +
    +
      +
    • + + +
    •   +
    • +
      +
    • +
      +
    +

    +
      +
      +

      + + + +

      +
      -
      -

        - - -

      -

      -

      + diff --git a/src/web_accessible_resources/README.txt b/src/web_accessible_resources/README.txt new file mode 100644 index 0000000000000..6b0a4c41aa4a7 --- /dev/null +++ b/src/web_accessible_resources/README.txt @@ -0,0 +1,11 @@ +IMPORTANT + +Content of this folder cannot be accessed without the internal secret token +created each time uBlock Origin is launched. + +Any fetch operation made without uBlock Origin's internal secret will result +in failure. This means that despite the content of the folder here declared as +"web accessible resources", it still cannot be seen by the outside world. + +Only uBlock Origin knows the secret token at runtime and hence only +uBlock Origin can access the content of this folder. diff --git a/src/web_accessible_resources/imported.txt b/src/web_accessible_resources/imported.txt new file mode 100644 index 0000000000000..2dad1d719fb85 --- /dev/null +++ b/src/web_accessible_resources/imported.txt @@ -0,0 +1,7 @@ +# List of resources imported as "web accessible resources". +# +# To ensure valid filename characters on any platform OS, the filenames are +# constructed using the md5 hash of the respective tokens. +# +# DO NOT REMOVE THIS LINE >>>>> + diff --git a/src/web_accessible_resources/to-import.txt b/src/web_accessible_resources/to-import.txt new file mode 100644 index 0000000000000..969b2efdf7811 --- /dev/null +++ b/src/web_accessible_resources/to-import.txt @@ -0,0 +1,55 @@ +# This is a list of resources (by token) which will be converted to +# "web accessible resources", such that uBO will be able to redirect +# to these through moz-extension: or chrome-extension: URLs. +# +# This addresses: +# - https://github.com/gorhill/uBlock/issues/3474 +# - https://github.com/gorhill/uBlock/issues/2823 +# +# uBO attaches a "secret" token internally when redirecting to any +# "web accessible resource", such that it is not possible for a web +# page to use one of these "web accessible resource" to directly +# detect the presence of uBO. +# +# To ensure valid filename characters on any platform OS, the filenames are +# constructed using the md5 hash of the respective tokens. +# +# In case uBO redirects to a resource which has not been converted into +# a "web accessible resource", the redirection code will fall back to +# using a data: URI. +# +# The list below was gathered manually from scanning the use of the +# "redirect=" option in uBO's own filter lists. Eventually a script could +# be written to generate the list below. + +1x1-transparent.gif +2x2-transparent.png +32x32-transparent.png +3x2-transparent.png +addthis.com/addthis_widget.js +amazon-adsystem.com/aax2/amzn_ads.js +antiAdBlock.js +d3pkae9owd2lcf.cloudfront.net/mb105.js +disqus.com/embed.js +disqus.com/forums/*/embed.js +doubleclick.net/instream/ad_status.js +fuckadblock.js-3.2.0 +google-analytics.com/analytics.js +google-analytics.com/cx/api.js +google-analytics.com/ga.js +google-analytics.com/inpage_linkid.js +googlesyndication.com/adsbygoogle.js +googletagmanager.com/gtm.js +googletagservices.com/gpt.js +hd-main.js +ligatus.com/*/angular-tag.js +noopframe +noopjs +noopmp3-0.1s +nooptext +popads-dummy.js +popads.net.js +scorecardresearch.com/beacon.js +silent-noeval.js +static.chartbeat.com/chartbeat.js +widgets.outbrain.com/outbrain.js diff --git a/src/whitelist.html b/src/whitelist.html index b2d724fa38371..8c31ed2df6d7d 100644 --- a/src/whitelist.html +++ b/src/whitelist.html @@ -4,30 +4,48 @@ uBlock — Whitelist - - - - + + + + + + + + + -
      - -

      -

      -   - -

      - -
      E
      -
      -

      -   - - +

      +
      + +

      +

      +

      + + +    + + +

      +
      +
      + + + + + + + + + + + diff --git a/tools/import-crowdin.sh b/tools/import-crowdin.sh index f12a800b5ee48..06c8af6e2366f 100755 --- a/tools/import-crowdin.sh +++ b/tools/import-crowdin.sh @@ -36,6 +36,7 @@ cp $SRC/id/messages.json $DES/id/messages.json cp $SRC/it/messages.json $DES/it/messages.json cp $SRC/ja/messages.json $DES/ja/messages.json cp $SRC/ka/messages.json $DES/ka/messages.json +cp $SRC/kk/messages.json $DES/kk/messages.json cp $SRC/kn/messages.json $DES/kn/messages.json cp $SRC/ko/messages.json $DES/ko/messages.json cp $SRC/lt/messages.json $DES/lt/messages.json @@ -43,7 +44,7 @@ cp $SRC/lv/messages.json $DES/lv/messages.json cp $SRC/ml-IN/messages.json $DES/ml/messages.json cp $SRC/mr/messages.json $DES/mr/messages.json cp $SRC/ms/messages.json $DES/ms/messages.json -cp $SRC/no/messages.json $DES/nb/messages.json +cp $SRC/nb/messages.json $DES/nb/messages.json cp $SRC/nl/messages.json $DES/nl/messages.json cp $SRC/pl/messages.json $DES/pl/messages.json cp $SRC/pt-BR/messages.json $DES/pt_BR/messages.json @@ -93,6 +94,7 @@ cp $SRC/id/description.txt $DES/description-id.txt cp $SRC/it/description.txt $DES/description-it.txt cp $SRC/ja/description.txt $DES/description-ja.txt cp $SRC/ka/description.txt $DES/description-ka.txt +cp $SRC/kk/description.txt $DES/description-kk.txt cp $SRC/ko/description.txt $DES/description-ko.txt cp $SRC/kn/description.txt $DES/description-kn.txt cp $SRC/lt/description.txt $DES/description-lt.txt @@ -100,7 +102,7 @@ cp $SRC/lv/description.txt $DES/description-lv.txt cp $SRC/ml-IN/description.txt $DES/description-ml.txt cp $SRC/ms/description.txt $DES/description-ms.txt cp $SRC/mr/description.txt $DES/description-mr.txt -cp $SRC/no/description.txt $DES/description-no.txt +cp $SRC/nb/description.txt $DES/description-nb.txt cp $SRC/nl/description.txt $DES/description-nl.txt cp $SRC/pl/description.txt $DES/description-pl.txt cp $SRC/pt-BR/description.txt $DES/description-pt_BR.txt diff --git a/tools/import-war.py b/tools/import-war.py new file mode 100755 index 0000000000000..70f36570fabd9 --- /dev/null +++ b/tools/import-war.py @@ -0,0 +1,92 @@ +#!/usr/bin/env python3 + +import base64 +import hashlib +import os +import re +import sys + +if len(sys.argv) == 1 or not sys.argv[1]: + raise SystemExit('Build dir missing.') + +# resource_dir = os.path.join(os.path.split(os.path.abspath(__file__))[0], '..') +build_dir = os.path.abspath(sys.argv[1]) + +# Read list of resource tokens to convert +to_import = set() +with open('./src/web_accessible_resources/to-import.txt', 'r') as f: + for line in f: + line = line.strip() + if len(line) != 0 and line[0] != '#': + to_import.add(line) + +# https://github.com/gorhill/uBlock/issues/3636 +safe_exts = { 'javascript': 'js' } + +imported = [] + +# scan the file until a resource to import is found +def find_next_resource(f): + for line in f: + line = line.strip() + if len(line) == 0 or line[0] == '#': + continue + parts = line.partition(' ') + if parts[0] in to_import: + return (parts[0], parts[2].strip()) + return ('', '') + +def safe_filename_from_token(token, mime): + h = hashlib.md5() + h.update(bytes(token, 'utf-8')) + name = h.hexdigest() + # extract file extension from mime + match = re.search('^[^/]+/([^\s;]+)', mime) + if match: + ext = match.group(1) + if ext in safe_exts: + ext = safe_exts[ext] + name += '.' + ext + return name + +def import_resource(f, token, mime): + isBinary = mime.endswith(';base64') + lines = [] + for line in f: + if line.strip() == '': + break + if line.lstrip()[0] == '#': + continue + if isBinary: + line = line.strip() + lines.append(line) + filename = safe_filename_from_token(token, mime) + filepath = os.path.join(build_dir, 'web_accessible_resources', filename) + filedata = ''.join(lines) + if isBinary: + filedata = base64.b64decode(filedata) + else: + filedata = bytes(filedata, 'utf-8') + with open(filepath, 'wb') as fo: + fo.write(filedata) + imported.append(token + '\n\t' + filename) + +# Read content of the resources to convert +# - At this point, it is assumed resources.txt has been imported into the +# package. +resources_filename = os.path.join(build_dir, 'assets/ublock/resources.txt') +with open(resources_filename, 'r') as f: + while True: + token, mime = find_next_resource(f) + if token == '': + break + import_resource(f, token, mime) + +# Output associations +content = '' +with open('./src/web_accessible_resources/imported.txt', 'r') as f: + content = f.read() + '\n'.join(imported) + filename = os.path.join(build_dir, 'web_accessible_resources/imported.txt') + with open(filename, 'w') as f: + f.write(content) + diff --git a/tools/make-chromium-meta.py b/tools/make-chromium-meta.py index 04990bfa70669..f88dda0bfa24d 100644 --- a/tools/make-chromium-meta.py +++ b/tools/make-chromium-meta.py @@ -9,24 +9,27 @@ raise SystemExit('Build dir missing.') proj_dir = os.path.join(os.path.split(os.path.abspath(__file__))[0], '..') +build_dir = os.path.abspath(sys.argv[1]) -manifest_in = {} -manifest_in_file = os.path.join(proj_dir, 'platform', 'chromium', 'manifest.json') -with open(manifest_in_file) as f1: - manifest_in = json.load(f1) +version = '' +with open(os.path.join(proj_dir, 'dist', 'version')) as f: + version = f.read().strip() + +manifest_out = {} +manifest_out_file = os.path.join(build_dir, 'manifest.json') +with open(manifest_out_file) as f: + manifest_out = json.load(f) + +manifest_out['version'] = version # Development build? If so, modify name accordingly. -match = re.search('^\d+\.\d+\.\d+\.\d+$', manifest_in['version']) +match = re.search('^\d+\.\d+\.\d+\.\d+$', version) if match: - build_dir = os.path.abspath(sys.argv[1]) dev_build = ' dev build' - manifest_out = {} - manifest_out_file = os.path.join(build_dir, 'manifest.json') - with open(manifest_out_file) as f2: - manifest_out = json.load(f2) manifest_out['name'] += dev_build manifest_out['short_name'] += dev_build manifest_out['browser_action']['default_title'] += dev_build - with open(manifest_out_file, 'w') as f2: - json.dump(manifest_out, f2, indent=2, separators=(',', ': '), sort_keys=True) - f2.write('\n') + +with open(manifest_out_file, 'w') as f: + json.dump(manifest_out, f, indent=2, separators=(',', ': '), sort_keys=True) + f.write('\n') diff --git a/tools/make-chromium.sh b/tools/make-chromium.sh index af36c29d1bca0..2f3f06cab72af 100755 --- a/tools/make-chromium.sh +++ b/tools/make-chromium.sh @@ -31,7 +31,11 @@ mv /tmp/contentscript.js $DES/js/contentscript.js rm $DES/js/vapi-usercss.js # Chrome store-specific -cp -R $DES/_locales/nb $DES/_locales/no +cp -R $DES/_locales/nb $DES/_locales/no + +echo "*** uBlock0.chromium: Generating web accessible resources..." +cp -R src/web_accessible_resources $DES/ +python3 tools/import-war.py $DES/ echo "*** uBlock0.chromium: Generating meta..." python tools/make-chromium-meta.py $DES/ diff --git a/tools/make-firefox-meta.py b/tools/make-firefox-meta.py index 875885fdb6355..efa77013c7272 100644 --- a/tools/make-firefox-meta.py +++ b/tools/make-firefox-meta.py @@ -20,6 +20,17 @@ def mkdirs(path): pj = os.path.join +# Find path to project root +proj_dir = os.path.split(os.path.abspath(__file__))[0] +while not os.path.isdir(os.path.join(proj_dir, '.git')): + proj_dir = os.path.normpath(os.path.join(proj_dir, '..')) + +# Check that found project root is valid +version_filepath = os.path.join(proj_dir, 'dist', 'version') +if not os.path.isfile(version_filepath): + print('Version file not found.') + exit(1) + build_dir = os.path.abspath(sys.argv[1]) source_locale_dir = pj(build_dir, '_locales') target_locale_dir = pj(build_dir, 'locale') @@ -60,23 +71,27 @@ def mkdirs(path): rmtree(source_locale_dir) # update install.rdf -proj_dir = pj(os.path.split(os.path.abspath(__file__))[0], '..') -chromium_manifest = pj(proj_dir, 'platform', 'chromium', 'manifest.json') +chromium_manifest = pj(proj_dir, 'platform', 'chromium', 'manifest.json') with open(chromium_manifest, encoding='utf-8') as m: manifest = json.load(m) +# Fetch extension version # https://developer.mozilla.org/en-US/Add-ons/AMO/Policy/Maintenance#How_do_I_submit_a_Beta_add-on.3F # "To create a beta channel [...] '(a|alpha|b|beta|pre|rc)\d*$' " -match = re.search('^(\d+\.\d+\.\d+)(\.\d+)$', manifest['version']) +version = '' +with open(version_filepath) as f: + version = f.read().strip() +match = re.search('^(\d+\.\d+\.\d+)(\.\d+)$', version) if match: buildtype = int(match.group(2)[1:]) if buildtype < 100: builttype = 'b' + str(buildtype) else: builttype = 'rc' + str(buildtype - 100) - manifest['version'] = match.group(1) + builttype + version = match.group(1) + builttype +manifest['version'] = version manifest['homepage'] = 'https://github.com/gorhill/uBlock' manifest['description'] = descriptions['en'] diff --git a/tools/make-opera.sh b/tools/make-opera.sh index 5dd9c91e3b4d2..7e00d3287511e 100755 --- a/tools/make-opera.sh +++ b/tools/make-opera.sh @@ -35,6 +35,7 @@ cp platform/opera/manifest.json $DES/ rm -r $DES/_locales/cv rm -r $DES/_locales/hi rm -r $DES/_locales/ka +rm -r $DES/_locales/kk rm -r $DES/_locales/mr rm -r $DES/_locales/ta diff --git a/tools/make-safari-meta.py b/tools/make-safari-meta.py index ad6a63cd4f568..1f1aa056a90d5 100644 --- a/tools/make-safari-meta.py +++ b/tools/make-safari-meta.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 import os +import re import json import sys from io import open @@ -58,6 +59,19 @@ def mkdirs(path): manifest['buildNumber'] = int(time()) manifest['description'] = description +# Fetch extension version +with open(os.path.join(proj_dir, 'dist', 'version')) as f: + version = f.read().strip() +manifest['version'] = version + +# Development build? If so, modify name accordingly. +match = re.search('^\d+\.\d+\.\d+\.\d+$', version) +if match: + dev_build = ' dev build' + manifest['name'] += dev_build + manifest['short_name'] += dev_build + manifest['browser_action']['default_title'] += dev_build + info_plist = pj(build_dir, 'Info.plist') with open(info_plist, 'r+t', encoding='utf-8', newline='\n') as f: diff --git a/tools/make-safari.sh b/tools/make-safari.sh index 94743628514bc..3622980ea92a5 100755 --- a/tools/make-safari.sh +++ b/tools/make-safari.sh @@ -27,8 +27,15 @@ cp platform/safari/Info.plist "$DES"/ cp platform/safari/Settings.plist "$DES"/ cp LICENSE.txt "$DES"/ +cp platform/chromium/vapi.js "$DES"/js/ + # Use chrome's usercss polyfill -cp platform/chromium/vapi-usercss.js "$DES"/js/ +echo "*** uBlock0.safariextension: Concatenating content scripts..." +cat platform/chromium/vapi-usercss.js > /tmp/contentscript.js +echo >> /tmp/contentscript.js +grep -v "^'use strict';$" $DES/js/contentscript.js >> /tmp/contentscript.js +mv /tmp/contentscript.js $DES/js/contentscript.js +echo ' ✔' # https://github.com/el1t/uBlock-Safari/issues/4 echo -n '*** uBlock0.safariextension: Adding extensions to extensionless assets...' diff --git a/tools/make-webext-meta.py b/tools/make-webext-meta.py index b74b06e19db45..6fc340f4a7b56 100644 --- a/tools/make-webext-meta.py +++ b/tools/make-webext-meta.py @@ -11,19 +11,16 @@ proj_dir = os.path.join(os.path.split(os.path.abspath(__file__))[0], '..') build_dir = os.path.abspath(sys.argv[1]) -# Import version number from chromium platform -chromium_manifest = {} -webext_manifest = {} - -chromium_manifest_file = os.path.join(proj_dir, 'platform', 'chromium', 'manifest.json') -with open(chromium_manifest_file) as f1: - chromium_manifest = json.load(f1) +version = '' +with open(os.path.join(proj_dir, 'dist', 'version')) as f: + version = f.read().strip() +webext_manifest = {} webext_manifest_file = os.path.join(build_dir, 'manifest.json') with open(webext_manifest_file) as f2: webext_manifest = json.load(f2) -match = re.search('^(\d+\.\d+\.\d+)(\.\d+)$', chromium_manifest['version']) +match = re.search('^(\d+\.\d+\.\d+)(\.\d+)$', version) if match: buildtype = int(match.group(2)[1:]) if buildtype < 100: @@ -32,7 +29,7 @@ builttype = 'rc' + str(buildtype - 100) webext_manifest['version'] = match.group(1) + builttype else: - webext_manifest['version'] = chromium_manifest['version'] + webext_manifest['version'] = version with open(webext_manifest_file, 'w') as f2: json.dump(webext_manifest, f2, indent=2, separators=(',', ': '), sort_keys=True) diff --git a/tools/make-webext.sh b/tools/make-webext.sh index 930dd21844043..37a23b3a9a506 100755 --- a/tools/make-webext.sh +++ b/tools/make-webext.sh @@ -42,6 +42,10 @@ rm $DES/img/icon_128.png rm $DES/options_ui.html rm $DES/js/options_ui.js +echo "*** uBlock0.chromium: Generating web accessible resources..." +cp -R src/web_accessible_resources $DES/ +python3 tools/import-war.py $DES/ + echo "*** uBlock0.webext: Generating meta..." python tools/make-webext-meta.py $DES/