"Copy clean URL" feature

[fmarier]

We currently have a privacy feature to strip out identifying parameters across sites. Users are asking us to go further and remove anything that's not needed when copying URLs.

For example, there are first-party parameters which are not in-scope for the existing query filter:

• Amazon has URLs like https://www.amazon.com/Echo-Dot/dp/B07FZ8S74R/ref=sr_1_1?crid=1JM4ODN4MKLZO&keywords=alexa+echo&qid=1654644603&sprefix=alexa%2Caps%2C136&sr=8-1 where only https://www.amazon.com/dp/B07FZ8S74R/ is needed.
• Twitter has unnecessary t and s parameters which seem to identify the application and user who shared the link.

There are also parameters which aren't privacy-harming but which clutter shared links needlessly.

This should be opt-in to avoid surprising users who may be thinking that Brave is broken and unable/unwilling the copy the full URL from the URL bar.

Spec: https://docs.google.com/document/d/1ea7eF3s0WTsTlmMPr8LYp5KUyNg8eK4omjizo27qvJE/edit#

[fmarier]

@evq also pointed out that we should also debounce URLs that people copy to the clipboard. So if you copy https://www.google.com/url?q=https://example.com, for example by copying it from Google Docs or Google Calendar, then it would put https://example.com into the clipboard.

So first, aggressively debounce URLs, then strip the query string down to just what's needed.

[Novack]

I think this pollutes the context menu unnecessarily. What about having only one "Copy Link" on context that removes trackers by default, while in the settings an option "Allow trackers on copied links" if the user wants or needs it for some reason?

On a side note, totally against editing content from the clipboard, except explicitly enabling that as an option.

[bsclifton]

@spylogsster is this implemented on Android too? I don't think it is (checked Nightly) so I'm removing OS/Android label

[stephendonner]

Verification PASSED using

Brave	1.45.87 Chromium: 106.0.5249.91 (Official Build) beta (x86_64)
Revision	fa96d5f07b1177d1bf5009f647a5b8c629762157-refs/branch-heads/5249@{#707}
OS	macOS Version 11.7 (Build 20G817)

Case 1: Share widget - PASSED

Steps

1. installed 1.45.87
2. launched Brave
3. opened each of the full URLs below
4. clicked on the Share this page icon in the URL bar
5. chose Copy clean link (only main URL)
6. pasted and compared to expected

Amazon - FAILED

URL: https://www.amazon.com/Hands-free-streaming-device-Alexa-Ultra/dp/B08XMDNVX6/ref=sr_1_3?crid=1GRSDRNDEL1NZ&keywords=alexa+fire&qid=1664992318&qu=eyJxc2MiOiIzLjU4IiwicXNhIjoiMy4wNyIsInFzcCI6IjIuNzQifQ%3D%3D&smid=ATVPDKIKX0DER&sprefix=alexa+fire%2Caps%2C180&sr=8-3

Cleaned URL: https://www.amazon.com/Hands-free-streaming-device-Alexa-Ultra/dp/B08XMDNVX6/ref=sr_1_3?crid=1GRSDRNDEL1NZ&keywords=alexa+fire&qid=1664992318&qu=eyJxc2MiOiIzLjU4IiwicXNhIjoiMy4wNyIsInFzcCI6IjIuNzQifQ%3D%3D&smid=ATVPDKIKX0DER&sprefix=alexa+fire%2Caps%2C180&sr=8-3

Copy clean link (only main URL) icon	resulting URL

dev-pages.bravesoftware.com (1) - PASSED

URL: https://dev-pages.bravesoftware.com/clean-urls/?brave_testing1=foo&brave_testing2=bar&brave_testing3=keep&&;b&d&utm_content=removethis&e=&f=g&=end

Cleaned URL: https://dev-pages.bravesoftware.com/clean-urls/?brave_testing3=keep&&;b&d&e=&f=g&=end

`Copy clean link (only main URL)	resulting URL

Confirmed the cleaned URL was https://dev-pages.bravesoftware.com/clean-urls/?brave_testing3=keep&&;b&d&e=&f=g&=end

dev-pages.bravesoftware.com (2) - PASSED

URL: https://dev-pages.bravesoftware.com/clean-urls/exempted?brave_testing1=foo&brave_testing2=bar&brave_testing3=keep&&;b&d&utm_content=removethis&e=&f=g&=end

Cleaned URL: https://dev-pages.bravesoftware.com/clean-urls/exempted?brave_testing3=keep&&;b&d&e=&f=g&=end

Copy clean link (only main URL)	resulting URL

Confirmed the cleaned URL was https://dev-pages.bravesoftware.com/clean-urls/exempted?brave_testing3=keep&&;b&d&e=&f=g&=end

Twitter - PASSED

URL: https://twitter.com/LBC/status/1577628501364146176?s=20&t=fpVMBMSfgI_pNGQxxpV6bw

Cleaned URL: https://twitter.com/LBC/status/1577628501364146176

Copy clean link (only main URL)	resulting URL

Confirmed the cleaned URL was https://twitter.com/LBC/status/1577628501364146176

Case 2: Context-click on URL bar - PASSED

Steps:

1. installed 1.45.87
2. launched Brave
3. opened each of the full URLs below
4. context-clicked on the URL bar
5. chose Copy clean link (only main URL)
6. pasted and compared to expected

Amazon - FAILED

URL: https://www.amazon.com/Echo-Dot/dp/B07FZ8S74R/ref=sr_1_1?crid=1JM4ODN4MKLZO&keywords=alexa+echo&qid=1654644603&sprefix=alexa%2Caps%2C136&sr=8-1

Cleaned URL: https://www.amazon.com/Echo-Dot/dp/B07FZ8S74R/ref=sr_1_1?crid=1JM4ODN4MKLZO&keywords=alexa+echo&qid=1654644603&sprefix=alexa%2Caps%2C136&sr=8-1

Twitter - PASSED

URL: https://twitter.com/LBC/status/1577628501364146176?s=20&t=fpVMBMSfgI_pNGQxxpV6bw

Cleaned URL: https://twitter.com/LBC/status/1577628501364146176

dev-pages.bravesoftware.com (1) - PASSED

URL: https://dev-pages.bravesoftware.com/clean-urls/?brave_testing1=foo&brave_testing2=bar&brave_testing3=keep&&;b&d&utm_content=removethis&e=&f=g&=end

Cleaned URL: https://dev-pages.bravesoftware.com/clean-urls/?brave_testing3=keep&&;b&d&e=&f=g&=end

dev-pages.bravesoftware.com (2) - PASSED

URL: https://dev-pages.bravesoftware.com/clean-urls/exempted?brave_testing1=foo&brave_testing2=bar&brave_testing3=keep&&;b&d&utm_content=removethis&e=&f=g&=end

Cleaned URL: https://dev-pages.bravesoftware.com/clean-urls/exempted?brave_testing3=keep&&;b&d&e=&f=g&=end

Results

Amazon	Twitter	dev-pages.bravesoftware.com (1)	dev-pages.bravesoftware.com (2)

Amazon clean	Twitter clean	dev-pages.bravesoftware.com (1) clean	dev-pages.bravesoftware.com (2) clean

Case 3: Window types - PASSED

Steps:

1. installed 1.45.87
2. launched Brave
3. opened each of the full URLs below
4. context-clicked on the URL bar
5. chose Copy clean link (only main URL)
6. pasted and compared to expected

Twitter - PASSED

URL: https://twitter.com/LBC/status/1577628501364146176?s=20&t=fpVMBMSfgI_pNGQxxpV6bw

Cleaned URL: https://twitter.com/LBC/status/1577628501364146176

Private window	Private window with Tor	Guest window	cleaned URL

dev-pages.bravesoftware.com (1) - PASSED

URL: https://dev-pages.bravesoftware.com/clean-urls/?brave_testing1=foo&brave_testing2=bar&brave_testing3=keep&&;b&d&utm_content=removethis&e=&f=g&=end

Cleaned URL: https://dev-pages.bravesoftware.com/clean-urls/?brave_testing3=keep&&;b&d&e=&f=g&=end

Private window	Private window with Tor	Guest window	cleaned URL

dev-pages.bravesoftware.com (2) - PASSED

URL: https://dev-pages.bravesoftware.com/clean-urls/exempted?brave_testing1=foo&brave_testing2=bar&brave_testing3=keep&&;b&d&utm_content=removethis&e=&f=g&=end

Cleaned URL: https://dev-pages.bravesoftware.com/clean-urls/exempted?brave_testing3=keep&&;b&d&e=&f=g&=end

Private window	Private window with Tor	Guest window	cleaned URL

Additionally, verified the Google-tracker case - PASSED

Steps:

1. loaded https://apnews.com/hub/russia-ukraine?utm_source=apnewsnav&utm_medium=featured
2. context-clicked the URL bar
3. chose `Copy clean link (only main URL)
4. pasted and checked the link
5. clicked on the Share this page
6. chose Copy clean link (only main URL)
7. pasted and checked the link

Confirmed the new link is https://apnews.com/hub/russia-ukraine

NOTE: filed #25822 to take care of the Amazon case

[snowbound]

Problem with this URL The ?taid= is not being stripped off the URL on Version 1.45.95 Chromium: 106.0.5249.103 . The URL loads fine without that parameter and its associated string on the URL.

[fmarier]

@snowbound It looks like you pasted the wrong URL. I suggest you file an issue in https://github.com/brave/adblock-lists/issues to get that unnecessary parameter added to the list.

[snowbound]

@fmarier As your link did not point to a specific thread I created this post to which I will continue to add parameters that are not being stipped out

[BrajBliss]

In the address bar, the URLs are still not cleaned. I do not want to use extensions like CleanURL. So, does brave remove those?Parameters from URLs including trackers but keeps that in the address bar or it just does not remove it from the URL?

[fmarier]

@vrindavan There are two different features in Brave:

1. The copy clean URLs feature (the topic of this issue) which is about removing all unnecessary elements from URLs when you copy a URL. It doesn't remove it from the URL otherwise, just when you copy (for example to share a link with someone). Note that this is not a privacy feature, but rather a convenience feature to allow users to share shorter URLs.
2. The query string filter which removes parameters which are known to track users across multiples sites. That privacy feature is described in https://github.com/brave/brave-browser/wiki/Query-String-Filter. Note that we do not remove all unnecessary elements from URLs in this filter, just the ones that meet our definition of tracking.

If you want to remove all unnecessary elements from URLs by default, you can always add a third-party list (like Adguard's) to brave://settings/shields/filters.

[BrajBliss]

Thank you for the info @fmarier. I tried putting the URL but looks like something's wrong. I am not sure if there is another format or URL I have to put here.

[fmarier]

@vrindavan Sorry, I should have given you the link to the raw TXT file instead of the GitHub page. This is what you need to put in the browser settings: https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_17_TrackParam/filter.txt

[BrajBliss]

Tried that @fmarier but looks like something is not working:

[BrajBliss]

Hey, it turns out this has something to do with my ISP. Their DNS is blocking raw.githubusercontent.com. I have contacted them, hoping they would whitelist this.

More about that: community/community#42655

Thanks for the filter list. I am using it with CloudFlare Warp but it still keeps the parameters in the URL. Only the extension works for me, it removes all parameters.

[fmarier]

Thanks for the filter list. I am using it with CloudFlare Warp but it still keeps the parameters in the URL. Only the extension works for me, it removes all parameters.

And the parameters you want to remove are listed in the Adguard list?

[fmarier]

@vrindavan If you can provide sample URLs, we can check. Support for these kinds of rules was recently added to Brave by @antonok-edm and so it's possible there are still bugs / limitations.

[BrajBliss]

@fmarier you can try opening Netflix for instance, and play anything and the address bar contains ?trackId which is mentioned in the Adguard list. In the issue it seems to be working as mentioned by others, but I think Brave is unable to implement these rules using filter list URL. You will find both the rules and example URL in the issue below.

If this is indeed a bug from Brave's side then I will keep using CleanURLs which is fine. Let me know if possible.

AdguardTeam/AdguardFilters#126395