[Youtube] Got server error HTTP Error 403: Forbidden(latest master version)

[Linux2010]

Checklist

• I'm reporting a broken site support issue
• I've verified that I'm running youtube-dl version 2024.08.07(the latest master version)
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar bug reports including closed ones
• I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2024.08.07.local
[debug] Python 3.8.5 (CPython x86_64 64bit) - macOS-10.16-x86_64-i386-64bit - OpenSSL 1.1.1h  22 Sep 2020
[debug] exe versions: ffmpeg 7.0.1-tessus, ffprobe 6.1.1, rtmpdump 2.4
[debug] Proxy map: {'http': 'http://127.0.0.1:58309', 'https': 'http://127.0.0.1:58309'}
Usage: youtube-dl [OPTIONS] URL [URL...]

youtube-dl: error: You must provide at least one URL.
Type youtube-dl --help to see a list of all options.

PASTE VERBOSE LOG HERE

Description

• The download will run normally for a short time, and then a 403 error will appear

youtube-dl 'https://www.youtube.com/watch?v=lLSkbZ3-EOs'

[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 10 of 10)...
[download] Skipping fragment 41...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 4 of 10)...

[dirkf]

Indeed this seems to be a pathological video where almost all video formats fail on the first fragment and 299 may fail later, regardless of Python 2.7/3.5/3.9 and User-Agent settings.

yt-dlp 2024.08.06 still works, apparently. It has fancy networking that we can't easily replicate: maybe punt to curl for all requests?

[tansy]

I have that with every single video I try. Curiously enough format '18' work all the time. Other formats that work are '136', '137', '248' and/or '160', but it depends on video - not always the case. Still, format '18' is the most reliable to work.

[dirkf]

So

1. poToken (I agree, this is always being detected today) randomly breaks the download rather than uniformly giving 403 as with the revised n-sig "throttling"?
2. different clients get more or less functional links for formats with the same itag?

[GregoriusT]

Can confirm that a lot of the video-only formats are just being 403-ed in the middle with their downloads, resulting in me getting files that stop after about 10-20 minutes into the video, but still have full sized audio.

By now I have written something into my scripts to just pick format 18 as long as a flag is set, because i foresee this issue happening again in the future once it is eventually fixed... >.>

[dirkf]

So has anyone tried fetching fragments in fragments of <1MB? We already had a work-around to download in fragments to avoid throttling IIRC.

Otherwise:

1. actually ignore the data with poToken and use the existing "punt to API" logic with a selected unafflicted client.
2. extend to user-specified other, or selection of, clients similar to the yt-dlp extractor logic.

[iactix]

Apparently the latest fix worked for not even a day, that doesn't bode well. Personally I keep getting "giving up after 0 fragment retries" in my python stuff.

From what I read in yesterdays thread, it seems like this will just not work out with fake JS interpretation if they try to combat this in the slightest. Like, that almost doesn't deserve the name attack vector, that's an attack landscape.

[tansy]

This change is significant. I checked old, pre quantum Firefox and videos don't work any more, when 3 days ago they did.

[dirkf]

Maybe the new player JS uses some G JS syntax extension (aka ECMA2021+) that hadn't been contemplated in those FF versions. Is there an error in the JS console?

[tansy]

It used to work as embedded or as mobile (when used mobile user agent). Now all of them display all saying error:

An error occurred. Please try again later. (Playback ID: j-bZsC_YehYVyZZ8)
Learn More (https://support.google.com/youtube/?p=player_error1&hl=en)

Loading any video at https://www.youtube.com/embed/1234567890a:

Content Security Policy: Couldn't process unknown directive 'require-trusted-types-for' <unknown>
mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create www-embed-player.js:26:77
InvalidStateError www-embed-player.js:1128:42

Then pressing play:

Error: WebGL: Error during native OpenGL init. base.js:11283:169
Error: WebGL: WebGL creation failed. base.js:11283:169

---

ED.

If it's of any help, despite what was said before, there are some videos that work.

First - this one doesn't, and gives following console log

Content Security Policy: Couldn't process unknown directive 'require-trusted-types-for' <unknown>
mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create www-embed-player.js:26:77
InvalidStateError www-embed-player.js:1128:42
Empty string passed to getElementById(). zVhcVoOEv7o line 2 > eval:795:28944

Error: WebGL: getParameter: parameter: invalid enum value <enum 0x9246> base.js:11283:254

This one does and, with this log:

Content Security Policy: Couldn't process unknown directive 'require-trusted-types-for' <unknown>
mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create www-embed-player.js:26:77
InvalidStateError www-embed-player.js:1128:42
Empty string passed to getElementById(). KAR4fAX5T7Y line 2 > eval:4650:33869

So, after clicking 'play' it gives this error: Error: WebGL: getParameter: parameter: invalid enum value <enum 0x9246> base.js:11283:254.

[TheHolyRoger]

No matter what chunk size I use I'm seeing hard 403 errors at 1Meg as others have reported - I'm able to download as many fragments as I want up to 1Meg and then get a 403.

Have experimented with generating cpns (nonces) and adding them to the format fragment URLs without any luck as well as using rn (request numbers) in the URL query instead of byte ranges. Have tried sleeping between fragments to mimick video playback also without joy.

It feels like they've added a check somewhere which fails at the 1Meg mark but I haven't found anything yet where that might be.

Checking via the browser I can see that youtube is happily downloading /videoplayback fragment URLs above 1Meg without any issue...

[dirkf]

But in the browser the media links have the pot parameter with its poToken challenge result, no? Which is what we can't haz.

In line with step 1 above, I'm gradually pulling stuff from the yt-dlp extractor, enough to download HLS with client ios, but plainly not yet enough to get unblocked links from tv or web_creator, eg with format 135. Should I be expecting that?

[TheHolyRoger]

But in the browser the media links have the pot parameter with its poToken challenge result, no? Which is what we can't haz.

In line with step 1 above, I'm gradually pulling stuff from the yt-dlp extractor, enough to download HLS with client ios, but plainly not yet enough to get unblocked links from tv or web_creator, eg with format 135. Should I be expecting that?

I'm not seeing a pot parameter in the query strings, I am seeing post data in the /videoplayback requests which is referred to in the source as playbackCookie

Edit: Looks like the playbackCookie / POST data is extracted from the bytes of the previous fragment response somehow

[8chanAnon]

This is the procedure that I am using in my own code.

Load https://www.youtube.com/embed/<id#> and find the base.js link. Do the usual to extract the sig and n-sig. Extract the signatureTimestamp for the next step.

Load https://www.youtube.com/youtubei/v1/player with the signatureTimestamp and TVHTML5_SIMPLY_EMBEDDED_PLAYER as the client name.

If the JSON response contains "formats" and/or "adaptiveFormats" then we're good. This covers most videos, including age-gated ones. The 403 problem occurs when we have go to the next step. We can't use "www.youtube.com". We must use "m.youtube.com" with the user agent set to something like "Mozilla/5.0 (Android 14)" which is what I'm using.

Load https://m.youtube.com/watch?v=<id#> and extract the JSON structures that you would otherwise have gotten from the previous step.

And that's it. The extra step is only required for videos that disallow embedding.

[dirkf]

Please don't bother to supply any "me too" reports unless the log shows some novelty that may help with rectification. Just "Like", or whatever, an existing similar report.

You can see how a poToken is being sent in POST data by the browser in the Invidious code that shows how to capture the value. But I understood from yt-dlp discussions that a pot query parameter was used in the media links associated with the pot-ified session.

@8chanAnon's algorithm is what is currently done for age-gate videos, up to the last step with m.youtube.com which is new and interesting. What happens if you skip straight to that step?

Step 2 will only work if TVHTML5_SIMPLY_EMBEDDED_PLAYER is not pot-ified, and that seems to be in question.

[dirkf]

Indeed, Android 14/FF 122 at m.youtube.com didn't list the poToken experiment IDs although yt-dlp has reported unsuccess with Android clients.

[TheHolyRoger]

I don't think it's worth trying to get around the poToken, it will eventually be required in all clients.

I keep digging into base.js when I get some time trying to understand how the token is created, it does seem to be extracted from the bytes of at least the first video fragment as far as I can tell, but not all fragments?...

There's a Uint8Array which appears to be the fragment response data?... manipulated several times and then 82/84/68 bytes of that array are stored as playbackCookie which is then sent in the POST data

[PatrickJRed]

@dirkf

yt-dlp 2024.08.06 still works, apparently. It has fancy networking that we can't easily replicate: maybe punt to curl for all requests?

On a lot of websites I want to download from youtube-dl and curl dont even get the correct html,
instead of the one i would get in my Browser , they receives a version that has the captcha.

Instead I have a bash-script that predownloads the non-captcha html via "https://github.com/lwthiker/curl-impersonate" (runs in docker container , i use tag: 0.5.2-ff-alpine)

AFAIK lwthiker/curl-impersonate is the only http client that completly impersonates an actual browser like firefox.
A lot of the problems I had with 403 errors in youtube-dl where captchas triggered by http-client not being exactly like an official version of firefox or chrome :) (this might even be valid for fragment-downloads)

[dirkf]

Yes, but so far as captcha is generally understood (G/recaptcha, hacaptcha, Cloudflare challenge aka breaks the Web), that is not the problem. Even if it solved the poToken issue, a dependency running under Docker would not be an acceptable solution for the main functionality of the program, though it might be a PoC for a solution.

[dirkf]

Probably a continuation of item 2 here.

[bashonly]

yt-dlp/yt-dlp@712d2ab#diff-b7b9f6790de4427214b61939432e667d95b929d07fd918b9da1a36d7996cc506R259

[Rekrullurker]

At this point, YouTube is doing it intentionally, right? People report error 403, a fix/workaround is found, and then a few days later, 403 errors again.

[dirkf]

I don't suppose that YT ships code accidentally, though with LLM assistance who knows. The imposition of the PO Token challenge in the various player clients looks like a plan being enacted.

The 403 error with MWEB client is now reproducing here, both with the 2025 nightly builds and with MWEB formats in yt-dlp (unlike the previous 403 storm).

On a limited test, format 18 seems unaffected, which is a work-around even if eagle-eyed users find the content unsatisfactory.

Presumably the change linked above will resolve the issue for some period. Maybe TV YT apps are less amenable to updating than mobile apps so that the period could be longer than a few days.

[Rekrullurker]

For what it's worth...

When youtube-dl stops working, I sitch to a version of yt-dlp, back-ported by nicolaasjan, which runs on my old system. I updated it yesterday, and so far it seems unaffected. All the formats seem to be downloading normally.

I just downloaded a video with f136 video (720p) and f140 audio, and it worked fine.

I know someone is going to ask why I don't just use yt-dlp all the time. Well, it's because the executable takes significantly longer to run than youtube-dl, and it seems more CPU intensive as well.

Don't get me wrong, I'm grateful for it working at all, and to nicolaasjan for continuing to back-port it for older Windows versions, but I'd prefer to use youtube-dl if possible.
.

[dirkf]

What would be useful to know is which client is being used in your yt-dlp to fetch the successful formats, or even just the version.

[Rekrullurker]

You mean the user agent? If so;

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36

I've tried setting that with youtube-dl, but it didn't work.

Here's a verbose log from a successful download of formats 136+140. I trimmed out the download progress messages as I don't think they add any useful information.

[debug] Command-line config: ['-f', '136+140', '--verbose', 'https://www.youtube.com/watch?v=Kr2HK_rurho']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out cp1252 (No VT), error cp1252 (No VT), screen cp1252 (No VT)
[debug] yt-dlp version nicolaasjan/[email protected] (win_x86_exe)
[debug] Python 3.9.13 (CPython x86 32bit) - Windows-XP-5.1.2600-SP3 (OpenSSL 3.1.0-dev )
[debug] exe versions: ffmpeg N-105053-gc417616-Reino (fdk,setts)
[debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.12.14, mutagen-1.47.0, requests-2.32.3, sqlite3-3.37.2, urllib3-2.3.0, websockets-14.1
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1838 extractors
[youtube] Extracting URL: https://www.youtube.com/watch?v=Kr2HK_rurho
[youtube] Kr2HK_rurho: Downloading webpage
[youtube] Kr2HK_rurho: Downloading ios player API JSON
[youtube] Kr2HK_rurho: Downloading tv player API JSON
[debug] [youtube] Kr2HK_rurho: ios client https formats require a PO Token which was not provided. They will be skipped as they may yield HTTP Error 403. You can manually pass a PO Token for this client with --extractor-args "youtube:po_token=ios+XXX". For more information, refer to  https://github.com/yt-dlp/yt-dlp/wiki/Extractors#po-token-guide . To enable these broken formats anyway, pass --extractor-args "youtube:formats=missing_pot"
[debug] Loading youtube-nsig.0b866fa6 from cache
[debug] [youtube] Decrypted nsig IvOwG2rTNL5Orb1Q => A4jDbUJHBHLyuA
[debug] Loading youtube-nsig.0b866fa6 from cache
[debug] [youtube] Decrypted nsig h3jEqmKeskuN_LYY => 5dSpnmMjse0SUw
[youtube] Kr2HK_rurho: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec, channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[info] Kr2HK_rurho: Downloading 1 format(s): 136+140
[debug] Invoking http downloader on "https://rr1---sn-j5caxoxu-i2o6.googlevideo.com/videoplayback?expire=1736926632&ei=SBGHZ8iLMpS_kucP6ouW8Q0&ip=24.44.111.244&id=o-ALvcXKFpKQJXy7R9zD9zLiRURZVO0OnnJ6pr8SXGVc7V&itag=136&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C271%2C278%2C313&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&met=1736905032%2C&mh=8S&mm=31%2C29&mn=sn-j5caxoxu-i2o6%2Csn-ab5l6nrl&ms=au%2Crdu&mv=m&mvi=1&pl=22&rms=au%2Cau&pcm2=yes&initcwndbps=3556250&bui=AY2Et-Ny3JQYIiRnlYQ49mAq_QD9ae_QP4PJQgXTtVamra5mVciXtcG8W2s3zMw_C85FRDAw_VA-NEH4&vprv=1&svpuc=1&mime=video%2Fmp4&ns=8iJSBIDlyLyK1XvWCsmqDoEQ&rqh=1&gir=yes&clen=20235034&dur=556.088&lmt=1723822885300421&mt=1736904531&fvip=2&keepalive=yes&lmw=1&fexp=51326932%2C51335594%2C51353498%2C51371294&c=TVHTML5&sefc=1&txp=4432434&n=5dSpnmMjse0SUw&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cpcm2%2Cbui%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRgIhAIAfDmgmrYKPzmdhLPCxmJP_3WbfEOTranF-ZkgBzlhfAiEA5VfpXrDBmx4fIN6UfE2LV6dl9PndLknjm8BsaV9g8Go%3D&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&lsig=AGluJ3MwRgIhAJ1kymnlV12fg9xybnWrBjZpvQ0ZRc1xSwPVmnhIQkZwAiEApokKXnmQDP7RIxd9Yxtdfs0WaWsRg6at6l9Fep7jEOg%3D"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW [Kr2HK_rurho].f136.mp4


[download] 100% of   19.30MiB in 00:00:04 at 4.01MiB/s
[debug] Invoking http downloader on "https://rr1---sn-j5caxoxu-i2o6.googlevideo.com/videoplayback?expire=1736926632&ei=SBGHZ8iLMpS_kucP6ouW8Q0&ip=24.44.111.244&id=o-ALvcXKFpKQJXy7R9zD9zLiRURZVO0OnnJ6pr8SXGVc7V&itag=140&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&met=1736905032%2C&mh=8S&mm=31%2C29&mn=sn-j5caxoxu-i2o6%2Csn-ab5l6nrl&ms=au%2Crdu&mv=m&mvi=1&pl=22&rms=au%2Cau&pcm2=yes&initcwndbps=3556250&bui=AY2Et-Ny3JQYIiRnlYQ49mAq_QD9ae_QP4PJQgXTtVamra5mVciXtcG8W2s3zMw_C85FRDAw_VA-NEH4&vprv=1&svpuc=1&mime=audio%2Fmp4&ns=8iJSBIDlyLyK1XvWCsmqDoEQ&rqh=1&gir=yes&clen=9001818&dur=556.141&lmt=1723814198052623&mt=1736904531&fvip=2&keepalive=yes&lmw=1&fexp=51326932%2C51335594%2C51353498%2C51371294&c=TVHTML5&sefc=1&txp=4432434&n=5dSpnmMjse0SUw&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cpcm2%2Cbui%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRQIhAPwt25P1coSz32ApBjagdsqC7b55D4Kfi2YeoCpiP6EiAiA7fcvPndNxBgpnkZGKGGGjiAJadkquHjMKeX2ayYFFDw%3D%3D&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&lsig=AGluJ3MwRgIhAJ1kymnlV12fg9xybnWrBjZpvQ0ZRc1xSwPVmnhIQkZwAiEApokKXnmQDP7RIxd9Yxtdfs0WaWsRg6at6l9Fep7jEOg%3D"
[download] Destination: Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW [Kr2HK_rurho].f140.m4a


[download] 100% of    8.58MiB in 00:00:02 at 3.54MiB/s
[Merger] Merging formats into "Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW [Kr2HK_rurho].mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i "file:Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW [Kr2HK_rurho].f136.mp4" -i "file:Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW [Kr2HK_rurho].f140.m4a" -c copy -map 0:v:0 -map 1:a:0 -movflags +faststart "file:Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW [Kr2HK_rurho].temp.mp4"
Deleting original file Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW [Kr2HK_rurho].f136.mp4 (pass -k to keep)
Deleting original file Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW [Kr2HK_rurho].f140.m4a (pass -k to keep)

[dirkf]

OK, so that's the version (2025.01.13) with the commit linked above, selecting the tv InnerTube client, that I'm right now shoe-horning into yt-dl.

[nicolaasjan]

that I'm right now shoe-horning into yt-dl.

Seems to work (for now...). 😀️

youtube-dl -v Kr2HK_rurho
[debug] System config: []
[debug] User config: ['--rm-cache-dir', '-i', '-o', '/dev/shm/test-ytd/%(title)s.%(ext)s', '-f', 'bestvideo[height<=1080][ext=mp4][vcodec^=avc]+bestaudio[ext=m4a]/best[ext=mp4]/best', '--no-mtime', '--embed-thumbnail', '--force-ipv4', '--cookies', 'cookies.txt']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'Kr2HK_rurho']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2025.01.15
[debug] Lazy loading extractors enabled
[debug] Single file build
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-5.15.0-130-generic-x86_64-with-glibc2.35 - OpenSSL 3.0.2 15 Mar 2022 - glibc 2.35
[debug] exe versions: ffmpeg N-118301-g851a84650e-20250113, ffprobe N-118301-g851a84650e-20250113, rtmpdump 2.4
[debug] Proxy map: {}
Removing cache dir /home/nico/.cache/youtube-dl ...
[youtube] Kr2HK_rurho: Downloading webpage
[youtube] Kr2HK_rurho: Downloading TVHTML5 API JSON
[youtube] Kr2HK_rurho: Downloading player 0b866fa6
[debug] [youtube] Decrypted nsig 9vuc06WSd4g24q8A => DPutHL0F5FYkWg
[debug] [youtube] Decrypted nsig s_8NWKjE4rk4adH2 => OE7n8z9PYILITg
[youtube] Kr2HK_rurho: Downloading thumbnail ...
[youtube] Kr2HK_rurho: Writing thumbnail to: /dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.jpg
[debug] Invoking downloader on 'https://rr3---sn-5hnekn7s.googlevideo.com/videoplayback?expire=1736949572&ei=5GqHZ6rgNvLk6dsPyYSK4A0&ip=<REDACTED>&id=o-AHQ9OKCt35TIK6j3mo2nkCihKSYoT0hy5h_6F-Y0ymNe&itag=137&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C271%2C278%2C313&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&met=1736927972%2C&mh=8S&mm=31%2C26&mn=sn-5hnekn7s%2Csn-4g5edndl&ms=au%2Conr&mv=m&mvi=3&pl=16&rms=au%2Cau&initcwndbps=3952500&bui=AY2Et-NwxkxjCFIZmFEtJMYI1Cwk5sbZDRvM6kBI2AhoNo8pRLpOMR59Q2igMZt0k1vILq7RYAnduz3E&spc=9kzgDffBRsJBZa5o&vprv=1&svpuc=1&mime=video%2Fmp4&ns=joR9vUnT9ELW1s8t3MW4kS8Q&rqh=1&gir=yes&clen=70654451&dur=556.088&lmt=1723822457414094&mt=1736927562&fvip=2&keepalive=yes&lmw=1&fexp=51326932%2C51335594%2C51353498%2C51371294&c=TVHTML5&sefc=1&txp=4432434&n=OE7n8z9PYILITg&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRQIhAPMj7Cr8dhmJcTGPIN21h7D-RhiMRLmD147XijDBKGB8AiBu9XD3swNxXOfFU_t6H94qKVWAtxYB60M5HabdRzDvag%3D%3D&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&lsig=AGluJ3MwRAIgaj24HT0Hp3juHAnVyo0Law7tMeJLNfy8kaDl1vXYwYcCICkMvUBo7HQbGii6AIdYboRHr07cGtvc_1-DhNKdowOy'
[dashsegments] Total fragments: 7
[download] Destination: /dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.f137.mp4
[download] 100% of 67.38MiB in 00:03
[debug] Invoking downloader on 'https://rr3---sn-5hnekn7s.googlevideo.com/videoplayback?expire=1736949572&ei=5GqHZ6rgNvLk6dsPyYSK4A0&ip=<REDACTED>&id=o-AHQ9OKCt35TIK6j3mo2nkCihKSYoT0hy5h_6F-Y0ymNe&itag=140&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&met=1736927972%2C&mh=8S&mm=31%2C26&mn=sn-5hnekn7s%2Csn-4g5edndl&ms=au%2Conr&mv=m&mvi=3&pl=16&rms=au%2Cau&initcwndbps=3952500&bui=AY2Et-NwxkxjCFIZmFEtJMYI1Cwk5sbZDRvM6kBI2AhoNo8pRLpOMR59Q2igMZt0k1vILq7RYAnduz3E&spc=9kzgDffBRsJBZa5o&vprv=1&svpuc=1&xtags=drc%3D1&mime=audio%2Fmp4&ns=joR9vUnT9ELW1s8t3MW4kS8Q&rqh=1&gir=yes&clen=9001872&dur=556.141&lmt=1723814401078158&mt=1736927562&fvip=2&keepalive=yes&lmw=1&fexp=51326932%2C51335594%2C51353498%2C51371294&c=TVHTML5&sefc=1&txp=4432434&n=OE7n8z9PYILITg&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cxtags%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRQIhALjrFcK7Qos_Z4F4w7sevgH8tSonlKDObtq_TrbvL5dPAiApjd0guW_eH0zjlVs9zMp4CDKuxyfNMYQwVkGwqmKgmw%3D%3D&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&lsig=AGluJ3MwRAIgaj24HT0Hp3juHAnVyo0Law7tMeJLNfy8kaDl1vXYwYcCICkMvUBo7HQbGii6AIdYboRHr07cGtvc_1-DhNKdowOy'
[dashsegments] Total fragments: 1
[download] Destination: /dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.m4a.f140-drc.m4a
[download] 100% of 8.58MiB in 00:01
[ffmpeg] Merging formats into "/dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:/dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.f137.mp4' -i 'file:/dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.m4a.f140-drc.m4a' -c copy -map 0:v:0 -map 1:a:0 'file:/dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.temp.mp4'
Deleting original file /dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.f137.mp4 (pass -k to keep)
Deleting original file /dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.m4a.f140-drc.m4a (pass -k to keep)
[ffmpeg] Adding thumbnail to "/dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:/dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.mp4' -i 'file:/dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.jpg' -c copy -map 0 -map 1 -disposition:v:1 attached_pic 'file:/dev/shm/test-ytd/Commodore Amiga 500 -  ACA500+ Accelerator Card - FULL REVIEW.temp.mp4'

[nicolaasjan]

Only issues when also passing cookies...
See: Issue #12085 (yt-dlp).

Comment

[dirkf]

On 15/01/2025 18:08, nicolaasjan wrote: Only issues when also passing cookies... See: Issue #12085 <yt-dlp/yt-dlp#12085> (yt-dlp).

This shouldn't affect the build you showed, because `web_creator` was never used in the first place.

[bashonly]

@dirkf TVHTML5 never returns m3u8 formats, so for the sake of livestreams you may still want to extract the WEB m3u8 formats from the webpage. m3u8 formats do not require a pot param (for now).

[dirkf]

That makes sense. But we don't have a general way to detect pot requirements for a given media URL other than bitter experience?

[bashonly]

That makes sense. But we don't have a general way to detect pot requirements for a given media URL other than bitter experience?

At this point, all HTTP and DASH formats except 18 from all clients except tv, tv_embedded and web_embedded (yt-dlp's names for them) either have a pot param requirement or the requirement is being A/B tested.

[dirkf]

Sadly no hlsManifestUrl is to be found in the tv API response (expected), nor in the webpage response, nor in the web API response. But it can be extracted from the ios response. Then a test video gets another 17 formats, making 40 altogether, or presumably just the number of HLS formats for a live stream.

[FMecha]

With 2025.01.06, some videos give 403, others don't.

What doesn't gives 403:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--newline', '-i', '-o', 'D:\\vdump\\mp3-dump\\%(title)s.%(ext)s', '-x', '--audio-format', 'mp3', '--ignore-config', '--hls-prefer-native', 'https://www.youtube.com/watch?v=y8PFWIl-xn8', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2025.01.06 [1036478d1] (single file build)
[debug] ** This version was built from the latest master code at https://github.com/ytdl-org/youtube-dl.
[debug] ** For support, visit the main site.
[debug] Python 3.4.4 (CPython AMD64 32bit) - Windows-10-10.0.19041 - OpenSSL 1.0.2d 9 Jul 2015
[debug] exe versions: ffmpeg 3.3.2, ffprobe 3.3.2
[debug] Proxy map: {}
[youtube] y8PFWIl-xn8: Downloading webpage
[youtube] y8PFWIl-xn8: Downloading API JSON
[debug] [youtube] Decrypted nsig NlTVv7M1N7cXrJcLx => DeveAYWDjlgq5A
[debug] [youtube] Decrypted nsig tIWb-Oa7mVAdE718I => mKyrNZpqxqrPWg
[debug] Invoking downloader on 'https://rr1---sn-2uuxa3vh-cqvr.googlevideo.com/videoplayback?requiressl=yes&ns=Y2m8n3P9PWuH7mbIPwpfHg0Q&ip=36.69.192.250&mh=JU&gir=yes&id=o-AKGjEeFApnjrs1YuRNH_Q7mVtMAJkEYwVVaz3NvQJCBJ&sig=AJfQdSswRgIhAJ_Do03lvHbM8UT709U9Lqe2PJSGHdkptvE-ANRB1gqGAiEA3XZnYwYhQ2WoF7gu2F0QvfoJWYSVeWJrVIttvWYPTys%3D&fexp=51326932%2C51331020%2C51335594%2C51353498%2C51371294%2C51384461&mvi=1&mm=31%2C29&n=mKyrNZpqxqrPWg&sefc=1&initcwndbps=751250&xtags=drc%3D1&lsig=AGluJ3MwRQIhAPfVf080DeiaOnbPGqb8hEMLqnDJjnTSG9Vua86WrG78AiA5LlmcewOZYzc5xGSTAyDWDRy6wyv-KTpCbSNPw64TBg%3D%3D&c=MWEB&source=youtube&vprv=1&rms=au%2Cau&mv=m&mn=sn-2uuxa3vh-cqvr%2Csn-npoldn7l&pl=21&ei=c9ePZ4_LOP-FssUPx9LsqQw&bui=AY2Et-Oe2XTRtVYypQK3bfJX1mZ4rl68ghUymFr9Mxe1EVal56-asRLhaOJzozRuTtH4F6kv1PhHa9wS&expire=1737501651&fvip=1&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&spc=9kzgDa6pb4Jd0OChBrLZs8gNzqxRcA8KrKukHG4qqHg3m1ZheHAxLtUliNxUSm95J2WA41M&rqh=1&itag=251&txp=6211224&ms=au%2Crdu&met=1737480051%2C&xpc=EgVo2aDSNQ%3D%3D&mt=1737479599&keepalive=yes&mime=audio%2Fwebm&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cxtags%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&clen=44098&dur=2.181&lmt=1667126454596870&svpuc=1'
[dashsegments] Total fragments: 1
[download] Destination: D:\vdump\mp3-dump\Power Macintosh 6100 startup sound.webm
[download]   2.3% of ~43.06KiB at  1.13KiB/s ETA 00:37
[download]   7.0% of ~43.06KiB at  3.40KiB/s ETA 00:11
[download]  16.3% of ~43.06KiB at  7.91KiB/s ETA 00:04
[download]  34.8% of ~43.06KiB at 16.93KiB/s ETA 00:01
[download]  72.0% of ~43.06KiB at 32.34KiB/s ETA 00:00
[download] 100.0% of ~43.06KiB at 44.88KiB/s ETA 00:00
[download] 100.0% of ~43.06KiB at 44.74KiB/s ETA 00:00
[download] 100% of 43.06KiB in 00:00
[debug] ffmpeg command line: ffprobe -show_streams "file:D:\vdump\mp3-dump\Power Macintosh 6100 startup sound.webm"
[ffmpeg] Destination: D:\vdump\mp3-dump\Power Macintosh 6100 startup sound.mp3
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:D:\vdump\mp3-dump\Power Macintosh 6100 startup sound.webm" -vn -acodec libmp3lame "-q:a" 5 "file:D:\vdump\mp3-dump\Power Macintosh 6100 startup sound.mp3"
Deleting original file D:\vdump\mp3-dump\Power Macintosh 6100 startup sound.webm (pass -k to keep)

What does:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--newline', '-i', '-o', 'D:\\vdump\\mp3-dump\\%(title)s.%(ext)s', '-x', '--audio-format', 'mp3', '--ignore-config', '--hls-prefer-native', 'https://www.youtube.com/watch?v=BYJTrtMg9qg', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2025.01.06 [1036478d1] (single file build)
[debug] ** This version was built from the latest master code at https://github.com/ytdl-org/youtube-dl.
[debug] ** For support, visit the main site.
[debug] Python 3.4.4 (CPython AMD64 32bit) - Windows-10-10.0.19041 - OpenSSL 1.0.2d 9 Jul 2015
[debug] exe versions: ffmpeg 3.3.2, ffprobe 3.3.2
[debug] Proxy map: {}
[youtube] BYJTrtMg9qg: Downloading webpage
[youtube] BYJTrtMg9qg: Downloading API JSON
[debug] [youtube] Extracting signature function js_6e1dd460_107
[debug] [youtube] Decrypted nsig arX1l88doEP5Epcjr => 3K6scmSHnha-5w
[debug] [youtube] Extracting signature function js_6e1dd460_111
[debug] [youtube] Decrypted nsig v39D36B9sEEdQhzpf => Ocp4E8p3eN-4sA
[debug] Invoking downloader on 'https://rr2---sn-2uuxa3vh-cqve.googlevideo.com/videoplayback?expire=1737501678&svpuc=1&gir=yes&sefc=1&id=o-AOdR0HKkdz-gQ2jZv66VBhepLOuvJFSB6ARDA6eipArC&mm=31%2C29&rms=au%2Cau&requiressl=yes&ip=36.69.192.250&itag=251&mn=sn-2uuxa3vh-cqve%2Csn-npoe7nl6&dur=244.881&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&initcwndbps=737500&lsig=AGluJ3MwRgIhAL3pUz_HZ_BkuIkt_uF5xv5EAu9FI2mGcbXgre8dHM05AiEA-XDkSYcrta501CW2yDEB374-XvUB6_WMxEXSSLIy7g4%3D&mime=audio%2Fwebm&ms=au%2Crdu&xpc=EgVo2aDSNQ%3D%3D&mt=1737479599&source=youtube&n=Ocp4E8p3eN-4sA&lmt=1731107021235608&vprv=1&rqh=1&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&txp=4532434&mv=m&fexp=51326932%2C51331020%2C51335594%2C51353498%2C51371294%2C51384461&pl=21&c=MWEB&keepalive=yes&mvi=2&sig=AJfQdSswRQIgPa6algGOYVCs5VcdzukzYC70QxsdfRqQ4YyyH_fD4wgCIQCvOWEmXUpU_5UT4AN-Tfm2T2x-VQtFxoX6aIHTv1BEnw%3D%3D&met=1737480078%2C&bui=AY2Et-MzrDsbIOI2TYJ81i9d8nVQIdh0jer4ffni7Cpz1SkzwJE6jFz2SANFsCPeKFHJX8mcPSF-2fwd&spc=9kzgDdDln8DnA993ScPPr0_3bK8GZIuAFcCSYhkNHVh3nk7FVGYeyV8Fb0QyS8ER5520JXw&mh=0S&ei=jtePZ-WSJNCYssUPxq3pOA&ns=W-a4eQmnsaT_iq-Gfs6k3RcQ&fvip=3&clen=4154020'
[dashsegments] Total fragments: 1
[download] Destination: D:\vdump\mp3-dump\Glenn Fredly & Andi Rianto - Dibalas Dengan Dusta (Official Music Video).webm
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 10 of 10)...
ERROR: giving up after 10 fragment retries
  File "__main__.py", line 19, in <module>
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\__init__.py", line 477, in main
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\__init__.py", line 467, in _real_main
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 2258, in download
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 872, in extract_info
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 879, in wrapper
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 986, in __extract_info
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 1020, in process_ie_result
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 1864, in process_video_result
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 142, in wrapper
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 2162, in process_info
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 2092, in dl
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\downloader\common.py", line 380, in download
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\downloader\dash.py", line 78, in real_download
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\downloader\common.py", line 175, in report_error
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 682, in report_error
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 634, in trouble

Using an user agent doesn't seem to resolve/work it around, though -f 18 does.

[dirkf]

2025.01.06 won't generally download successfully, except format 18, because of #32905 (comment). You may also get #32960, with that version and with 2025.01.16 at least. Try the new version from last night.

[nicolaasjan]

I can confirm that the last video example from @FMecha works with the youtube-dl version built from the latest master:

youtube-dl -vF BYJTrtMg9qg
[debug] System config: []
[debug] User config: ['--rm-cache-dir', '-i', '-o', '/dev/shm/test-ytd/%(title)s.%(ext)s', '-f', 'bestvideo[height<=1080][ext=mp4][vcodec^=avc]+bestaudio[ext=m4a]/best[ext=mp4]/best', '--no-mtime', '--embed-thumbnail', '--force-ipv4']
[debug] Custom config: []
[debug] Command-line args: ['-vF', 'BYJTrtMg9qg']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2025.01.20
[debug] Lazy loading extractors enabled
[debug] Single file build
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-5.15.0-130-generic-x86_64-with-glibc2.35 - OpenSSL 3.0.2 15 Mar 2022 - glibc 2.35
[debug] exe versions: ffmpeg N-118328-g504df09c34-20250120, ffprobe N-118328-g504df09c34-20250120, rtmpdump 2.4
[debug] Proxy map: {}
Removing cache dir /home/user/.cache/youtube-dl ...
[youtube] BYJTrtMg9qg: Downloading webpage
[youtube] BYJTrtMg9qg: Downloading TVHTML5 API JSON
[debug] [youtube] Extracting signature function js_6e1dd460_111
[youtube] BYJTrtMg9qg: Downloading player 6e1dd460
[debug] [youtube] Decrypted nsig I6jHYdh1q7OtXBjbU => kHJxeNV2EADScg
[debug] [youtube] Decrypted nsig o0TCe4UJHwfraViui => JBxzipOBdw19bQ
[debug] [youtube] Extracting signature function js_6e1dd460_107
[info] Available formats for BYJTrtMg9qg:
format code  extension  resolution note
249          webm       audio only audio_quality_low   51k , webm_dash container, opus  (48000Hz), 1.51MiB
250          webm       audio only audio_quality_low   69k , webm_dash container, opus  (48000Hz), 2.02MiB
140          m4a        audio only audio_quality_medium  129k , m4a_dash container, mp4a.40.2 (44100Hz), 3.78MiB
251          webm       audio only audio_quality_medium  135k , webm_dash container, opus  (48000Hz), 3.96MiB
160          mp4        256x128    144p   39k , mp4_dash container, avc1.4d400c, 30fps, video only, 1.15MiB
394          mp4        256x128    144p   52k , mp4_dash container, av01.0.00M.08, 30fps, video only, 1.54MiB
278          webm       256x128    144p   65k , webm_dash container, vp9, 30fps, video only, 1.93MiB
395          mp4        426x188    240p   65k , mp4_dash container, av01.0.00M.08, 30fps, video only, 1.92MiB
133          mp4        426x188    240p   78k , mp4_dash container, avc1.4d400d, 30fps, video only, 2.28MiB
242          webm       426x188    240p   92k , webm_dash container, vp9, 30fps, video only, 2.70MiB
396          mp4        640x284    360p  114k , mp4_dash container, av01.0.01M.08, 30fps, video only, 3.33MiB
134          mp4        640x284    360p  140k , mp4_dash container, avc1.4d401e, 30fps, video only, 4.10MiB
243          webm       640x284    360p  200k , webm_dash container, vp9, 30fps, video only, 5.86MiB
397          mp4        854x378    480p  190k , mp4_dash container, av01.0.04M.08, 30fps, video only, 5.57MiB
135          mp4        854x378    480p  242k , mp4_dash container, avc1.4d401e, 30fps, video only, 7.07MiB
244          webm       854x378    480p  279k , webm_dash container, vp9, 30fps, video only, 8.16MiB
398          mp4        1280x566   720p  327k , mp4_dash container, av01.0.05M.08, 30fps, video only, 9.56MiB
136          mp4        1280x566   720p  440k , mp4_dash container, avc1.4d401f, 30fps, video only, 12.87MiB
247          webm       1280x566   720p  504k , webm_dash container, vp9, 30fps, video only, 14.72MiB
399          mp4        1920x850   1080p  560k , mp4_dash container, av01.0.08M.08, 30fps, video only, 16.35MiB
248          webm       1920x850   1080p  814k , webm_dash container, vp9, 30fps, video only, 23.77MiB
137          mp4        1920x850   1080p 1152k , mp4_dash container, avc1.640028, 30fps, video only, 33.64MiB
400          mp4        2560x1134  1440p 1498k , mp4_dash container, av01.0.12M.08, 30fps, video only, 43.73MiB
271          webm       2560x1134  1440p 2082k , webm_dash container, vp9, 30fps, video only, 60.79MiB
401          mp4        3840x1700  2160p 2816k , mp4_dash container, av01.0.12M.08, 30fps, video only, 82.22MiB
313          webm       3840x1700  2160p 5765k , webm_dash container, vp9, 30fps, video only, 168.29MiB
18           mp4        640x284    360p  412k , avc1.42001E, 30fps, mp4a.40.2 (44100Hz), 12.04MiB (best)

[dirkf]

When testing nowadays, there are 3 things to check rather than the 1 that used to be enough:

• is the expected format list extracted?
• does this happen without a high probability of Sign in to confirm you're not a bot?
• can formats, other than 18, be fully downloaded (considering that one YT flag causes 403 failure after 1MB)?

[Vangelis66]

@nicolaasjan wrote:

I can confirm that the last video example from @FMecha works
with the youtube-dl version built from the latest master:

... But have you actually attempted a download? 😉

can formats, other than 18, be fully downloaded
(considering that one YT flag causes 403 failure after 1MB)?

yt-dl -vf 136+140 "BYJTrtMg9qg" =>

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ffmpeg-location', '.\\FFmpeg', '--external-downloader-args', '-v 8 -stats', '-vf', '136+140', 'BYJTrtMg9qg']
[debug] Encodings: locale cp1253, fs mbcs, out cp737, pref cp1253
[debug] youtube-dl version 2025.01.21 [63fb0fc41] (single file build)
[debug] ** This version was built from the latest master code at https://github.com/ytdl-org/youtube-dl.
[debug] ** For support, visit the main site.
[debug] Python 3.4.4 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 - OpenSSL 1.0.2d 9 Jul 2015
[debug] exe versions: ffmpeg n6.1-dev-2615-N-112499-ga76b409, ffprobe n6.1-dev-2615-N-112499-ga76b409, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] BYJTrtMg9qg: Downloading webpage
[youtube] BYJTrtMg9qg: Downloading TVHTML5 API JSON
[debug] [youtube] Extracting signature function js_6e1dd460_107
[debug] [youtube] Decrypted nsig Bzj_i0DDUpTWKisyb => ESh-TZPMvAWdlA
[debug] [youtube] Extracting signature function js_6e1dd460_111
[debug] [youtube] Decrypted nsig k_Of0XWt67DI45jTC => kjA6Odff7elycA
[debug] Invoking downloader on 'https://rr1---sn-4vguioxu-n3bz.googlevideo.com/videoplayback?mime=video%2Fmp4&mm=31%2C29&sefc=1&svpuc=1&met=1737510463%2C&rms=au%2Cau&dur=244.844&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&fvip=1&mn=sn-4vguioxu-n3bz%2Csn-nv47lnsk&fexp=51326932%2C51335594%2C51353498%2C51355912%2C51371294%2C51384461&ms=au%2Crdu&gir=yes&pl=18&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C271%2C278%2C313%2C394%2C395%2C396%2C397%2C398%2C399%2C400%2C401&ns=bj_li9XlU0Nhvib2F2yx7PcQ&keepalive=yes&source=youtube&clen=13495911&mv=m&ei=P06QZ9q6LIPsi9oPpMve6Q0&n=kjA6Odff7elycA&itag=136&mt=1737510061&lmt=1731107774736890&spc=9kzgDZ4367xx299ZGbO-J0D9bYk1m_Ks5vmSViqZSB8u0lVIea52k0WCUfQNtUUaOnfOrV0&xpc=EgVo2aDSNQ%3D%3D&vprv=1&lsig=AGluJ3MwRQIhALooqK491zjaOCtZGeyvwA3wjCgMVukg6P4-HFPmnOtyAiAatt8LLCmqz4uMM8_qofNr0KoNQyJbjlO8EwJwHDkCeQ%3D%3D&ip=REDACTED&c=WEB&requiressl=yes&id=o-AFG-FRUxP16g72HfV-_X05srZFahAUV-OLbu-Q993Krg&expire=1737532063&rqh=1&sig=AJfQdSswRgIhAPtpUL34tNPO8hYAxWsHluVcJYGbazqfLSVxaWK0BmCnAiEAvoQERBqd939CiiEjCIwGelTU276a6OuT4aFu6zyqb3k%3D&initcwndbps=1321250&bui=AY2Et-NROzskx6UXLGPzCqYUwuZPqqjdbAFTMxTZN4bHVixLd5bYkUUi-5YE0a6tJG0QH-6QQxNOTdEK&mh=0S&mvi=1&txp=4532434&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt'
[dashsegments] Total fragments: 2
[download] Destination: Glenn Fredly & Andi Rianto - Dibalas Dengan Dusta (Official Music Video)-BYJTrtMg9qg.f136.mp4
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 10 of 10)...
ERROR: giving up after 10 fragment retries
  File "__main__.py", line 19, in <module>
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\__init__.py", line 477, in main
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\__init__.py", line 467, in _real_main
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 2258, in download
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 872, in extract_info
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 879, in wrapper
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 986, in __extract_info
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 1020, in process_ie_result
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 1864, in process_video_result
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 142, in wrapper
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 2156, in process_info
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 2092, in dl
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\downloader\common.py", line 380, in download
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\downloader\dash.py", line 78,in real_download
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\downloader\common.py", line 175, in report_error
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 682, in report_error
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 634, in trouble

... so the answer is NO here 😢 ...

[dirkf]

That's because some idiot regression was introduced that gives you the WEB formats instead of the TVHTML5 formats that are also fetched and supposed to be extracted. OTOH I don't get much #32960 with this version, compared with 2025.01.16 which seems to get that very regularly.

[dirkf]

@Rekrullurker posted

...I've noticed that when it asks for a sign in, it runs through the various status
messages that it prints fairly quickly, but when a download works, it usually
pauses after the first couple lines are printed to the window.

That's because the sign-in error comes when fetching the JSON containing the download links. The nsig challenge, which can take seconds, is executed only when processing those links. -v makes this apparent.

[dirkf]

The tv player client (now) needs the extra header to avoid Sign in to confirm you're not a bot, that I added to resolve #32905 (comment) and then removed when it appeared to have fixed itself. The 2025.01.21 version was not detecting the problem but just failing to overwrite the broken formats from the web page player response and so extracting those instead. This also explains why in recent versions either 403 or Sign in to confirm you're not a bot was reported but generally not both.

[nicolaasjan]

... But have you actually attempted a download? 😉

Oops...
You're right .
It doesn’t work.

[dirkf]

https://github.com/ytdl-org/ytdl-nightly/releases/tag/2025.01.23

[Vangelis66]

... Thanks a bunch 👍 ; can confirm that the latest nightly can indeed complete a download attempt:

yt-dl -f 136+140 "BYJTrtMg9qg" =>

[youtube] BYJTrtMg9qg: Downloading webpage
[youtube] BYJTrtMg9qg: Downloading TVHTML5 API JSON
[youtube] BYJTrtMg9qg: Downloading player 37364e28
[dashsegments] Total fragments: 2
[download] Destination: Glenn Fredly & Andi Rianto - Dibalas Dengan Dusta (Official Music Video)-BYJTrtMg9qg.f136.mp4
[download] 100% of 12.87MiB in 00:10
[dashsegments] Total fragments: 1
[download] Destination: Glenn Fredly & Andi Rianto - Dibalas Dengan Dusta (Official Music Video)-BYJTrtMg9qg.m4a.f140.m4a
[download] 100% of 3.78MiB in 00:03
[ffmpeg] Merging formats into "Glenn Fredly & Andi Rianto - Dibalas Dengan Dusta (Official Music Video)-BYJTrtMg9qg.mp4"
Deleting original file Glenn Fredly & Andi Rianto - Dibalas Dengan Dusta (Official Music Video)-BYJTrtMg9qg.f136.mp4 (pass -k to keep)
Deleting original file Glenn Fredly & Andi Rianto - Dibalas Dengan Dusta (Official Music Video)-BYJTrtMg9qg.m4a.f140.m4a (pass -k to keep)

😄