Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--embed-thumbnail conversion failed, ffmpeg and .webp images. #25687

Closed
5 of 6 tasks
thunduaga opened this issue Jun 15, 2020 · 12 comments
Closed
5 of 6 tasks

--embed-thumbnail conversion failed, ffmpeg and .webp images. #25687

thunduaga opened this issue Jun 15, 2020 · 12 comments

Comments

@thunduaga
Copy link

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dl version 2020.06.16.1
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Verbose log

youtube-dl -x -i --add-metadata --embed-thumbnail --audio-format mp3 -v -o '20200615_001/%(title)s.(ext)s' "https://www.youtube.com/watch?v=_mCc3ROOspg"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-x', u'-i', u'--add-metadata', u'--embed-thumbnail', u'--audio-format', u'mp3', u'-v', u'-o', u'20200615_001/%(title)s.(ext)s', u'https://www.youtube.com/watch?v=_mCc3ROOspg']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.06.16.1
[debug] Python version 2.7.16 (CPython) - CYGWIN_NT-6.1-7601-WOW64-3.1.4-340.i686-i686-32bit
[debug] exe versions: ffmpeg git-2020-06-15-9d80f3e, ffprobe git-2020-06-15-9d80f3e
[debug] Proxy map: {}
[youtube] _mCc3ROOspg: Downloading webpage
[youtube] _mCc3ROOspg: Downloading thumbnail ...
[youtube] _mCc3ROOspg: Writing thumbnail to: 20200615_001/SIERRA - UNBROKEN.webp
[debug] Invoking downloader on u'https://r3---sn-ni5f-t8gz.googlevideo.com/videoplayback?expire=1592285498&ei=2gToXvXeC9eSkwbm2ZTwCw&ip=70.69.34.113&id=o-AOVncKkHIH0gplcdeRiavD58Jqim4lxLPWxDaEcYzPUY&itag=251&source=youtube&requiressl=yes&mh=Nf&mm=31%2C26&mn=sn-ni5f-t8gz%2Csn-vgqsrn76&ms=au%2Conr&mv=m&mvi=2&pl=22&initcwndbps=1817500&vprv=1&mime=audio%2Fwebm&gir=yes&clen=3328603&dur=194.941&lmt=1561544859300681&mt=1592263777&fvip=3&keepalive=yes&fexp=23882513&c=WEB&txp=5431432&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgSPt868wMa_RWdeyP2Pl-ViiOEvdpKOByjttG3FBfOB0CIQCy4-M1CjhBtSOLKnx_7btUlfwcU_FF6V-_xzIx9PTTqQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhALYCFaL3SY3MYgKYe81vs8tf_pKvHkrEsV9IK1tYh5BdAiEArAnYxtdkKG2jV6Ic2PFX2GPRHgkbIhdSihU0FBVPRVY%3D&ratebypass=yes'
[download] Destination: 20200615_001/SIERRA - UNBROKEN.(ext)s
[download] 100% of 3.17MiB in 00:00
[debug] ffmpeg command line: ffprobe -show_streams 'file:20200615_001/SIERRA - UNBROKEN.(ext)s'
[ffmpeg] Destination: 20200615_001/SIERRA - UNBROKEN.mp3
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:20200615_001/SIERRA - UNBROKEN.(ext)s' -vn -acodec libmp3lame '-q:a' 5 'file:20200615_001/SIERRA - UNBROKEN.mp3'
Deleting original file 20200615_001/SIERRA - UNBROKEN.(ext)s (pass -k to keep)
[ffmpeg] Adding metadata to '20200615_001/SIERRA - UNBROKEN.mp3'
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:20200615_001/SIERRA - UNBROKEN.mp3' -c copy -metadata 'comment=► Subscribe to Electronic Body Synth Music : https://goo.gl/qjhdSk
► Subscribe to The 80'"'"'s Guy B-Sides : https://bit.ly/2S5Azwl
► Subscribe to The 80'"'"'s Guy : https://goo.gl/PkmN4M


▼ Support:
https://lazerdiscs.bandcamp.com/album/gone


▬
Note for the new Artists:

►► Want to submit your track, visual art for promotion ?
►► Missing any kind of information which belongs to the video (audio or visual) ?
►► Having any issues regarding any of the videos ?

Please look for my email address in my channel'"'"'s about page.

▬
Submission Requirements:

►► Must be Darksynth, EBSM, Dark Electro, Dark Clubbing
►► Please specify the genre on submission
►► Please provide all your social media links for description.

▬
Copyright Info ©

♫ Be aware all music belong to the original artists.
♫ I am not in position to give anyone permission to use this.

#ebm #ebsm #darkclubbing' -metadata 'description=► Subscribe to Electronic Body Synth Music : https://goo.gl/qjhdSk
► Subscribe to The 80'"'"'s Guy B-Sides : https://bit.ly/2S5Azwl
► Subscribe to The 80'"'"'s Guy : https://goo.gl/PkmN4M


▼ Support:
https://lazerdiscs.bandcamp.com/album/gone


▬
Note for the new Artists:

►► Want to submit your track, visual art for promotion ?
►► Missing any kind of information which belongs to the video (audio or visual) ?
►► Having any issues regarding any of the videos ?

Please look for my email address in my channel'"'"'s about page.

▬
Submission Requirements:

►► Must be Darksynth, EBSM, Dark Electro, Dark Clubbing
►► Please specify the genre on submission
►► Please provide all your social media links for description.

▬
Copyright Info ©

♫ Be aware all music belong to the original artists.
♫ I am not in position to give anyone permission to use this.

#ebm #ebsm #darkclubbing' -metadata 'artist=EBSM Electro Body Synth Music' -metadata 'title=SIERRA - UNBROKEN' -metadata 'date=20190605' -metadata 'purl=https://www.youtube.com/watch?v=_mCc3ROOspg' 'file:20200615_001/SIERRA - UNBROKEN.temp.mp3'
[ffmpeg] Adding thumbnail to "20200615_001/SIERRA - UNBROKEN.mp3"
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:20200615_001/SIERRA - UNBROKEN.mp3' -i 'file:20200615_001/SIERRA - UNBROKEN.webp' -c copy -map 0 -map 1 '-metadata:s:v' 'title="Album cover"' '-metadata:s:v' 'comment="Cover (Front)"' 'file:20200615_001/SIERRA - UNBROKEN.temp.mp3'
ERROR: Conversion failed!
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2065, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/embedthumbnail.py", line 51, in run
    self.run_ffmpeg_multiple_files([filename, thumbnail_filename], temp_filename, options)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 235, in run_ffmpeg_multiple_files
    raise FFmpegPostProcessorError(msg)
FFmpegPostProcessorError: Conversion failed!

Description

From what I can tell, when attempting to embed the video thumbnail (into an MP3 file in my case) ffmpeg dies when decoding the .webp file. I tried running the ffmpeg command manually and below is a small snippet of the output:

    TPE1            : EBSM Electro Body Synth Music
    TIT2            : SIERRA - UNBROKEN
    TDRC            : 20190605
    TSSE            : Lavf58.46.101
    Stream #0:0: Audio: mp3, 48000 Hz, stereo, fltp, 136 kb/s
    Metadata:
      encoder         : Lavc58.92
    Stream #0:1: Video: webp, yuv420p(tv, bt470bg/unknown/unknown), 1280x720, q=2-31, 25 tbr, 25 tbn, 25 tbc
    Metadata:
      title           : "Album cover"
      comment         : "Cover (Front)"
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mp3 @ 0000000002630600] No mimetype is known for stream 1, cannot write an attached picture.
av_interleaved_write_frame(): Invalid argument
[mp3 @ 0000000002630600] No packets were sent for some of the attached pictures.
frame=    1 fps=0.0 q=-1.0 Lsize=       3kB time=00:00:00.04 bitrate= 605.0kbits/s speed=N/A
video:34kB audio:1kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

However, when I simply run a conversion from .webp to .jpg

ffmpeg -i "SIERRA - UNBROKEN.webp" "SIERRA - UNBROKEN.jpg"

then repeat the ffmpeg command spat out by youtube-dl when embedding the thumbnail replacing the .webp with the .jpg file it works just fine.

Is there any way to have youtube-dl specify the file type it downloads the video thumbnails as? I believe the --embed-thumbnail flag would work if the thumbnail was originally downloaded as something other than a .webp file.

Thank you for your time,
@Zirro
Copy link

Zirro commented Jun 16, 2020

The default format for the highest resolution thumbnail seems to have changed to .webp, but a .jpg version remains available as well. youtube-dl ought to use the latter for compatibility.

https://i.ytimg.com/vi_webp/HOYDj769s6k/maxresdefault.webp

https://i.ytimg.com/vi/HOYDj769s6k/maxresdefault.jpg

@Janbong
Copy link

Janbong commented Jun 16, 2020

Same issue here, youtube-dl creates the .webp file, but fails to embed it in the (in my case) .mp3 file. Then outputs two mp3 files:

  • one working mp3 file without the thumbnail
  • one .temp.mp3 file, couple of kilobytes, doesn't seem to contain anything

@alexanderadam
Copy link

alexanderadam commented Jun 17, 2020 •
edited
Loading

Could it be that it is unrelated to webp? Because it fails for jpg, too.

Try this one:

$ youtube-dl -v --prefer-ffmpeg --extract-audio --audio-format mp3 --embed-thumbnail 'https://www.youtube.com/watch?v=YJoAZqT8mTs'

@alexmerkel
Copy link
Contributor

Could it be that it is unrelated to webp? Because it fails for jpg, too.

Try this one:

$ youtube-dl -v --prefer-ffmpeg --extract-audio --audio-format mp3 --embed-thumbnail 'https://www.youtube.com/watch?v=YJoAZqT8mTs'

I think this is an unrelated problem, the JPEG thumbnail seems to be corrupted. I am unable to open the downloaded file on its own using Preview or Photoshop, and I assume thats the reason ffmpeg is unable to embed it as well.

@bubbleguuum
Copy link

Shouldn't the default thumbnail be the jpg version though, for better compatibility ? Webp is not universally supported.

@barsnick
Copy link

Shouldn't the default thumbnail be the jpg version though, for better compatibility ? Webp is not universally supported.

Indeed, the ID3 spec says:

The "image/png" or "image/jpeg" picture format should be used when interoperability is wanted.

Embedding of WEBP into MP3 isn't supported by ffmpeg anyway, and that's the (original) issue, reported as No mimetype is known for stream 1, cannot write an attached picture. So youtube-dl should either choose to download JPEG for embedding, or convert WEBP to JPEG first, or let ffmpeg convert.

@noname8753
Copy link

I've also seen it download and add a jpg extension while still webp format and ffmpeg will complain about the mime-type.

postprocessor/embedthumbnail.py temp workaround telling ffmpeg to convert thumb before it adds it:

    if not os.path.exists(encodeFilename(thumbnail_filename)):
        self._downloader.report_warning(
            'Skipping embedding the thumbnail because the file is missing.')
        return [], info

    temp_thumb = encodeFilename(thumbnail_filename, True)
    thumbnail_filename = encodeFilename(thumbnail_filename, True) + ".jpg"

    if self._downloader.params.get('verbose', False):
        self._downloader.to_screen('Converting thumbnail')
    p1 = subprocess.Popen(["ffmpeg", "-i", temp_thumb, thumbnail_filename], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    stdout1, stderr1 = p1.communicate()
    if p1.returncode != 0:
        msg1 = stderr.decode('utf-8', 'replace').strip()
        raise EmbedThumbnailPPError(msg1)

    os.remove(temp_thumb)

    if info['ext'] == 'mp3':

Which works for what I need for the time being (only tested mp3).

@alexmerkel alexmerkel mentioned this issue Jun 18, 2020
Closed
9 tasks
@koloshor
Copy link

Could it be that it is unrelated to webp? Because it fails for jpg, too.

Actually, it doesn't. Google, for reasons unknown, is sending webp files mislabeled with .jpg extensions. If you're comfortable with a hex editor, have a look: you'll see the ASCII for RIFF, 4 bytes of payload length, and the ASCII for WEBP.

Or try opening one of your failed jpgs with IrfanView, which is aware of magic numbers and will pop up "some poop-head gave you a webp file with a jpg extension. Would you like it renamed?

@alexmerkel
Copy link
Contributor

You're right @koloshor, I added a detection for this mislabeling to my pull request #25717.

@MrChocolatine
Copy link

MrChocolatine commented Jun 22, 2020 •
edited
Loading

Same problem as well when using --embed-thumbnail option.
I was about to create an Issue but I found this one, /following thread.

@yuceltoluyag
Copy link

Same problem as well when using --embed-thumbnail option.

@C1eriC
Copy link

C1eriC commented Jun 25, 2020

Same problem with .m4a file and AtomicParsley.exe (--embed-thumbnail).

@ytdl-org ytdl-org locked and limited conversation to collaborators Jun 25, 2020
@dstftw dstftw closed this as completed in a31a022 Sep 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[postprocessor/embedthumbnail] Fix Youtube's WebP thumbnails by converting them to JPG