Download speed throttled when remuxing the best video and audio

[jeffli678]

New issue

Your environment

Please answer these questions when reporting a new issue:

What is your operating system (Windows, Linux, OSX, etc.)? Ubuntu 16.04 Server

What is your web server (Apache, IIS, etc.)? Nginx

What version of AllTube are you using? 2.0.1

How did you install AllTube (with Git or with a release package)? release package

What version of PHP are you using? PHP 7.0.33-0ubuntu0.16.04.3 (cli) ( NTS )

What version of Python are you using? Python 2.7.12 (default, Nov 12 2018, 14:36:49)

What version of youtube-dl are you using? 2019.04.24

Do you get any PHP-related errors in your webserver's logs? No

What is the content of your config/config.yml file?

---
# Path to your youtube-dl binary
youtubedl: /usr/local/bin/youtube-dl

# Path to your python binary
python: /usr/bin/python

# An array of parameters to pass to youtube-dl
params:
    - --no-warnings
    - --ignore-errors
    - --flat-playlist
    - --restrict-filenames
    - --no-playlist

# True to enable audio conversion
convert: false

# True to enable advanced conversion mode
convertAdvanced: false

# List of formats available in advanced conversion mode
convertAdvancedFormats: [mp3, avi, flv, wav]

# Path to your avconv or ffmpeg binary
avconv: /usr/bin/ffmpeg

# avconv/ffmpeg logging level.
avconvVerbosity: error

# Path to the directory that contains the phantomjs binary.
phantomjsDir: vendor/bin/

# True to disable URL rewriting
uglyUrls: false

# True to stream videos through server
stream: true

# True to enable remux mode (merge best audio and best video)
remux: true

# MP3 bitrate when converting (in kbit/s)
audioBitrate: 128

# App name
appName: AllTube Download

Please provide the URL of a video that causes the issue.

https://www.youtube.com/watch?v=gXkSF0RWauA

Describe your issue

I verify that the previous YouTube slow download issue is fixed in the 2.0.1 release. However, I found the download speed is still slow (throttled to 1MB/s) when I choose to remux the best video and audio. The server CPU and memory usage are low at the time so it is not a performance issue.

I checked the FFmpeg command line and find the FFmpeg is directly downloading the video and audio. This is probably the reason why it gets throttled -- directly downloading the video in big chunks will trigger the server to throttle the download. I think this is not a huge issue -- because we can still download the best video and audio and remux locally. However, it is still better to solve this problem.

One viable approach is to have alltube download the video and audio and send it to FFmpeg for remuxing. I do not read your code so I have no idea currently how alltube avoids the throttling, but it works!

[Rudloff]

I do not read your code so I have no idea currently how alltube avoids the throttling, but it works!

For regular downloads, we use range requests to get a multiple chunks of the video and assemble them.
Unfortunately, ffmpeg does not support this out of the box.

One viable approach is to have alltube download the video and audio and send it to FFmpeg for remuxing.

While it is theoretically possible to do this, I don't think we have an easy way to do it because the Symfony Process library does not provide a way to write to multiple input streams.

(It could also be possible to use posix_mkfifo but I want to avoid writing to the file system.)

Anyway, I will think about it 🙂

[amanalar]

is there anything that can be done on my end to speed up audio only downloads.

[Rudloff]

Audio-only downloads (not conversion) are not throttled anymore (see #217) so if they are slow it is likely an issue with your server ressources (bandwidth, disk speed, etc.).