Introduce Gapless Playback

[s0]

This PR introduces gapless playback to museeks!

The core of the functionality is actually in an external library that I put together (mostly for the purposes of getting gapless playback to work in museeks).

The result is a somewhat less intrusive PR, most of the changes are in src/ui/reducers/player.ts to ensure that whenever the player or queue state changes, the PreciseAudio instance is updated with the expected queue.

This now also introduces an AudioContext and audio graph into museeks, which means that we can go ahead and implement #127 and other stuff after this!

Fixes #128
Fixes #31

I've tested this mostly on LAME-Encoded MP3s, but there are many more codecs that need to be supported, the good news is that now that the boilerplate is in, adding new codecs is simply a matter of adding them in @synesthesia-project/gapless-meta, and won't require any more plumbing. If you have any examples of gapless audio files that don't work, please let me know.

Limitations:

• This change means that all audio files are decoded into memory in full (as opposed to chunked loading / decoding). This will probably lead to particularly large memory usage for long tracks (such as hour-long mixes). One thing I was thinking about doing (but haven't yet done) is creating a (configurable) threshold where songs that are longer than that threshold will be loaded and played using a classic HTMLAudioElement, meaning that they won't support gapless... this seems like it may be a worthwhile trade-off though?

[ghost]

Congratulations 🍻. DeepCode analyzed your code in 0.105 seconds and we found no issues. Enjoy a moment of no bugs ☀️.

💬 This comment has been generated by the DeepCode bot, installed by the owner of the repository. The DeepCode bot protects your repository by detecting and commenting on security vulnerabilities or other critical issues.

---

☺️ If you want to provide feedback on our bot, here is how to contact us.

[martpie]

A lot of things (but actually less than I expected), thank you very much for the PR, I will try to have a look at it when I can :)

[qcasey]

Thanks for the PR, this is very interesting.

On my machine the memory usage at idle is about the same, which is nice.

Unfortunately loading songs from disk takes a lot longer and a lot more memory than master. 3 min songs take about 3 seconds, while 2+ hour audiobooks take ~20 seconds. The UI seems to hang in the meantime. When the audiobook is finally playing Museeks takes about 2 gigs of memory.

I know #128 in particular is a big change and I will try to help where I can. I like your idea of configuring a threshold for memory usage, but I don't think that will solve load times.

Are you experiencing the same? Or is it time for me to upgrade my SSD 😄 ?

[s0]

Thanks for the PR, this is very interesting.

On my machine the memory usage at idle is about the same, which is nice.

Unfortunately loading songs from disk takes a lot longer and a lot more memory than master. 3 min songs take about 3 seconds, while 2+ hour audiobooks take ~20 seconds. The UI seems to hang in the meantime. When the audiobook is finally playing Museeks takes about 2 gigs of memory.

I know #128 in particular is a big change and I will try to help where I can. I like your idea of configuring a threshold for memory usage, but I don't think that will solve load times.

It should solve the load times too for the longer tracks, as the HTMLAudioElement allows us to inspect the duration before having loaded the entire file. So if the threshold was somewhere between 3 mins and 2 hours, for your case above, you'd experience a 3 second load delay for the song, and no delay for the audiobook (as it wouldn't try to load all of it). In other words, it shouldn't be any worse than current master for long tracks.

Are you experiencing the same? Or is it time for me to upgrade my SSD ?

Memory yes, but those load times are significantly longer than I've experienced! Switching to an SSD isn't a bad idea ;)

I'll start working on the threshold idea now.

[s0]

@qcasey I've updated the library with the threshold capability, which uses an HTMLAudioElement instead of an AudioBuffer for longer tracks, and updated this PR to use that.

I've currently configured it with a threshold of 10 minutes, so anything below that will play gaplessly, and anything above will use the old method, I'm thinking it probably makes sense to make that a user-configurable option in this PR?

Could you test these updates with the same files you used earlier?

[qcasey]

Hope everyone is well!

No unfortunately I don't notice a difference, in either >10 or <10 min songs

[s0]

Hope everyone is well!

No unfortunately I don't notice a difference, in either >10 or <10 min songs

Hi @qcasey, sorry for the late reply, it's been a very few hectic months for me! 😂

Have you got an example track that we could try, i'm surprised that you didn't see a difference with RAM usage, because the Audio should never have been decoded in full with the latest HEAD for this PR.

I'm thinking about perhaps another alternative where we decode the whole song, but only keep a chunk of that in memory and drop the rest, and when we get near to the end of the chunk, we decode the song again and keep the next chunk. This would probably be something quite long (e.g. 10min chunks). This would keep RAM usage down on average, but every ~10 mins, we'd need a bunch of ram to decode the song and then drop it again.

This would be improved if we could find a way to decode only a portion of a song and not require the whole thing be in RAM, but unfortunately the current audio APIs don't provide a way of doing this. Perhaps we could call out to something else (like ffmpeg) for the decoding part? Though this isn't something iv'e done before.

[martpie]

I had a look at the solution and I'm not sure if the trade-offs are worth it, and apparently it's not working for everyone.

This in nonetheless a very interesting solution, and will definitely help nail gapless playback once we play audio with AudioContext