How does high-level reference-counting, sound channel spawning, and loading from memory work exactly?

[ns-studios]

Hi,

I'm trying to use high-level interface, and really confused about some things.

1. Reference-counting

If I understand correctly, when you load a sound with e.g., ma_sound_init_from_file_w, loading another instance of the same file shouldn't load it again as long as at least one instance of the sound is present, effectively making what some libraries would call a sound channel/voice. If that were the case, I'd expect the load time to be greatly decreased to 1 or 2-digit ms numbers, but although the second instance loads about twice as fast, it still doesn't seem to exhibit sound channel spawning-like behavior.
I am using code similar to the libvorbis decoder example for ogg support, in case that has anything to do with it, but I have also tried this with sound formats MA supports natively.

2. Sound channels/voices
   If my understanding of reference-counting is wrong, and that isn't how you make channels/voices of a single sound, then how would one do this?
   I have tried ma_sound_init_copy(&engine, parent, flags, NULL, chan), and that seems to behave just like ma_sound_init_from_file; it is a bit faster, but not as fast as I'd expect.

3. Loading from memory

Looking at some other posts on here, I ran into a reply that helped me find a way to do this with ma_resource_manager_register_encoded_data_w and using the file label/ID with ma_sound_init_from_file_w, however the memory buffer/data doesn't seem to be copied, making it necessary for me to keep it allocated and manually free it (not a huge issue, but if there is a way to let Miniaudio handle that for me, it would be nice).
Is there a way to accomplish this?
I have ran into other posts mentioning raw buffer chunks, etc. all I wish to do is load a whole encoded sound from memory instead of loading it from a file.

In all of the cases with ma_sound, I use malloc to allocate it before using. E.g., ma_sound *snd=malloc(sizeof(ma_sound)); and free once I call sound uninit.

I'd really appreciate some clarification/help as I'm quite lost/stuck at the moment.

[mackron]

Your understanding of ma_sound and it's instancing is correct. Loading a second instance should be fast so it's a bit strange that would be slow for you. What flags are you passing into ma_sound_init_from_file()?

ma_sound_init_copy() is just a helper, and is doing the same reference counting logic as init_from_file() so it's no surprise that would be the same in terms of performance.

The buffers you pass into the registration functions in the resource manager are indeed self-managed - miniaudio will not take a copy of that data for you, and there are no functions to do so.

Just some general comments, unrelated to your performance issue. miniaudio is different to other audio libraries like FMOD so you might need to alter your mental model a bit. You might be used to the concept of having a "sound", and then from the sound spawning an instance of that sound. In miniaudio it's a bit different. Instead you have a data source (ma_data_source) which is what a ma_sound will pull from for it's audio data. All of the underlying instancing stuff is done at the data source level, but it's still a 1:1 relationship between ma_data_source/ma_sound. It's very confusing if you're coming from FMOD or similar, but it's a very flexible system when you wrap your head around it because allows you to source your audio data from anywhere, such as a sine wave, a network stream, or even a microphone if you wanted to. But basically, ma_sound_init_from_file() is ultimately just a wrapper around ma_resource_manager (which manages your instancing and creates a ma_data_source) and ma_sound_init_from_data_source(). Can go into more detail about that if you care enough.

[ns-studios]

@mackron Thank you so much for the response and clarifications.
Your explanation regarding data_source and resource manager helps clear things up a great deal.
My implementation uses load functions that return ma_sound *, they malloc ma_sound and then return the allocated memory for other functions to use.
I have resource manager config, resource manager, engine config, and engine declared as globals, I have the decoder setup static funcs, do initialization inside of init func, then.
Example (error checking omitted for brevity here):

ma_sound *load(ma_wchar_win32 *file, ma_uint32 flags) {
ma_sound *snd=malloc(sizeof(ma_sound));
ma_result result=ma_sound_init_from_file_w(&engine, file, flags, NULL, NULL, snd);
return snd;
}

ma_result play(ma_sound *snd) {
return ma_sound_start(snd);
}
// and use them with
ma_sound *s1=load(L"file.ogg", 0);
play(s1);
ma_sound *s2=load(L"file.ogg", 0);
play(s2);

Timing load/play for each sound usually shows that the second instance of the sound loads faster (if I run the compiled executable first time after a while, for a 60 mb ogg file (huge for non-streaming, but just for testing purposes), it first takes ~210 ms for the first instance, then ~110 for the other one. Running the second time seems to bring it down to first=~110, second=~90, but that's as good as it seems to get).
Am I doing something wrong? IS there any easy way to diagnose if something's not registering with the reference-counting mechanism?
I have tried replacing ma_wchar_win32 with char, and switching from ..._w funcs to the other ones, but still the same result.
I'm using MA version

#define MA_VERSION_MAJOR    0
#define MA_VERSION_MINOR    11
#define MA_VERSION_REVISION 21

[mackron]

If you load, say, 50 of your sounds in a loop, does your memory usage explode? That would be an obvious indication that the reference counting system is being bypassed for some reason. There's also the trip hazard in section 6.2.2 of the documentation (scroll down a bit), though looking at your code you shouldn't be hitting that at all.

I'll have to run some tests on my end and try to replicate it. What are you using for your Vorbis decoding by the way, just so I can replicate your set up?

[ns-studios]

@mackron I appreciate your help very much.
I think I have narrowed my problem down to flags.
If I use MA_SOUND_FLAG_DECODE, the initial load takes a while (~11 seconds for a 60 mb ogg), but then making instances of it happens instantly.
If I use the default flags, the initial load is faster (~100-200 ms), but other instances still take some time to load. Is this expected behavior?
I'm using setup from examples/custom decoders engine with Xiph vorbis.lib and ogg.lib statically linked, MA_ENABLE_ONLY_SPECIFIC_BACKENDS, MA_ENABLE_WASAPI, MA_NO_ENCODING, MA_DEBUG_OUTPUT, and MA_NO_VORBIS defined, and extras/miniaudio_libvorbis.h included.

[mackron]

OK, that is a big clue and although I've not verified it, I think I know what's going on. Some background. When not streaming (MA_SOUND_FLAG_STREAM), there's essentially two ways of loading the sound into memory. The first is to load the contents of the raw file data into memory and decode on the fly, and the second is to pre-decode the file into memory at initial load time. The second is where MA_SOUND_FLAG_DECODE comes into it. When not using MA_SOUND_FLAG_DECODE, which is the default, every time you initialize a sound, you're actually initializing a decoder for each sound with ma_decoder_init_memory(). I think it's the initialization of this decoder that is causing the slow down. When you are using DECODE, there's no need for an internal ma_decoder per-sound because it's already been decoded at first-load time.

As an experiment, is it possible to modify miniaudio_libvorbis.h and wrap your timer around the ov_open_callbacks() call? That's the libvorbis initialization routine. I'll bet that's where the time is being spent. If that is indeed where it's at, I don't think there's anything miniaudio can do about that.

Regarding timing of your DECODE path, it's not surprising that it takes significantly longer to initialize on first load because it's doing a full decode of the entire file, but I'm rather surprised it takes ~11s. Out of curiosity, are you running that on a fully optimized build? I just mention in because I've seen in practice that optimized builds can make a dramatic difference to decoding times.

[ns-studios]

@mackron Thank you for the insight/suggestions.
I have added timer around libvorbisResult = ov_open_callbacks(pVorbis, &pVorbis->vf, NULL, 0, libvorbisCallbacks);, and it returns 0 no matter if I'm using the decode flag or not.
I have tried loading a flac file of the similar size as decode, and it was much faster, loading in a little less than a second.
What would constitute a fully optimized build?
I compile with Visual Studio Build Tools 2022 and /MD /Ox /arch:AVX2 switches.
I think it's possible the fault could be with my ogg.lib/vorbis.lib since I couldn't get their cmake project to work with vs2022, having to resort to vs2019, but I am fairly certain I chose max optimization and release.

[mackron]

I was just going through my outstanding tickets and just realised something - do you think this might be causing the slowness? #765

As an experiment, what happens if you use MA_NO_MP3?

[ns-studios]

@mackron That's an excellent find!
The fully pre-decoding mode benefits slightly from no mp3 too, but the default (decode on the fly) mode seems to benefit greatly, dropping the load times for spawning multiple instances from 40-140 ms to 2-15.
This makes the default mode a much more viable alternative to pre-decode mode, very useful for sounds that don't need to play absolutely immediately.
I've also compared my setup in pre-decode mode with other software that uses ogg/vorbisfile, and the load times match up (even if loading a flac of the similar size seems to load much faster, but I assume it's just the nature of the compression at play).
I'm very grateful for your help and explanations. Thank you.
I saw your suggestion to use libvorbis in the linked issue, which I am, but I'm not sure how to make it work with mp3.
It also seems as though my setup prefers to use ma_decoding_backend_init__libvorbis over ma_decoding_backend_init_file__libvorbis even if I use init_from_file instead of init_from_file_w. Not sure why this is, but it's probably not much of an issue.
It would be lovely to see a way to work around the mp3 "issue", or a fix eventually.
Thank you so much once again. Not using MP3 decoder is an acceptable sacrifice for the moment.

[mackron]

That's good to hear. I suppose I'll have to figure out a solution for that. I have some ideas, but I think I'll be making the changes in the 0.12 branch and breaking the API slightly.

ma_decoding_backend_init__libvorbis() is being used because the resource manager actually loads the file into memory and uses ma_decoder_init_memory() internally. And since the libvorbis decoder does not implement an onInitMemory callback, it falls back to ma_decoding_backend_init__libvorbis(). That's not a bug and is not causing any significant slowdown.

[ns-studios]

That sounds good!
Thank you for the clarification and help, also for Miniaudio as a whole, of course.

[mackron]

I've implemented a solution for this. It's in the dev-0.12 which means it won't be released for a while. See my comment here for more details: #765 (comment)

[ns-studios]

@mackron Thank you so much!
That sounds like a perfect solution.