Glitches although oboeStream->getXRunCount() returns 0 ? Huawei P, 8.0

[PindongaFlores]

I have glitches even when oboeStream->getXRunCount() returns zero.
What are the possible causes?
My code is based on the example LiveEffectEngine.

[philburk]

What device and version of Android are you using? Note that GetXRunCount does not work with openSL ES for callback mode. Glitches may be caused by getting less data than you expect when you read from the input stream. It can take a while for the input and output streams to synchronize so that there's always data in the buffer. preemption could cause a glitch low down in the audio stack and that would not show up and oboe. That could be caused by a flakey device driver. Phil Burk

…

On Sat, Mar 9, 2019, 2:18 AM PindongaFlores ***@***.***> wrote: I have glitches even when oboeStream->getXRunCount() returns zero. What are the possible causes? My code is based on the example LiveEffectEngine. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#402>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AE76aVeN6v7Yfdi_aixse0jN2uL2S-eqks5vU4p_gaJpZM4bmmaM> .

[philburk]

Try turning on airplane mode to see if the glitches go away.

[PindongaFlores]

Huawei P Smart on Android 8.0.0 with AAudio (in airplane mode already).
I read enough data to output, I check that already (and this shall otherwise be informed by GetXRunCount, right?).
The synchronization problem at the beginning also happens, but the problem I mentioned happens later.
You confirm what I suspected, preemption may cause this but I don't know how to avoid it.
Up to now I tried:
// don't LOG: ok
// don't do MEMORY ALLOCATION: ok
// don't WAIT on other threads: ok
// don't do FILE IO: ok
// don't SLEEP: ok
// use NON-BLOCKING-QUEUE or CIRCULAR-BUFFER: I use double-buffer for now
// avoid CORE-MIGRATION: done, seems to help reduce the number of glitches
// setSustainedPerformanceMode(): NOT-SUPPORTED !!!

Any other ideas?

[PindongaFlores]

found the problem:

• USB-sound-card -> 1920 capacity -> 240 frames in callback -> glitches
• Headphone -> 7680 capacity -> 480 frames in callback -> NO glitches
  So the solution would be using setBufferSizeInFrames(). Unfortunately, I am currently stuck here:
  setBufferSizeInFrames() conflicts with LatencyTuner #400
  Please see my comment there.
  In my case with a buffer capacity 16 times bigger than the frames passed in the callback it works, while it does not work when the buffer capacity is only 8 times bigger.
  Also the number of frames in each callback may have an effect.

[philburk]

Huawei P Smart on Android 8.0.0 with AAudio (in airplane mode already).

There was a problem with AAudio on 8.0 involving reference counting. #40
So Oboe will use Open SL ES by default on 8.0 devices.
Are you specifically requesting AAudio API for Oboe?

OpenSL ES does not support getXRunCount(). #216

USB-sound-card -> 1920 capacity -> 240 frames in callback -> glitches
Headphone -> 7680 capacity -> 480 frames in callback -> NO glitches

Are those for input or output streams?

So the solution would be using setBufferSizeInFrames()

You may need to increase the BufferCapacity using the Builder.

The "BufferCapacity" is the allocated size. The "BufferSize" is how much of the capacity is being used and determines the current latency.

The Capacity must be set using the builder. Size can sometimes be varied after the stream is created. Size cannot exceed the Capacity.

[PindongaFlores]

There was a problem with AAudio on 8.0 involving reference counting. #40
So Oboe will use Open SL ES by default on 8.0 devices.
Are you specifically requesting AAudio API for Oboe?

• I have a library from December 2018. That version selects AAudio automatically.

Are those for input or output streams?

• It's an output stream (callback) with audio from input-stream (obtained with read())

So the solution would be using setBufferSizeInFrames()
You may need to increase the BufferCapacity using the Builder.
The "BufferCapacity" is the allocated size. The "BufferSize" is how much of the capacity is being used and determines the current latency.
The Capacity must be set using the builder. Size can sometimes be varied after the stream is created. Size cannot exceed the Capacity.

• I did all that without success. See setBufferSizeInFrames() conflicts with LatencyTuner #400 (comment)
  I must assume that the USB-sound-card is causing the glitches. With headset they don't occur.

By the way, my question about the function getXRunCount() has been already answered.
Thank you!

[philburk]

I have a library from December 2018. That version selects AAudio automatically.

In the 1.1.1 release, Oboe should pick OpenSL ES on 8.0 and earlier versions. Oboe should use AAudio on 8.1 and later versions. If not then that is a bug.

I must assume that the USB-sound-card is causing the glitches. With headset they don't occur.

The data path is very different for those devices. The problem could be in Android. We will try to reproduce the issue. The "USB sound card" are you using?

[PindongaFlores]

Here are USB sound cards I tried so far:

• ICY BOX (USB to Audio Adapter)
• VIGO from speedlink
• Renkforce 7.1 Soundkarte
  Besides the default sample rate from 48000 I also tried 44100 and 38000.
  In all cases I get glitches of 1ms or 2ms, often these glitches come in pairs, e.g. 1ms glitch + 1ms signal + 1ms glitch.
  I have chosen intentionally to lower price category, also for my smarphone:
  - HUAWEI P smart - model FIG-LX1 - with EMUI 8.0.0 (Android 8.0.0, Kernel 4.4.23+), 3GB RAM.
  As I said before the smartphone does not support sustained performance mode.
  I still need to make several tests e.g. with systrace, profiler, tracelog. Up to now I only tested with the debug version, which I know is up to 10x slower than the release version.
  I have java-threads and I also use POSIX-threads in the C/C++ code, but as far as I can tell my implementation for passing data between threads is not the problem.

[philburk]

My code is based on the example LiveEffectEngine.

Does LiveEffecEngine glitch on that phone?

In all cases I get glitches of 1ms or 2ms,

That sounds like a low level USB issue.
Are you seeing this on any other phones?
If not then I will report this to Huawei.

Do you get more glitches at the beginning and then they happen less often over time? That would be a sign that the streams are slowly getting into sync. Additional buffering between inout and output may be needed.

// use NON-BLOCKING-QUEUE or CIRCULAR-BUFFER: I use double-buffer for now

Sometimes AAudio will have a very short callback of just a few frames. Do you see any small callbacks?

[philburk]

I read enough data to output, I check that already (and this shall otherwise be informed
by GetXRunCount, right?).

No, if you try to read more data than is available then the read method will just return the smaller amount. That does not count as an XRun. An XRun for input would happen if you do not read enough data and the input stream fills up and overruns.

VIGO from speedlink

We are ordering one of those. It's the only one we can get in the US.

Have you tried any other USB devices that do work? I am trying to figure out whether the problem is in the USB device, the Huawei phone, your app, or Android in general.

Does does the unmodified LiveEffec demo glitch on that phone?

[PindongaFlores]

The problem is definitely Huawei and/or Android. I tested different things "without" Oboe and I still see the problem. Sorry I didn't check that before.
Anyways, can you please report this to Huawei and Android? Problems in OpenSL ES or AAudio cannot be discarded, right? The Huawei music-player uses one of those internally I guess.

That sounds like a low level USB issue.

That is exactly what I thought. I want to investigate that too.

Are you seeing this on any other phones?

I have only that phone to test.

Do you get more glitches at the beginning and then they happen less often over time?

Not really. After making more tests I see that the glitches happen just randomly. Though, up to now I have only tested < 1 Min.

Do you see any small callbacks?

Yes, I can see clear cycles in the callback, decreasing and then increasing nr. of samples again.

Have you tried any other USB devices that do work?

No. But I think 3 out of 3 points clearly in direction Huawei/Android, not the USB devices.
Thank you for your support.

[PindongaFlores]

// use NON-BLOCKING-QUEUE or CIRCULAR-BUFFER: I use double-buffer for now

Sometimes AAudio will have a very short callback of just a few frames. Do you see any small callbacks?

I think I now understand what you mean: but I don't call the double-buffer with every call to the callback, I first wait for the double-buffer to get filled, and it has enough space for several calls (e.g. 10ms). Anyways, as you can see from my previous comment the glitches occurr playing normal .wav files on the Huawei music player, e.g. in the 300Hz test-audio the glitches can be clearly seen happening quite randomly.

[PindongaFlores]

@philburk : am I getting an answer yet? is Huawei still supported at all?...shall I throw my phone away?...still waiting here...please give me an update

[philburk]

The sample code, including LiveEffect has been refactored. There have been improvements in the LiveEffect full-duplex stream so that it synchronizes better between input and output. Does that work any better.

Also we have added more tests to OboeTester. Have you tried the Glitch test in OboeTester?

Also what happens if you try the Record and Play test from OboeTester?

Can you upgrade the phone to Android P, 9.0?

is Huawei still supported at all?

Yes. There are complications from trade restrictions. But we can definitely support Huawei for older phones. And Huawei continues to support their phones.

Also we have seen issues where input and output streams can get opened at different rates internally, which can result in garbled audio.

Can you attach a very short WAV files so we can hear the glitching? We can tell a lot from the sound.

[PindongaFlores]

@philburk: Hi Phil, don't get me wrong. I am happy that you come back to my question BUT I raised it 6 months ago!
Even if I make an "exponential" progress right now by following the news, I feel there is no guarantee that the next question will not take again 1/2 an year (?) to be answered.
Unfortunately, I have already given up with my (great) project. I see no chances to continue working on this basis. The support is too poor and the problems are too big.
On the other side, if you could this time HELP me with my request to support more than ONE Audio Device at the same time (the same way normal desktop systems do!) then I would definitely continue working on this and would provide you all the data you want. I am a good tester.
I made this request several times here (on Android I may have not done it properly).
Could you please check this? I am sure that Android / Oboe is perfectly capable of supporting more than one audio device simultaneously for those users that don't request low latency.
If I could tell you what I was working on I am sure you would be fascinated.
You all (Oboe, Android, etc.) have done a great job but, if you continue to ignore requests like mine I am afraid this will have no happy end for you...what I mean is that there might be soon a huge competitor that may "hear" such requests and may support the use-case I request to realize soon that it is indeed a VERY IMPORTANT Use-Case for the next future.
Thanks in advance!

[philburk]

Sorry that we do not always respond quickly to issues. It is because we have been busy working on trying to solve these glitch issues in the underlying code.

Regarding your original question about getting glitches when there are no XRun counts. There are several possible reasons:

1. MMAP data patch relies on timestamps from the DSP and sometimes those timestamps have unexpected behavior. The timing model does not account for that so it cannot detect the XRun. We have made changes in Q to help with that issue and are working with SOC partners on this as well.

2. The glitches may be happening at a low level. The XRun count is only for the buffers closest to the app.

3. OpenSL ES does not support XRun count. On Android 8.0, Oboe will use OpenSL ES.

4. USB devices are more likely to do resampling. The resampling can cause odd buffer sizes or timing, which can lead to glitches.

We are also working on automated glitch tests using Oboe Tester. Then we can test more devices more quickly.

We obtained a VIGO speedlink device and will be testing with that.

my request to support more than ONE Audio Device at the same time

That sounds like a separate Issue. Please open a separate Issue so the two issues do not get tangled up. Also in the new Issue, please give more details about your request, (Input? Output? Both?) Do you want to combine two USB devices or USB+speaker or USB+microphone? What have you tried? What happened? Please answer in the new Issue, not here.

[PindongaFlores]

Can you attach a very short WAV files so we can hear the glitching? We can tell a lot from the sound.

attached you find 2 files I recorded by connecting the smartphone to my PC microphone and playing a test.wav file with the standard preinstalled app. I did that once with the USB-Audio device ICY BOX (there you find some glitches) and once with the direct audio jack output (no glitches here as far as I can tell).
Glitches.zip

[philburk]

Thanks for the recordings. Very interesting.
The glitches appear to be 1 msec regions of silence INSERTED into the stream. Normally regions are swapped. This is a clue that points towards clock drift.

There was also lots of low energy noise.
And the Audio Jack file was NOT a normal size wave. They are appear to suffer from non-linear distortion like soft clipping. There is also a discontinuity at the zero crossing. Very odd. Probably unrelated to the glitching.

This gives me some concrete clues that I can give to Huawei.

[PindongaFlores]

Yes, I actually created some time ago a function to remove those glitches and it worked really well because removing inserted silence restored the signal. But I remember there were also many cases where the silences were NOT inserted (the silence region was exactly the length of the missing signal).
I cannot find the logs right now but I can look if that really helps, just tell me.
Yes, there are all sorts of strange things in those signals. I still wonder that my application worked so good despite all that.
As I said. All this happens without me doing anything (not my code, not use of oboe, etc.), just a normal audio file being played. Of course the connection with the audio cable between devices may not be optimal, it is just a normal stereo audio cable connected pretty straight forward. There might be maladjustment due to connection of speaker (line-level) to mic. This may explain some of the phenomena but not the glitches.

[philburk]

We have not heard back from Huawei and there is not much we can do about this.
Closing as not feasible. Sorry.