Prettifying the output sound file: filtering, noise removal, normalization

[morrowwm]

Hi, I am here from mcguirepr89's repository at the suggestion of @alexbelgium with some work I did as discussed at mcguirepr89#253 (comment)

Briefly, I was getting much better sounding output files by reducing noise, filtering frequencies below 1 KHz, and normalizing the volume to a consistent level. Look at the discussion thread cited above for an example.

Note this is only modifying the output sound file. It has no effect on detection. That's a separate area of research.

This new Nachtzuster fork has some substantial changes, so my previous pull request to mcguirepr89/BirdNET-Pi won't fit. It looks like I could modify this new repository in scripts/utils/reporting.py here:

def extract(in_file, out_file, start, stop):

replace the following sox call with python code to:

• trim the output (as sox currently is doing)
• reduce noise
• highpass filter
• normalize

    result = subprocess.run(['sox', '-V1', f'{in_file}', f'{out_file}', 'trim', f'={start}', f'={stop}'],
                            check=True, capture_output=True)
    ret = result.stdout.decode('utf-8')
    err = result.stderr.decode('utf-8')
    if err:
        raise RuntimeError(f'{ret}:\n {err}')
    return ret

Does this look like the right spot?

[alexbelgium]

What is nice is that as it is post analysis, it will improve sound of mp3 files without needing people to force gain on input and risking signal saturation

[lloydbayley]

If implemented, I believe it should be an optional setting. I would rather hear the exact sound it recorded and analysed.

[alexbelgium]

Not in this fork, it was removed to improve time of analysis and decrease resources usages, although I have reimplemented it in my homeassistant add-on based on this fork

[lloydbayley]

What Speedy Gonzalez above said! ^^^^^^^^^ :)

[lloydbayley]

@morrowwm Please, do also understand that I am not against the idea, in principal, it's just that I quite like to keep the original intent of the project as the 'prime-directive' as I've seen way too many projects turn into a "What would you like to add today?" kind of thing. Suddenly a robust package is wearing eyeshadow and party-hats.

I just feel that an option would be a good idea for those data-driven people who don't want to forcibly tidy the recording up. I believe raw and 'pretty' recordings would both have their place.

The only thing I advocate for is that the raw recordings are sent to BirdWeather for processing. Having said that, ultimately, it is NOT my fork and is in the hands of @Nachtzuster .

[morrowwm]

@lloydbayley, totally in agreement with keeping projects slim and svelte. No need for eye-shadow or party hats.

Just out of curiosity, what do you have for a microphone, and recording setup in general?

[lloydbayley]

At the risk of disappointing, I am using the second mic I bought for the project which is a cheap and cheerful USB lapel mic Here. It's not out of stock on that link but I'm sure it could be found as it's a generic thing. Wind shield installed and in a little 3D Printed "House" I made for it up on the roof. The USB cable is extended with USB over UTP extenders I an use CAT6 instead of an enormously-long USB extension. Nothing too fancy, no tuning needed, I just threw it out there and tested and it works fine so haven't had the need to do any more. :) I also use the old model as the new one detects less than half the birds the old model does here in Australia.

[morrowwm]

I've made a preliminary change here: https://github.com/morrowwm/BirdNET-Pi/blame/main/scripts/utils/reporting.py

Only needed to change the one file.

If you want to try it, you will also likely need to install an additional python module:

source ~/BirdNET-Pi/birdnet/bin/activate
python3 -m pip install noisereduce

[YvedD]

tried this (replaced reportings.py with your version in "filemanager") but i lack "Live Audio" now, any ideas on that?
I have noisereduce installed and reporting OK on my installation.

[morrowwm]

tried this (replaced reportings.py with your version in "filemanager") but i lack "Live Audio" now, any ideas on that? I have noisereduce installed and reporting OK on my installation.

Anything interesting in /var/log/syslog?

[YvedD]

I finally did get it up and running...but...the sound was totaly not fit to do a good detection, looked phase shifted and pitched and overly loud (normalize!)
The lowest noise level was very well removed...but the rest was to heavy tweaked to use...
Any ideas?

[morrowwm]

Can you attach a sample?

I'm looking at how to add configuration. My thoughts now are to only do the normalization for now, and probably make the level configurable.

[YvedD]

i think (i'm no sound ing.):
The noise reduction works not to bad at all. someone must invent a proper AI-noise-cancelling for backgroundsound :-)

Problem is the phase shifting and over-normalizing ... if one could keep the correct pitch in the soundclips, and then gradualy set a highpass filter, again without disturbing the pitch and "feel" of the sound, no normalization would be needed, there will always be birds who are to far to detect correctly - as we do in human detectiong in the field ... no?

from next week on i'll have heaps of spare time, as in retired :-)
I can't program myself (not yet in Phyton 3), if i know how to make a good backup and how to set that back (so i won't lose my current NFC's) i can test things out on a daily basis...

Grtz,
Yves

[morrowwm]

@Nachtzuster , what do you think of a reduced scope pull request which just replaces the sox call with "native" python, and normalizes to output to full volume? And perhaps makes the normalization optional?

We can leave out the filtering and noise removal for now.

[Nachtzuster]

This would be nice when we got rid of the other issue with sox #104
In itself I don't see much value, to be honest

[alexbelgium]

Hi, I've tested a new noise reduction script that gives interesting results it seems. I've tested it on several wav files and it gives either equivalent or similar detections levels. Here is how to test it : sed -i 's|-ac 2 -ar 48000|-ac 2 -ar 48000 -af "highpass=f=250,afftdn=nf=-20"|g' "$HOME"/BirdNET-Pi/scripts/birdnet_recording.sh
and revert it sed -i 's|-ac 2 -ar 48000 -af "highpass=f=250,afftdn=nf=-20"|-ac 2 -ar 48000|g' "$HOME"/BirdNET-Pi/scripts/birdnet_recording.sh

Edit : After a couple hours, the noise reduction allowed a significantly better Signal-Noise ratio (seen from the cleaner spectrogram) for some birds while at least equivalent average detection. There was also more false positives. However, I disabled it for the moment as I prefer the natural sound.

Before / after (same bird singing)

[dalogue1]

For people who are curious about the audio filter afftdn options, they are documented here.
https://ffmpeg.org/ffmpeg-filters.html#afftdn

[jkrauska]

is this at a point to be made in to an official issue?

[morrowwm]

I've come around to Nachtzuster's opinion expressed on Jul 8 above, actually. I don't think it has much value on its own.

[alexbelgium]

I've stopped using it too and instead improved my recording system. I've just kept a highband at 250hz for the mp3 recording, but not before the model itself