Skip to content

Commit

Permalink
Rework the event detection/post-processing algorithm
Browse files Browse the repository at this point in the history 
Issue #451 Rewrite the event detection/post-processing algorithm so that post-processing steps are performed for each level of decibel detection threshold. The guide.md file has been edited to explain this change and to make it consistent with changes to the config parameter names.
  • Loading branch information
@towsey
towsey committed Feb 24, 2021
1 parent c7ccc06 commit 049c901
Show file tree
Hide file tree
Showing 5 changed files with 131 additions and 50 deletions.
24 changes: 14 additions & 10 deletions docs/guides/generic_recognizers.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -383,21 +383,23 @@ Some of these algorithms have extra parameters, some do not, but all do have the
| Oscillation | [!OscillationParameters](xref:AnalysisPrograms.Recognizers.Base.OscillationParameters) |
| Harmonic | [!HarmonicParameters](xref:AnalysisPrograms.Recognizers.Base.HarmonicParameters) |

### [PostProcessing](xref:AudioAnalysisTools.Events.Types.EventPostProcessing.PostProcessingConfig)

The post processing stage is run after event detection (the `Profiles`).
Note that these post-processing steps are performed on all acoustic events collectively, i.e. all those "discovered"
by all the *profiles* in the list of profiles.

Add a post processing section to you config file by adding the `PostProcessing` parameter and indenting the sub-parameters.
### [PostProcessing](xref:AudioAnalysisTools.Events.Types.EventPostProcessing.PostProcessingConfig)

Post-processing of events is performed after event detection. However it is important to understand that post-processing is performed once for each of the DecibelThresholds. As an example: suppose you have three decibel thresholds (6, 9 and 12 dB is a typical set of values) in each of two profiles. All the events detected at threshold 6 dB (by both profiles) will be collected together and subjected to the post processing steps. Typically some or all of the events may fail to be accepted as "true" events based on your post-processing parameters. Then all the events detected at 9 dB will be collected and independently subjected to post-processing. Then, likewise, all events detected at the 12 dB threshold will be post-processed. In other words, one round of post-processing is performed for each decibel threshold. This sequence of multiple post-processing steps gives rise to one or more temporally nested events. Think of them as Russion doll events! The final post-processing step is to remove all but the longest duration event in any nested set of events.

[!code-yaml[post_processing](./Ecosounds.NinoxBoobook.yml#L34-L34 "Post Processing")]

Post processing is optional. You may just want to combine or filter component events using code you have written yourself.
Post processing is optional - you may decide to combine or filter the "raw" events using code you have written yourself. To add a post-processing section to your config file, insert the `PostProcessing` parameter and indent the sub-parameters. There are five post-processing possibilities each of which you may choose to use or not. Note that the post-processing steps are performed in this order which cannot be changed by the user:
- Combine events having temporal _and_ spectral overlap.
- Combine possible sequences of events that constitute a "call".
- Remove (filter) events whose duration is outside an acceptable range.
- Remove (filter) events whose bandwidth is outside an acceptable range.
- Remove (filter) events having excessive acoustic activity in their sidebands.

#### Combining overlapping syllables into calls

Combining syllables is the first of two *post-processing* steps.
### Combine events having temporal _and_ spectral overlap

[!code-yaml[post_processing_combining](./Ecosounds.NinoxBoobook.yml#L34-L42 "Post Processing: Combining")]

Expand All @@ -407,7 +409,8 @@ set this true for two reasons:
- the target call is composed of two or more overlapping syllables that you want to join as one event.
- whistle events often require this step to unite whistle fragments detections into one event.

#### Combining syllables into calls

### Combine possible sequences of events that constitute a "call"

Unlike overlapping events, if you want to combine a group of events (like syllables) that are near each other but not
overlapping, then make use of the `SyllableSequence` parameter.
Expand All @@ -430,7 +433,7 @@ constraints defined by `SyllableMaxCount` and `ExpectedPeriod`.

See the <xref:AudioAnalysisTools.Events.Types.EventPostProcessing.SyllableSequenceConfig> document for more information.

### Remove events whose duration or bandwidth lies outside an expected range.
### Remove events whose duration is outside an acceptable range

[!code-yaml[post_processing_filtering](./Ecosounds.NinoxBoobook.yml?start=34&end=62&highlight=20- "Post Processing: filtering")]

Expand All @@ -442,6 +445,7 @@ This filter removes events whose duration lies outside three standard deviations
- `DurationStandardDeviation` defines _one_ SD of the assumed distribution. Assuming the duration is normally distributed, three SDs sets hard upper and lower duration bounds that includes 99.7% of instances. The filtering algorithm calculates these hard bounds and removes acoustic events that fall outside the bounds.


### Remove events whose bandwidth is outside an acceptable range
Use the parameter `Bandwidth` to filter out events whose bandwidth is too small or large.
This filter removes events whose bandwidth lies outside three standard deviations (SDs) of an expected value.

Expand Down
64 changes: 55 additions & 9 deletions src/AnalysisPrograms/Recognizers/GenericRecognizer.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -135,12 +135,55 @@ public override RecognizerResults Recognize(
var results = RunProfiles(audioRecording, configuration, segmentStartOffset);

// ############################### POST-PROCESSING OF GENERIC EVENTS ###############################

var postprocessingConfig = configuration.PostProcessing;
results.NewEvents = EventPostProcessing.PostProcessingOfSpectralEvents(
results.NewEvents,
postprocessingConfig,
results.Sonogram,
segmentStartOffset);
var postEvents = new List<EventCommon>();

// count number of events detected at each decibel threshold.
for (int i = 1; i <= 24; i++)
{
var dbEvents = EventFilters.FilterOnDecibelDetectionThreshold(results.NewEvents, (double)i);

if (dbEvents.Count > 0)
{
//Log.Debug($"Profiles detected {dbEvents.Count} events at threshold {i} dB.");
var ppEvents = EventPostProcessing.PostProcessingOfSpectralEvents(
dbEvents,
postprocessingConfig,
(double)i,
results.Sonogram,
segmentStartOffset);

postEvents.AddRange(ppEvents);
}
}

// Running profiles with multiple dB thresholds produces nested (Russian doll) events.
// Remove all but the outermost event.
// Add a spacer for easier reading of the debug output.
Log.Debug($" ");
Log.Debug($"Event count BEFORE removing enclosed events = {postEvents.Count}.");
results.NewEvents = CompositeEvent.RemoveEnclosedEvents(postEvents);
Log.Debug($"Event count AFTER removing enclosed events = {postEvents.Count}.");

// Write out the events to log.
//Log.Debug($"FINAL event count = {postEvents.Count}.");
if (postEvents.Count > 0)
{
int counter = 0;
foreach (var ev in postEvents)
{
counter++;
var spEvent = (SpectralEvent)ev;
Log.Debug($" Event[{counter}]: Start={spEvent.EventStartSeconds:f1}; Duration={spEvent.EventDurationSeconds:f2}; Bandwidth={spEvent.BandWidthHertz} Hz");
}
}

//results.NewEvents = EventPostProcessing.PostProcessingOfSpectralEvents(
// results.NewEvents,
// postprocessingConfig,
// results.Sonogram,
// segmentStartOffset);
return results;
}

Expand Down Expand Up @@ -186,10 +229,6 @@ public static RecognizerResults RunProfiles(
profileResults.Sonogram = thresholdResults.Sonogram;
}

// Running profiles with multiple dB thresholds produces nested (Russian doll) events.
// Remove all but the outermost event.
profileResults.NewEvents = CompositeEvent.RemoveEnclosedEvents(profileResults.NewEvents);

// Add additional info to the remaining acoustic events
profileResults.NewEvents.ForEach(ae =>
{
Expand Down Expand Up @@ -357,6 +396,13 @@ public static RecognizerResults RunOneProfile(
Sonogram = null,
};

//add info about decibel threshold into the event.
//This info is used later during post-processing of events.
foreach (var ev in spectralEvents)
{
ev.DecibelDetectionThreshold = decibelThreshold.Value;
}

allResults.NewEvents.AddRange(spectralEvents);
allResults.Plots.AddRange(plots);
allResults.Sonogram = spectrogram;
Expand Down
6 changes: 6 additions & 0 deletions src/AudioAnalysisTools/Events/EventCommon.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@ public abstract class EventCommon : EventBase, IDrawableEvent
/// </summary>
public string Profile { get; set; }

/// <summary>
/// Gets or sets the Decibel threshold at which the event was detected.
/// This is used during post-processing to group events according to the threshold of their detection..
/// </summary>
public double DecibelDetectionThreshold { get; set; }

/// <summary>
/// Gets the component name for this event.
/// The component name should indicate what type of event this.
Expand Down
43 changes: 35 additions & 8 deletions src/AudioAnalysisTools/Events/EventFilters.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,17 +77,19 @@ public static List<EventCommon> FilterOnBandwidth(List<EventCommon> events, doub

var filteredEvents = new List<EventCommon>();

var count = 0;
foreach (var ev in events)
{
count++;
var bandwidth = ((SpectralEvent)ev).BandWidthHertz;
if ((bandwidth > minBandwidth) && (bandwidth < maxBandwidth))
{
Log.Debug($" Event accepted: Actual bandwidth = {bandwidth}");
Log.Debug($" Event{count} accepted: Actual bandwidth = {bandwidth}");
filteredEvents.Add(ev);
}
else
{
Log.Debug($" Event rejected: Actual bandwidth = {bandwidth}");
Log.Debug($" Event{count} rejected: Actual bandwidth = {bandwidth}");
continue;
}
}
Expand All @@ -113,6 +115,24 @@ public static List<SpectralEvent> FilterLongEvents(List<SpectralEvent> events, d
return outputEvents;
}

/// <summary>
/// Filters lists of spectral events based on their DecibelDetectionThreshold.
/// </summary>
/// <param name="events">The list of events.</param>
/// <param name="threshold">The Decibel Detection Threshold.</param>
/// <returns>The filtered list of events.</returns>
public static List<EventCommon> FilterOnDecibelDetectionThreshold(List<EventCommon> events, double threshold)
{
if (threshold <= 0.0)
{
throw new Exception("Invalid Decibel Detection Threshold passed to method EventExtentions.FilterOnDecibelDetectionThreshold(). Minimum threshold <= 0 seconds");
}

// The following line does it all BUT it does not allow for feedback to the user.
var outputEvents = events.Where(ev => (ev.DecibelDetectionThreshold == threshold)).ToList();
return outputEvents;
}

/// <summary>
/// Filters lists of spectral events based on their duration.
/// Note: The typical sigma threshold would be 2 to 3 sds.
Expand Down Expand Up @@ -145,17 +165,19 @@ public static List<EventCommon> FilterOnDuration(List<EventCommon> events, doubl

var filteredEvents = new List<EventCommon>();

var count = 0;
foreach (var ev in events)
{
count++;
var duration = ((SpectralEvent)ev).EventDurationSeconds;
if ((duration > minimumDurationSeconds) && (duration < maximumDurationSeconds))
{
Log.Debug($" Event accepted: Actual duration = {duration:F3}s");
Log.Debug($" Event{count} accepted: Actual duration = {duration:F3}s");
filteredEvents.Add(ev);
}
else
{
Log.Debug($" Event rejected: Actual duration = {duration:F3}s");
Log.Debug($" Event{count} rejected: Actual duration = {duration:F3}s");
continue;
}
}
Expand Down Expand Up @@ -197,12 +219,16 @@ public static List<EventCommon> FilterEventsOnSyllableCountAndPeriodicity(List<E

var filteredEvents = new List<EventCommon>();

int count = 0;
foreach (var ev in events)
{
count++;

// ignore non-composite events
if (ev is CompositeEvent == false)
{
filteredEvents.Add(ev);
Log.Debug($" Event{count} accepted one syllable.");
continue;
}

Expand All @@ -225,12 +251,12 @@ public static List<EventCommon> FilterEventsOnSyllableCountAndPeriodicity(List<E
}

string strArray = DataTools.Array2String(actualPeriodSeconds.ToArray());
Log.Debug($" Actual periods: {strArray}");
Log.Debug($" Event{count} actual periods: {strArray}");

// reject composite events whose total syllable count exceeds the user defined max.
if (syllableCount > maxSyllableCount)
{
Log.Debug($" EventRejected: Actual syllable count > max: {syllableCount} > {maxSyllableCount}");
Log.Debug($" Event{count} rejected: Actual syllable count > max: {syllableCount} > {maxSyllableCount}");
continue;
}

Expand All @@ -240,6 +266,7 @@ public static List<EventCommon> FilterEventsOnSyllableCountAndPeriodicity(List<E
// there was only one event - the multiple events all overlapped as one event
// accept this as valid outcome. There is no interval on which to filter.
filteredEvents.Add(ev);
Log.Debug($" Event{count} accepted - only one syllable");
}
else
{
Expand All @@ -255,12 +282,12 @@ public static List<EventCommon> FilterEventsOnSyllableCountAndPeriodicity(List<E
// Require that the actual average period or interval should fall between required min and max period.
if (actualAvPeriod >= minExpectedPeriod && actualAvPeriod <= maxExpectedPeriod)
{
Log.Debug($" EventAccepted: Actual average syllable interval = {actualAvPeriod:F3}");
Log.Debug($" Event{count} accepted: Actual average syllable interval = {actualAvPeriod:F3}");
filteredEvents.Add(ev);
}
else
{
Log.Debug($" EventRejected: Actual average syllable interval = {actualAvPeriod:F3}");
Log.Debug($" Event{count} rejected: Actual average syllable interval = {actualAvPeriod:F3}");
}
}
}
Expand Down
Loading

0 comments on commit 049c901

Please sign in to comment.