Logical audio sensor plugin

[adlarkin]

Here's my design/approach for audio sensors in ign-gazebo3 (ignition citadel). For more information about the approach being taken, take a look at this discussion, and in particular, this comment.

The idea is to create source and microphone objects, with the following key characteristics:

• source
  • volumelevel: The volume level emitted from the source (ranges from 0.0 to 1.0). The volume will decrease as the distance from the source increases. The way the volume decreases is based on the attenuationshape and attenuationfunction of the source (currently, the only values for these parameters are sphere and linear, respectively).
  • playduration: How long the audio source plays for (in simulation seconds). 0 means infinite play time.
• microphone
  • volumethreshold: The minimum volume level the microphone can detect audio at (ranges from 0.0 to 1.0).

At the end of each simulation step, microphones check to see if they have detected audio emitted from any of the audio sources that are currently playing. Audio sources can be started/stopped through the SDF file and/or through service calls. If a microphone can detect a source, a message is published to the microphone's "detection topic". This message contains the ID of the source that was detected, along with the volume level that was heard at the microphone's location.

Take a look at the tutorial to learn how to use/test this plugin.

---

TODO:

• System Plugin Integration Test
• Do something useful with microphone detections (TODO for this is linked here)
• Write a tutorial

Signed-off-by: Ashton Larkin [email protected]

[codecov]

Codecov Report

Merging #401 (1a9f5d0) into ign-gazebo3 (1672d2b) will increase coverage by 0.22%.
The diff coverage is 87.18%.

@@               Coverage Diff               @@
##           ign-gazebo3     #401      +/-   ##
===============================================
+ Coverage        77.32%   77.54%   +0.22%     
===============================================
  Files              205      208       +3     
  Lines            11055    11375     +320     
===============================================
+ Hits              8548     8821     +273     
- Misses            2507     2554      +47     

Impacted Files	Coverage Δ
...al_audio_sensor_plugin/LogicalAudioSensorPlugin.cc	80.19% <80.19%> (ø)
include/ignition/gazebo/components/LogicalAudio.hh	100.00% <100.00%> (ø)
...ystems/logical_audio_sensor_plugin/LogicalAudio.cc	100.00% <100.00%> (ø)
src/SimulationRunner.cc	93.59% <0.00%> (-1.17%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1672d2b...1a9f5d0. Read the comment docs.

[adlarkin]

@iche033 @nkoenig This PR is now ready for initial review. I tried to give a detailed overview and example usage in the PR description. If anything is still unclear, please let me know.

Once I address your initial feedback, there are a few more things I'd like to do before merging this:

1. Use ignition topics (or something similar) for the following:
   • whenever a microphone has detected an audio source
   • whenever an audio source's playing state has changed (either start->stop or stop->start)
2. Verify that I am computing simulation seconds correctly for an audio source's play duration. Right now, I am basically treating the dt of each simulation step as a simulation second - is this correct? I'm not sure if it is, because based on some tests I ran, it looks like setting an audio source's play duration to something like 100 results in 100 simulation seconds being reached almost instantaneously (I'd think that 100 simulation seconds wouldn't be instantaneous). If I need to fix how I calculate simulation seconds, do you have any resources/tips for how to do this?

[adlarkin]

@iche033 @nkoenig I actually just realized that the current implementation will not work if we have sources/microphones attached to different models, because this would result in separate <plugin>...</plugin> tags for each source/microphone (the current implementation only handles one set of <plugin>...</plugin> tags). I'll re-mark this PR as a draft while I fix this (I'm going to make use of ecm.each()), and will re-mark the PR as ready for review once that fix is in.

[adlarkin]

@nkoenig @iche033 @chapulina @mjcarroll this PR should be ready for actual review this time. I put up-to-date instructions in the PR description, and also updated the example SDF file to allow for moving sensors.

Let me know if you have any questions!

[adlarkin]

Here are a few lingering questions that I have after my initial draft.

[adlarkin]

Is the logical_audio namespace okay here? Or should it be a sub-namspace of components?

[adlarkin]

Is the logical_audio namespace okay here? Or should it be a sub-namspace of systems?

[iche033]

since we're grouping a bunch of functions specific to logical audio, I think maybe we don't want to pollute the systems namespace so I'm fine with either sticking with logical_audio namespace or having it as a sub-namespace of systems

[adlarkin]

Sounds good, I'll leave the logical_audio namespace as is for now. If someone else points out a reason to change it, then I can update it accordingly.

[adlarkin]

I modified the serialization for an audio source's play duration in 53467b2 based on the conversations had in #433.

[iche033]

since we're grouping a bunch of functions specific to logical audio, I think maybe we don't want to pollute the systems namespace so I'm fine with either sticking with logical_audio namespace or having it as a sub-namespace of systems

[nkoenig]

Would it be possible to move the logical audio source and sink to the ign-sensors library? I'd like to keep sensors together for consistency.

[adlarkin]

Would it be possible to move the logical audio source and sink to the ign-sensors library? I'd like to keep sensors together for consistency.

I think this could be possible, but if you take a look at the original brainstorming conversation that motivated this PR, I decided to make this an ignition gazebo system plugin so that we wouldn't have to make any changes to the SDF spec. I think another reason for not placing this in ign-sensors is because this is a "logical" audio sensor, not a "real" audio sensor (i.e., no sound is actually emitted to a device with the logical audio sensor) - I think the plan was to have the "real" audio sensor be placed in ign-sensors whenever the development for it begins. That being said though, I guess that we could have both a "logical" audio sensor and "real" audio sensor in ign-sensors.

Should I look into what would be involved in moving this over to ign-sensors? Or should we leave it here as a system plugin?

Edit: There also seem to be other sensors as system plugins, like the logical camera, altimeter, and magnetometer.

[adlarkin]

@nkoenig @iche033 this is ready for another round of review. Here are a few notes about what has been added/modified since the last round of reviews:

1. Models can have multiple sources and/or microphones attached to them. Sources and microphones still need an ID attached to them so that there's a way to identify which source/microphone you'd like to interact with when dealing with services/topics. However, ID uniqueness is no longer enforced at the global scope, but rather by the audio object type at the model scope - so, for example, if model "a" has a microphone attached to it with ID "1", no other microphones attached to "a" can have an ID of "1" (but other models can have an ID attached to it with an ID of "1"). A source can be attached to "a" with an ID of "1" though since sources and microphones are different audio objects.
2. Topic/service names are now generated using the scopedName method, which avoids potential collisions of microphones/sources attached to different models with the same ID.
3. A tutorial has been added.
4. I've added an integration test that makes sure a microphone close to a playing source can detect it and a microphone far from this source cannot, but there are probably still some more test cases I need to handle (let me know which scenarios from the list below you think I should test, and feel free to bring up any other test cases I may be missing):
   • make sure a microphone does not hear a source if the microphone is too far away (added this test in 6748cc4) and/or the source is not playing
   • test a source's play/stop service calls (added in 163df3d)
   • test that a source's play duration is accounted for correctly (if the source doesn't have an infinite play duration)
   • test multiple audio objects attached to a model
   • test invalid SDF configurations

[iche033]

the design of the parameters looks good to me. Nice tutorial. I left a few comments.

[nkoenig]

Just a few minor comments. Looks good!

[iche033]

looks good. Just a couple small comments