Hey Willow Dataset

[StuartIanNaylor]

Ask users to record them speaking 'Hey Willow' and some phonetic pangrams.
Even better create a 'Willow recorder' image so you have samples from the device and maybe setup as USB or Websocket.

I created a very rough and ready cli dataset builder 'boutique' that maybe a spring board for the app ideas?
https://github.com/StuartIanNaylor/Dataset-builder

[kristiankielhofner]

Hey @StuartIanNaylor - great to see you here!

We're currently very heads down to prepare for our 1.0 release to address many of the pain points you'll see in our issue tracker, discussions, etc. The reality is we didn't have plans for additional wake word support for another month at least so the timing is perfect here.

That said, we (as well as the community) are definitely interested in wake training process that doesn't depend on Espressif (and could potentially lead to better outcomes)!

I'll be able to review this and any other provided resources in the next couple of days. Feel free to drop in any other references, existing projects, papers, etc that you think could be of benefit.

Thanks, the potential here is exciting!

[StuartIanNaylor]

I think https://github.com/espressif/tflite-micro-esp-examples and the upstream https://github.com/tensorflow/tflite-micro as that might give some real nice tricks like on device training where its actually on another device that is the local server to ship OTA.

You have to go back to the full tensorflow to get the readme https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/experimental/microfrontend/README.md

I think there might be some inspiration from the above in the Espressif blobs and could never work out if the LSTM layer was licenced and maybe why a blob. The BcResNet is compatible.
I would maybe think of an image as purely a recorder to test resource occupation as https://docs.espressif.com/projects/esp-sr/en/latest/esp32s3/audio_front_end/README.html as that is where AFE_CONFIG_DEFAULT() is defined

#define AFE_CONFIG_DEFAULT() { \
    // Configures whether or not to enable AEC
    .aec_init = true, \
    // Configures whether or not to enable BSS/NS
    .se_init = true, \
    // Configures whether or not to enable VAD (only for speech recognition)
    .vad_init = true, \
    // Configures whether or not to enable WakeNet
    .wakenet_init = true, \
    // Configures whether or not to enable voice communication (cannot be enabled when wakenet_init is also enabled)
    .voice_communication_init = false, \
    // Configures whether or not to enable AGC for voice communication
    .voice_communication_agc_init = false, \
    // Configures the AGC gain (unit: dB)
    .voice_communication_agc_gain = 15, \
    // Configures the VAD mode (the larger the number is, the more aggressive VAD is)
    .vad_mode = VAD_MODE_3, \
    // Configures the wake model. See details below.
    .wakenet_model_name = NULL, \
    // Configures the wake mode. (corresponding to wakeup channels. This should be configured based on the number of mic channels)
    .wakenet_mode = DET_MODE_2CH_90, \
    // Configures AFE mode (SR_MODE_LOW_COST or SR_MODE_HIGH_PERF)
    .afe_mode = SR_MODE_LOW_COST, \
    // Configures the internal BSS/NS/MISO algorithm of AFE will be running on which CPU core
    .afe_perferred_core = 0, \
    // Configures the priority of BSS/NS/MISO algorithm tasks
    .afe_perferred_priority = 5, \
    // Configures the internal ringbuf size
    .afe_ringbuf_size = 50, \
    // Configures the memory allocation mode. See details below.
    .memory_alloc_mode = AFE_MEMORY_ALLOC_MORE_PSRAM, \
    // Configures the linear audio amplification level. See details below.
    .agc_mode = AFE_MN_PEAK_AGC_MODE_2, \
    // Configures the total number of audio channels
    .pcm_config.total_ch_num = 3, \
    // Configures the number of microphone channels
    .pcm_config.mic_num = 2, \
    // Configures the number of reference channels
    .pcm_config.ref_num = 1, \
}

As the default is .afe_mode = SR_MODE_LOW_COST and swap to HIGH_PERF with WakeNet8, I keep procrastinating but the AFE itself is also interesting.
Likely I could supply test models, but the algs are likely more sensitive to resampling but the 16bit output resampled to 8bit for WakeNet9 would be a quicker test.