01 / THE PROBLEMHow much computation can be avoided while preserving accuracy and deadlines?
An always-on audio classifier sees many overlapping segments of the same event, yet conventional execution repeatedly pays for deep inference. Early exits reduce work, but an exit decision made inside the network arrives after some computation has already happened. In an MCU that stays active between inference windows, finishing sooner also does not automatically save proportional energy. The useful decision is both how far to execute and how slowly the remaining work can run.
02 / THE APPROACHPredict depth early, then use the available time
01Check temporal consistency before classification
After feature extraction and quantization, a lightweight predictor compares the current audio tensor with a retained reference using L1 distance. A class-specific threshold identifies segments likely to remain in the same event, and a lookup table supplies an appropriate exit depth. The reference is retained while the stream stays consistent.
02Use current-segment evidence at event transitions
When the distance exceeds the threshold, an in-classifier predictor uses the first-exit representation to choose the remaining depth. The resulting tensor and class refresh the reference state. This fallback handles changing or highly variable sounds without forcing every segment to reuse the previous decision.
03Select a deadline-feasible operating point
As soon as the remaining workload is known, choose the lowest measured voltage/frequency setting that can finish it before the segment deadline. HEED accounts for feature extraction, prediction, classifier blocks, and switching overhead; the evaluated MCU offers four operating frequencies from 20 to 160 MHz.
03 / MY CONTRIBUTIONPrediction, DVFS, and MCU evaluation
As first author, I developed lightweight pre-classifier workload prediction, the two-stage exit-depth mechanism, and proactive DVFS integration. I deployed the framework on an STM32U575ZI Cortex-M33 MCU and evaluated ESC-50-derived audio streams.
Embedded AIEarly exitsDVFS
04 / EVALUATIONSetup and results
I deployed HEED on an STM32U575ZI Cortex-M33 MCU using a six-exit YAMNet-256 classifier. Evaluation uses ten ESC-50 classes arranged into 300-second streams with class changes every 5 or 15 seconds. Audio windows are 960 ms long with a 720 ms hop and deadline. Segment energy is calculated from measured MCU power and block latency, including voltage/frequency transition overhead.
- Lower inference energy
- 16.17–17.55%
- Versus EENet at matched accuracy across the two stream settings.
- Missed deadlines
- 0
- Across the evaluated streams with a 720 ms per-segment deadline.
- Additional runtime memory
- ≈6 KiB
- For the prediction state and supporting data structures.
05 / OUTCOME & SCOPEEarlier workload knowledge reduces energy
At matched accuracy, HEED improves computation savings by 14.83–16.73 percentage points over EENet. The two-stage design uses temporal similarity when it is useful and falls back to current-segment prediction when the sound changes. Its energy benefit comes from coordinating the chosen exit with the MCU operating point, rather than treating fewer operations as an energy result by itself.