Teaching Six Security Cameras to Birdwatch
I have six UniFi cameras around the house, and every one of them has a microphone I had never once used for anything. Meanwhile there's a garden full of birds I can't identify and, if the flattened lawn is anything to go by, a nightly hedgehog and fox traffic jam. So when I stumbled on BirdNET-Go, a self-hosted analyser that runs the BirdNET model against live audio, the plan wrote itself: point it at the camera microphones and let the machines do the birdwatching.
There's a Proxmox community script for it, so I assumed the install was the boring part. It wasn't.
Two failed containers before breakfast
The first deploy died silently on an interactive storage prompt. It turns out newer community scripts read the storage pool from the per-app vars file, not the global defaults I'd been relying on for thirty other containers. The second deploy came up with no default gateway, because in appdefaults mode the global gateway setting was ignored too. The installer then sat spinning at 80% CPU probing Debian mirrors it couldn't resolve. Third time, with all six keys in the app file, it built in three minutes.
Getting the audio in was the pleasant surprise. The UniFi Protect integration API accepted my existing controller key, and one call per camera returned ready-made RTSPS URLs:
GET /proxy/protect/integration/v1/cameras/<id>/rtsps-streamSix streams, six ffmpeg processes stripping the video, and within a minute the dashboard had a wren on it.
The morning of the impossible birds
The first dawn chorus produced over a thousand detections, and the species list was a giveaway. A Dipper, which is a river bird. Sixty-one Redwings in early September, six weeks before any Redwing arrives. A Kingfisher, a Gadwall, a Bearded Tit. This was road noise. Tyre hiss on tarmac is broadband, and to a model trained on rivers it sounds like a Dipper.
The real amplifier was subtler. The average confidence was 0.5 against a 0.7 threshold, which should be impossible until you learn about dynamic thresholds: one confident false hit drops that species' bar to 0.2 for a day, and then everything similar sails in underneath. Twenty-six species had their bars lowered by noise.
The fix was the project's own tuning stack, applied properly this time. Deep detection, so a species needs several corroborating hits within a window. A tighter range filter, which alone dropped the eligible list from 6,522 species to 139 and ejected the Dipper entirely. Per-species 0.9 thresholds for the three noise magnets still in range. A 250 Hz high-pass to strip engine rumble. And I wiped the database so everything after that point is a clean benchmark.
Then I over-did it. Level 3 of the false-positive filter wants five repeat hits, and I watched 29 candidates in a row get discarded, a dunnock reaching four of five and still binned. The privacy filter, set to mute anything that might be human speech, was also firing on traffic at a confidence of 0.06. Level 2 and a privacy threshold of 0.3 got the birds back.
The GPU that crashed, then didn't
The node has an Intel iGPU and BirdNET-Go can run inference on it through OpenVINO. Enabling it crashed the service in a loop: free(): invalid next size, a glibc heap-corruption abort inside the driver's device enumeration. I pinned it to CPU and went to bed.
The next morning I tried to actually fix it. A minimal C program doing the same enumeration was clean. Under valgrind: zero errors. Then the real binary stopped crashing too. Between the two days I'd changed the container's core count, the thread setting, and warmed the driver's shader cache, and I still don't know which one mattered. Both models now run on the iGPU and CPU usage fell from 30% to under 10%.
What I could fix was the failure mode. A driver abort during a probe shouldn't take the whole analyser down, so I wrote a patch that runs the device enumeration in a short-lived child process and falls back to ONNX Runtime if the child dies. Fault-injected an abort() into the enumeration shim, watched the child crash with the exact original signature, and the service shrugged and carried on. That's now an issue and a pull request upstream, and the two review bots found three genuine problems with my first version, which was humbling and useful in equal measure.
The hedgehogs, and the catch
For the night shift I added Google's Perch v2 as a second model, whose label set includes Red Fox, European Hedgehog and Badger. Nocturnal taxa go on the daylight filter's list so a daytime "fox" gets discarded as noise.
And then the catch, found while trying to work out why birds I could hear from the doorstep scored a miserable 0.55. I ffprobed the camera streams. Every quality tier carries the same audio: AAC, mono, 16 kHz. That caps the signal at 8 kHz, and a great deal of songbird energy lives above that. The model is hearing a muffled, top-sliced version of the garden, and no amount of tuning recovers frequencies that were never captured.
So the cameras will keep the corvids, the starlings and the foxes, all comfortably under 8 kHz. For the wrens and goldcrests the next step is a proper 48 kHz microphone on a small board in the garden, served back as a seventh stream. The hedgehogs, I'm told, are due tonight.