For a nonprofit's field teams · via Camel Case Labs
Voice agent for field teams
A mobile app where field staff log their visits by speaking, and the fixes that made the agent reliable in the field.
liveMay 2026 – Present
My roleMobile app, matching, evaluation, and production fixes (a teammate built the agent's backend orchestration)
- locations matched from speech
- 922
- locations matched from speech
- of live field testing behind the reliability fixes
- 3 weeks
- of live field testing behind the reliability fixes
- released to field staff
- Google Play
- released to field staff
01The problem
Field staff visit sites all day and had to log every visit in forms afterward. A voice agent lets them just say what happened. The hard part was making it dependable: speech-to-text mishears place names, and an agent that logs the wrong site, or says it logged something it didn't, is worse than no agent at all.
02What I built
- The React Native app: record a visit, transcribe it with Groq Whisper, and let the agent draft the activity log for review
- A matcher that resolves a spoken location against 922 records by checking how many of the name's distinctive words were heard, ignoring filler words, and tolerating transcription typos; it had only been scoring the first 500 of 900+ sites, so I fixed that and backed it with 27 regression tests
- Confidence and ambiguity thresholds: when two locations score too close, the app asks the user to pick instead of guessing
- The start of a read-only evaluation harness: production failures turned into test cases, replayed against a snapshot of production data, with a runner and a review page
- Fixes for failures found in live use: the agent reporting outreach it never logged, action types hardcoded instead of read from the database, and permission gaps that locked out 2 manager roles
03Key decisions
Ask instead of guess
- Why
- A transcription error like "Southlight" for "Southlake" can land on the wrong site with full confidence.
- Result
- Below a confidence threshold, or when two sites score too close, the app shows what it heard and lets the user choose.
Turn production failures into test cases
- Why
- Every prompt or tool change risked breaking something that worked the day before.
- Result
- Each failure became a test case replayed against a snapshot of production data, so a fix is checked against the exact case that broke.
Handle errors by what the user can do about them
- Why
- A network blip and a failed recording need different fixes, but both showed the same error.
- Result
- Network failures offer a retry; failed recordings ask for a new one. Retry now resends the exact saved request, which fixed a bug where it resubmitted the previous report.
Make recording recover on its own
- Why
- Three separate bugs could leave the microphone stuck until the app restarted, and a 2-minute timer silently cut off long recordings.
- Result
- The recorder always releases cleanly, and long recordings are carried through to a plan instead of thrown away.
04Results
- Released on Google Play for field staff
- The agent no longer reports work it didn't do, and both manager roles have access again
- Recording, retry, and cancel behave predictably after three weeks of field-test fixes
05What I learned
- An agent that sounds right but logs the wrong thing loses trust faster than having no agent. Evaluation has to use real cases.
- Most of the failures weren't the model. They were everything around it: recorders, retries, sessions, and permissions.
06Stack
- React Native
- Expo
- FastAPI
- Groq Whisper
- LLM tool calling
- pytest
want the longer version?
I'm happy to walk through how I built it.