NeuroNav
An on-device AI safety system that warns blind pedestrians about silent electric and hybrid vehicles before they become a near-miss. I led it end to end, from research with blind users to a working prototype on their phones.
NeuroNav turns a smartphone into an electronic eye: on-device YOLOv8 detects vehicles and translates danger into synchronized voice and haptic alerts, at crosswalks, parking lots, turn lanes, and quiet intersections where auditory cues fail.
Phase 1 answers the question every safety product has to answer first: will blind users understand, trust, and act on AI-generated alerts? Perception benchmarking and field deployment come next.
Product Snapshot
- • Problem: Silent electric vehicles break the sound-based survival strategies blind pedestrians rely on.
- • Solution: On-device AI + voice-first multimodal alerts.
- • Users: 8 blind and visually impaired participants.
- • Result: 100% understood the system’s purpose within 2 minutes.
- • Stage: Interface validated, ready for perception benchmarking and field trials.
- • Constraint: Safety-critical latency under 500ms with near-zero tolerance for false negatives.
"This would boost my confidence walking around outside."
- Participant P6, Iowa Department for the Blind User Testing
This was a trust problem disguised as a detection problem. The hard part was designing a human-AI interaction model that could earn trust where one missed alert ends adoption permanently.
- Role Lead designer and developer across user research, UX architecture, AI interaction design, implementation, and testing.
- Project Type 0→1 AI safety product, designed and built end to end
- Timeline March 2025 – Dec 2025
- Tech Stack Unity 6 LTS, C#, AR Foundation, YOLOv8, Barracuda Inference
- Collaborators Iowa Department for the Blind (Testing Partner)
- Status Phase 1 Complete • Interface Validated • Ready for Field Trials
- Focus Accessibility, safety-critical AI, multimodal interaction, trust calibration
- GitHub View Repository →
The hard calls, and how I made them
Every trade-off below resolved toward trust rather than capability.
Cloud inference was the easier engineering path. Latency, connectivity dependence and remote video processing each undermine trust in a system people rely on to cross a street.
How I solved itI rejected cloud inference and ran all safety-critical detection on-device, reserving the cloud for slower workflows like analytics and retraining.
Principle: Privacy reinforces reliability, on-device processing strengthens both.Responsiveness fought battery life. Higher frame rates improved detection responsiveness but drained the battery, and a dead phone is a failed safety device.
How I solved itI settled on 10 FPS as the balance point for pedestrian speed, with adaptive frame processing that eases off when the user is stationary.
Principle: Sustainable performance beats peak specification.Users needed three facts, but not all at once. Direction, distance and urgency all matter, delivered simultaneously to someone mid-crossing, they become noise. Testing later confirmed early alerts were too verbose.
How I solved itProgressive disclosure: minimal initial alerts carrying only what's actionable, with deeper context available on request through voice query.
Principle: Context determines content, actionable direction and urgency beat richer description.Trust in this category is binary. Users either trust the system completely or stop using it. A single false negative means permanent abandonment; participants said they'd rather have extra false alarms than one miss.
How I solved itThe core philosophy became: when in doubt, alert. The threshold was tuned toward over-warning rather than staying quiet.
Principle: Safety over convenience, false positives are recoverable, false negatives are not.Design principles & psychology at work
The principles that came out of Phase 1, and now carry into everything I build:
- Trust is binaryThere is no partial adoption in safety-critical products, the system is trusted or abandoned.
- Multimodal redundancyAny single sensory channel can fail in noise, rain or sensory overload, so feedback is duplicated across voice and haptics.
- User agency builds adoptionCustomisation measurably increased trust; people want meaningful control over safety technology.
- Design with, not forEight blind participants shaped the product rather than validating it at the end.
- Cognitive load is a safety constraintPhase 1 evaluated comprehension and load, because an accurate system still fails if it can't be interpreted in motion.
- Sensory translationThe breakthrough was reframing a silent vehicle as a visual threat needing conversion into language users already trust.
01The Challenge: Silence Became a Hazard
Electric and hybrid vehicles have created an invisible accessibility crisis. A Vision Australia and Monash University Accident Research Centre study found that 35% of blind and low-vision respondents had experienced a collision or near-collision with an electric or hybrid vehicle.
The danger peaks at low speeds, exactly when pedestrians cross streets, move through parking lots, or navigate intersections. In the moments where engine noise used to communicate safety, silence now creates ambiguity.
Blind pedestrians rely on traffic pattern recognition and sound mapping. Silent vehicles break those strategies at the exact moment a correct decision matters most.
User Voices from Research
"Those stupid quiet cars… you think it's safe, then zap."
"If it's electric, I won't even know it's there."
"I've stepped out thinking it's safe when it really wasn't."
The Gap
Existing tools cover wayfinding, obstacle avoidance, or on-demand description. None is a proactive, always-on safety layer for silent moving vehicles, the gap that produces real fear and near-misses.
Pain Point Severity Mapping (8 Participants)
I mapped every breakdown by frequency, emotional intensity, and workaround effectiveness. Three were critical with no reliable workaround: silent electric vehicles, snow-related sensory deprivation, and close calls at intersections.
- • Silent electric vehicle detection was mentioned by 8/8 participants (100%).
- • Snow conditions caused complete navigation breakdowns for 7/8 participants.
- • Close calls at intersections were common for 6/8 participants, with no scalable strategy beyond “walk slower” or “be extra careful.”
- • GPS lag, overhead obstacles, turn lane prediction, and uncontrolled crossings informed roadmap opportunities beyond the MVP.
02Why This Matters: A Systems Perspective
This is a systems problem spanning transportation, urban design, AI trust, public safety, and equity. Electric mobility solved one problem and quietly created another.
Why NeuroNav Is Different
Most products for blind users are generic GPS routing or reactive visual description. NeuroNav is proactive, real-time hazard detection for high-stakes moments.
- • Built specifically for the silent vehicle crisis.
- • Proactive hazard monitoring instead of reactive image interpretation.
- • Architected around sub-500ms latency and safety-first decision making.
- • Designed with blind users from day one, making trust and co-design part of the product moat.
Core User Needs Identified
Instant Detection
Threat-level communication in real time as vehicles approach.
Absolute Reliability
One false negative can destroy adoption; reliability is the product.
Independence Without Fear
Confidence in public movement without constant anxiety.
Multimodal Alerts
Redundant feedback that still works in noise, rain, and sensory overload conditions.
Hands-Free Interaction
The system cannot interfere with a cane, guide dog, or existing navigation strategies.
Sensory Translation
Translate a visual threat into language users already trust and can act on immediately.
03Research: How Blind Pedestrians Actually Cross
8 contextual inquiry interviews with blind and visually impaired participants, observing real navigation strategies, failure modes, trust thresholds, and adoption patterns.
Four Anchor Personas
Four anchor personas, each a distinct safety profile. Every major decision traces back to one of them.
- • Marcus – The Silent Vehicle Survivor. Hard-of-hearing cane user who relies heavily on traffic sound patterns. NeuroNav had to prioritize clear directional warnings over rich descriptions.
- • Sarah – The Zero-Tolerance Skeptic. A single false negative would mean permanent abandonment. NeuroNav had to expose uncertainty and preserve user control.
- • David – The Overhead Obstacle Magnet. His needs pushed the roadmap toward broader hazard detection beyond vehicles alone.
- • Alex – The Construction Zone Navigator. His context surfaced the need for dynamic environmental awareness and future community reporting features.
Critical Breakdowns Observed
- • Silent electric vehicles approaching from any direction
- • Turning vehicles that change trajectory unpredictably
- • Weather conditions masking audio cues
- • Multi-lane confusion at complex intersections
- • Inability to judge vehicle distance, speed, or trajectory
User-Expressed Requirements
- • Detection accuracy target of at least 90%
- • False positive rates below 10%
- • Combined haptic and sound alerting
- • A simple, learnable interface
- • Adverse condition support
- • Offline capability
How Blind Pedestrians Actually Cross Intersections
Shadowing participants at real intersections revealed a structured four-stage crossing model, and exposed exactly where silent vehicles break it.
- Pre-approach: Determine whether the intersection is controlled or uncontrolled by reading traffic patterns.
- Corner alignment: Use cane and curb feedback to align the body before crossing.
- Crossing decision: Listen for reliable traffic cues that indicate whether movement is safe.
- During crossing: Maintain alignment while continuously monitoring for new threats.
Observed failure modes:
- • Misalignment causing diagonal drift into traffic lanes
- • Silent vehicles breaking the auditory model entirely
- • Turn lane vehicles violating assumptions about parallel traffic
- • Quiet streets providing too little information to cross confidently
Technology Adoption & Abandonment Pattern
Assistive tools spread through trusted networks, not app stores, and the trial window is brutal: fail in the first few uses and the product is abandoned for good.
Key adoption barriers
- • Complexity and steep learning curves
- • Lag during real movement
- • Daily errors that erode trust
- • Information overload
Successful adoption factors
- • Simple flows with only essential actions
- • Clear communication of orientation and situation
- • A product that feels usable on day one
Current App Usage & Satisfaction
I catalogued the tools participants use, keep, and abandoned, so NeuroNav complements them instead of duplicating them.
- • BlindSquare - valuable for POI awareness but described as too slow.
- • Google/Apple Maps - used for walking directions, but satisfaction was low.
- • Be My Eyes / Seeing AI - high value for visual description, but not designed for active crossing safety.
- • Nearby Explorer - useful but costly relative to feature depth.
- • Transit tools - some were abandoned after reliability problems.
- • Soundscape - remembered fondly, but also described as overwhelming.
- • Meta Glasses + AI - attractive hands-free interaction, but cost remains a barrier.
- • NaviLens - strong for indoor use cases, limited by deployment footprint.
Feature Preferences That Shaped the MVP
- • Real-time vehicle detection with direction was the unanimous top need.
- • Distance-based urgency scaling mattered as much as detection itself.
- • Voice + haptics clearly outperformed tones alone.
- • Participants preferred extra false positives over a missed real threat.
- • Personalization mattered, but not at the cost of consistency and predictability.
04The Solution and the Decisions Behind It
On-Device vs. Cloud
On-device only: latency, connectivity dependence, and remote video processing all undermine trust. It was a product trust decision, not just a technical one.
Battery Life vs. Detection Quality
10 FPS balances responsiveness against battery, with adaptive processing when stationary.
Information Richness vs. Cognitive Load
Progressive disclosure: minimal initial alerts, deeper context on request.
Trust Brittleness
Trust is binary, so: when in doubt, alert. False positives are recoverable; false negatives are not.
The Solution
The product detects vehicles, estimates distance, infers direction and urgency, and communicates it through synchronized multimodal feedback. Phase 1 proved users could interpret and trust that layer under pressure.
Real-Time Vehicle Detection Pipeline (<500ms)
- • Image capture from a 30fps camera feed
- • Pre-processing with ROI extraction, normalization, and motion filtering
- • YOLOv8-nano inference on-device
- • Distance estimation using monocular depth and bounding-box heuristics
- • Direction and velocity tracking frame-to-frame
- • Threat scoring based on distance, speed, and approach angle
- • Multimodal alert generation matched to urgency and context
Key Technical Breakthroughs
- • Context-aware orientation for left/right/front/behind alerts
- • Three haptic severity tiers users learned quickly
- • VoiceOver bridge for critical alert interruption
- • Simplified semantic hierarchy for screen-reader-first interaction
- • GPU inference scheduling for consistent performance
- • Tight synchronization across audio and haptic outputs
Voice Interaction Model
The product works as an ambient safety layer first, with conversational clarification layered on top. The model is simple by design:
- • Ambient alert
- • User voice query
- • Short contextual AI response
- • Automatic return to passive monitoring
Supported queries include: “What’s ahead of me?”, “Where is the vehicle coming from?”, “How far is it?”, “Is it safe to cross?”, and “Repeat last alert.”
05Technical Architecture
On-Device First, Cloud-Augmented Architecture
All safety-critical detection, urgency scoring, and alerts run on the device. Optional cloud services handle slower workflows like analytics and retraining, protecting privacy and reliability.
- • Application layer: UI manager, VoiceOver/TalkBack interface, and haptics controller
- • Core logic layer: Alert manager, context detector, distance estimator, and direction tracker
- • Sensor layer: Camera, GPS, and IMU fusion
- • Data layer: Local preferences, logs, and offline cache
- • Backend services: Training, reporting, and analytics support
06User Testing Results
Phase 1 evaluated trust calibration, alert comprehension, cognitive load, accessibility, and decision confidence, because an accurate system still fails if people can't interpret it in motion.
The following video captures a user testing session with visually impaired participants at the Iowa Department for the Blind, where we evaluated the multimodal interaction layer including voice announcements, haptic feedback patterns, and customization controls.
What Testing Proved
- • Users quickly understood the system’s purpose
- • Voice + haptics outperformed tone-heavy alerting
- • Customization directly increased trust and perceived usability
- • Accessibility architecture, not just detection, determines adoption
What Needed Correction
- • Audio tones overloaded sensory channels for 40% of participants
- • TalkBack integration emerged as an adoption prerequisite
- • Early alerts were too verbose for fast decision moments
- • Timing conflicts between modalities had to be tightened
Participant Quotes
"The voice is perfect, just tell me left, right, and distance."
- P2
"Tones get too much. Let me turn them off."
- P4
What 66% Adoption Intent Actually Tells Us
The two non-adopters raised addressable barriers, not a fit failure. P1 felt safe with monocular vision and a cane, outside the primary segment. P2's technology anxiety exposed an onboarding gap, not a rejection of the value.
The four adopters' intent hinged on TalkBack/VoiceOver integration, a solvable engineering dependency. 100% understood the problem, and everyone who faced it wanted the solution: a usability gap, not a relevance gap.
07Final Design Principles
- Safety Over Convenience - false positives are recoverable; false negatives are not.
- Trust Is Binary - users either trust the system completely or stop using it.
- Multimodal Redundancy - any single sensory channel can fail.
- User Agency Builds Adoption - people want meaningful control over safety technology.
- Context Determines Content - actionable direction and urgency beat extra description.
- Privacy Reinforces Reliability - on-device processing strengthens both.
- Design With, Not For - users shaped the product, not just validated it.
Immediate Next Steps
- • Full TalkBack and VoiceOver integration
- • Haptic pattern validation study
- • Natural TTS voice selection system
- • Alert throttling refinement
Scaling Path
- • Environmental mode switching
- • User-facing ML confidence scoring
- • Wearable integration pathways
- • Controlled field reliability studies
Long-Term Vision
- • Historical analytics for dangerous intersections
- • Open-source release of core architecture
08Learnings
NeuroNav taught me that accessibility is not a layer you add after the fact. It changes the product from the ground up. It changes what counts as clarity, speed, trust, and success.
I started with technical confidence in mobile neural networks, AR, and multimodal interfaces. What I underestimated was how deeply visual bias shapes product assumptions.
The breakthrough came when I stopped thinking about accessibility features and started thinking about sensory translation. A silent vehicle is a visual threat. My job was to convert that threat into audio and haptic language users could trust fast enough to act on.
In safety-critical AI, the interface isn't decoration around the model. It's the contract between human judgment and machine output.
Key Takeaway
When P6 said “this would boost my confidence walking around outside,” I understood that confidence was the real deliverable. Phase 1 proved the interaction layer could create it. The next phase must prove the perception stack deserves it.