Computer Vision / Edge AI / Retail Intelligence
Restaurant Vision Analytics
A multi-camera retail intelligence platform that transforms live video and operational data into customer-flow, occupancy and behavioral analytics.
The problem
Restaurant cameras and operational systems capture different parts of the same activity. Video shows how a physical space is used; POS and delivery systems record transactions. Bringing those signals together requires both spatial interpretation and backend integration.
I independently architected and built a multi-camera computer vision platform with a live pilot and validation deployment at Dach & Nona.
From cameras to a shared floor view
The pipeline combines real-time detection, tracking, and cross-camera re-identification. Homography-based world-space fusion maps activity into a shared spatial frame to maintain continuity across overlapping views.
- 01 IP Cameras
- 02 Edge Video Processing
- 03 Detection
- 04 Tracking
- 05 Re-ID / Identity Fusion
- 06 World-Space Mapping
- 07 Analytics Engine
- 08 Store Analytics
Parallel integrations
- POS / Petpooja
- Zoho
- Delivery platforms
Vision-derived occupancy, dwell and flow are joined with POS and operations data in the backend services.
The implementation uses YOLO12s detection, Kalman filtering and Hungarian matching for tracking, and OSNet/ArcFace with FAISS indexing for re-identification and facial recognition.
Operational analytics
The spatial layer derives:
- Dwell time and table occupancy.
- Customer movement flows and zone-level activity.
- Customer retention and continuity across camera views.
These outputs depend on the quality of detection, identity matching, and spatial calibration in the deployed setting.
Integration and infrastructure
Django and FastAPI services connect vision-derived customer analytics with POS and operational data from Petpooja, Zoho, and delivery systems through REST APIs.
The edge/cloud architecture uses Docker, PostgreSQL, GPU infrastructure, CI/CD, Cloudflare security, and production monitoring.
Engineering constraints
The main challenges include maintaining identity across camera views, mapping observations into a consistent floor plan, and joining vision events with operational data. Camera coverage and calibration, compute resources, and traffic conditions all shape deployment behavior.
A measured two-camera pilot
On 30 July 2026, a 59.7-minute live soak collected 359 health samples from two simultaneous RTSP cameras, each at 2688 × 1520 resolution. The worker ran on a Vast.ai NVIDIA GeForce RTX 3060 with 12 GB VRAM, using ONNX Runtime CUDA and OpenCV software decode. A serial scheduler processed one camera tick after the other; this was not a parallel camera pipeline.
| Camera | Active tracks | Median worker tick | p95 worker tick |
|---|---|---|---|
| Camera 1 | 8 | 64.2 ms | 108.7 ms |
| Camera 2 | 13 | 82.7 ms | 140.7 ms |
The timer covers one camera’s hot-loop wall time: latest-frame retrieval, detection when due, tracking, face recognition, and that camera’s projection, identity, and occupancy processing. It excludes RTSP grab/decode, preview JPEG encoding, cross-camera fusion, central sync, and the UI. Median capture-frame age during this hour was 37 and 39 ms respectively.
These are worker-tick observations, not camera-to-dashboard latency or a service-level guarantee. The measurement covers two 2.7K-resolution streams, not thousands of cameras. The 8 and 13 active tracks describe this measurement window; higher visitor counts from other observations should not be attached to these latency figures.
A separate 47-minute run measured approximately 24 ms for a gated YOLO call, with preprocessing performed in the capture thread and body-ReID association disabled. That narrower detector timer is not the full camera tick and is not combined with the table above.
Current status
This is a live pilot and validation deployment. The next step for another location is a review of its cameras, layout, hardware, and integration requirements.
See the product overview or discuss a pilot.
Technology
Related