Behind the Moving Dot: Live Delivery Tracking with Kafka
How would you build Swiggy- or Zomato-style live tracking? Follow a GPS ping through Kafka, ordered processing, a location cache, and WebSockets—and see what happens when the network fails.
· 8 min read · Tanmay Singh
Your delivery partner takes a left. A moment later, the tiny scooter on your screen takes the same left.
It feels like you are watching a live video of a coordinate. You are actually watching a distributed system reconstruct a journey from incomplete, delayed observations.
Now imagine dinner rush: hundreds of thousands of phones sending updates, customers reopening the app, mobile networks dropping out, and analytics jobs asking for the same events.
How does that moving dot keep up—and where does Kafka fit?
What Swiggy and Zomato actually disclose
There is public evidence for Kafka at both companies:
| Company | Published evidence | What it establishes |
|---|---|---|
| Swiggy | A 2025 talk by a Swiggy architect describes Confluent Kafka supporting order tracking, predictions, and analytics. | Kafka supports multiple real-time workloads. |
| Zomato | Its New Year's Eve 2023 engineering report reports more than 450 million Kafka messages per minute across its event-driven infrastructure. | Kafka operates at substantial platform scale; that figure is not a GPS-update count. |
Neither source establishes the exact end-to-end location pipeline below. The rest of this article is a proposed architecture for Swiggy- or Zomato-style tracking. Topic names, settings, numbers, and failure policies are design examples, not their internal configuration.
1. Follow one location update
Let's name our delivery partner Asha. Her phone captures a position and sends it to our authenticated location API. The backend validates it and publishes a record to Kafka. Processing services update the latest known position; a push service forwards it to the customer watching Asha's assigned order.
Kafka sits behind the application servers. In this design, phones do not connect to brokers or join consumer groups. They use HTTPS and WebSockets; backend services own Kafka access.
Kafka supplies a retained event log that independent readers can process at their own pace. The cache serves the current view; the gateways maintain customer connections. Those are different jobs.
2. Start with an event you can trust
A latitude and longitude are insufficient. Our example event looks like this:
{
"schema_version": 1,
"partner_id": "partner_42",
"session_epoch": 7,
"sequence": 1842,
"captured_at": "2026-11-11T14:30:05Z",
"latitude": 12.9716,
"longitude": 77.5946,
"accuracy_m": 14,
"speed_mps": 6.2,
"heading_deg": 92
}
The API derives the partner identity from authentication rather than trusting an arbitrary payload ID, adds received_at, validates bounds, and rate-limits abusive clients. A server-issued session epoch increases when a new tracking session takes over, fencing off a disconnected old session.
The epoch and sequence establish observation order. Capture time helps measure age, but device clocks can be wrong: compare with server receipt time and reject implausible timestamps. Accuracy and movement checks help detect bad GPS fixes; they do not prove that a reported location is genuine.
Suppose we sample every four seconds while moving. That is an illustrative policy, not a company fact. Slow down when stationary, account for operating-system background restrictions, and bound the phone's retry buffer. On reconnect, prioritize a fresh observation; old buffered points can be tagged for history without dragging the live marker backwards.
3. Partition by the thing that must stay ordered
Use partner_id as the Kafka record key. With a stable partitioning scheme, Asha's records land in one partition, preserving their log order. Different partners spread across partitions and can be processed concurrently.
Why not use the order ID? A partner can carry two orders. One physical location stream should feed both authorized order views. Order assignment is a separate, versioned relationship.
Within a conventional consumer group, each partition is assigned to one consumer at a time. Separate tracking, ETA, and history groups each read the stream independently. Putting all three workloads in one group would divide records between them instead of giving each workload its own copy. See the Kafka consumer API.
Partition order is arrival order, not necessarily capture order. If sequence 1842 reaches Kafka before a delayed 1841, the processor must reject 1841 for the live view. Maintain a durable last-accepted (session_epoch, sequence) per partner, and preserve per-key order inside workers too.
Adding partitions can change key placement. Plan a controlled migration with version checks and state transfer instead of assuming old and new partitions preserve one uninterrupted order.
4. Size the stream before picking a partition count
Assume 200,000 active partners, one event every 4 seconds, and 200 bytes per encoded record:
| Quantity | Back-of-the-envelope result |
|---|---|
| Incoming events | 200,000 ÷ 4 = 50,000/s |
| Payload ingress | 50,000 × 200 = 10 MB/s |
| One day at that constant rate | 864 GB, before replication |
| Three replicas for that day | About 2.59 TB, before compression and overhead |
These are sizing assumptions. A peak rarely lasts all day; record headers, indexes, network overhead, compression, and extra topics change the real bill.
The sampling interval has a direct cost:
Choose partitions using measured producer and consumer throughput, recovery time, and headroom. If an expensive processor handles 2,000 events/s, 50,000 events/s needs at least 25 such workers before headroom—and enough partitions to keep them busy. Benchmark that capacity; Kafka's throughput alone does not size your processing layer.
5. Deliver reliably without promising magic
For our raw stream, a starting configuration is replication factor 3, min.insync.replicas=2, producer acks=all, and enable.idempotence=true. With too few in-sync replicas, writes fail rather than receiving the intended durable acknowledgment. Small batches and compression trade a little latency for efficiency. These settings are explained in Kafka's producer configuration and topic configuration.
The ingestion API acknowledges acceptance after Kafka acknowledges the write. A timeout can leave the phone uncertain whether its event arrived, so a retry reuses the same event identity. Producer idempotence handles a class of producer retries; it does not deduplicate every new HTTP request or an entire end-to-end business operation.
Use at-least-once processing with version-aware effects. A cache update performs an atomic compare-and-set: accept a newer epoch/sequence; ignore an older one. Equal-version retries are harmless.
There is a subtle crash window: the tracking consumer updates the cache, then crashes before notifying the gateway. On replay, it must still allow notification for an equal version, or use a durable notification mechanism. The client can discard duplicate versions. Commit the consumer offset only after the intended side effects succeed.
Kafka transactions can coordinate Kafka reads and writes, but they do not automatically make a Redis write and a WebSocket delivery atomic. Kafka's delivery-semantics documentation explains this boundary. Our tracking UI also periodically reconciles with the latest snapshot, so a lost transient notification does not freeze it indefinitely.
6. The last hop is a routing problem
Suppose Asha's event reaches worker A, but the customer is connected to gateway B. A consumer group does not magically deliver the event to B.
Our fanout layer resolves partner → active order → subscribed gateway connections. A shared subscription directory and internal gateway messaging route the update to the right sockets. Multiple customer devices may watch the same order, so this is a one-to-many mapping.
Before subscribing, authenticate the customer and verify access to that active order. Revoke subscriptions when delivery finishes, access expires, or the assignment changes. Include the assignment version in tracking messages so an update for a previous partner cannot overwrite the current one. Do not expose unrelated stops or a partner's full location history.
For reconnects, avoid the gap between “read snapshot” and “start listening”:
Buffer first, fetch the snapshot second, then flush only newer versions. Bound the buffer; if it overflows, fetch another snapshot. Periodic resynchronization repairs dropped fanout messages. After a long disconnect, the customer needs the latest position, not a replay of every turn taken while their phone was locked.
7. Live is a freshness promise
The marker can interpolate between confirmed observations to make motion easier to follow, accepting a small display delay. Prediction beyond the latest point should be brief and uncertainty-aware. After that, stop and show “Last updated 20 seconds ago.”
Kafka cannot repair absent GPS measurements. Nor does a healthy broker guarantee a current map. Track the age of the observation actually displayed, alongside ingestion latency, consumer lag, cache freshness, and gateway delivery delay.
| Failure | Response in our design |
|---|---|
| Phone loses signal | Keep a bounded local buffer; show stale status; resume with a fresh observation. |
| Consumer restarts | Resume from committed offsets; replay safely using versions. |
| Tracking processing falls behind | Add capacity within partition limits; coalesce obsolete points in the live path while history processes its retained stream. |
| Customer connection is slow | Keep a bounded latest-position queue per order; replace superseded positions rather than replaying a long animation. |
| Cache is unavailable | Show tracking temporarily unavailable; recover the current view from retained accepted events with age checks. |
| Kafka cannot accept writes | Retry with backoff and bounded buffering; report degraded freshness rather than acknowledging unpersisted updates. |
Keep order completion and cancellation in a reliable lifecycle path: a “latest location wins” policy must never discard a terminal order event. A separate history group isolates consumer progress, but shared broker resources still need quotas and capacity planning.
Raw coordinates also need short, explicit retention, restricted access, and a deletion strategy spanning downstream copies. A compacted topic can help rebuild latest state, but compaction is asynchronous and is not an immediate deletion guarantee. Kafka exposes separate retention and compaction controls in its topic settings.
The moving dot is the result, not the system
Kafka earns its place when location events serve multiple independently scaling consumers and replay matters. A small delivery fleet may start with an ingestion API, a current-location store, and a push service; Kafka adds operational work as well as capability.
At larger scale, the complete design is GPS capture → authenticated ingestion → ordered event processing → current state → authorized fanout → an honest screen.
The most revealing question is not “How many messages can our Kafka cluster handle?” It is: “How old is the location the customer is looking at right now?”
Would you trust a scooter that keeps moving during a network outage, or one that pauses and tells you when it last checked in? Share your take below.
Continue the journey: why the food and partner miss each other at pickup, and why a driver can be across the street but minutes away.