Computer vision · on a real robot
The robot's camera streams to a screen. Software draws a box around every face, turns each one into a string of numbers, and decides if it's someone it knows — then the robot greets them. This guide is how it works, and how you can push it.
Two different questions, two different jobs.
People do it in one glance, but for a computer, “there's a face” and “that's Maria” are two separate problems — solved by two separate models you can swap.
Scan the image and draw a box around anything that looks like a face. It doesn't care who — just where. Fast, and needs no database.
Take each boxed face and compare it to people the system has met before. This is the slower, heavier step — and it needs a memory of faces.
Recognition can't compare pictures pixel-by-pixel — lighting and angle change every pixel. Instead a neural network turns each face into a faceprint: a fixed list of numbers (an embedding) that captures the shape of a face. The same person's faceprints land close together; different people land far apart. Comparing two faces becomes measuring the distance between two number-lists.
The network learned to do this by studying huge collections of faces, so the numbers capture what genuinely makes people look different — not the lighting or the background. Mathematically, a faceprint is a single point in a space of hundreds of dimensions, and “how alike” is just how close two points sit. That's the quiet idea under most modern AI: turn messy real-world things into points, then do geometry.
“Close enough to be the same person” is a dial called the match threshold. In this lab it's a distance — smaller distance means more alike — so lower = stricter:
There's no “correct” setting — only the right tradeoff for the job. Tuning that dial is real engineering.
From lens to “hello” — and where the work actually happens.
The camera lives on the robot, but the thinking can happen somewhere else. Frames flow one way; a greeting flows back.
The camera hands over frames far faster than a heavy model can study them. Sending the video is cheap — each frame is compressed to a couple hundred kilobytes. The expensive part is the thinking. So the network is almost never the bottleneck; the compute is. That single fact shapes every choice that follows.
Runs the largest, most accurate models. Heavy on memory, but you've got plenty.
A lightweight model runs right on the robot — no laptop needed, and a tiny memory footprint.
A dedicated graphics chip runs big models fast — the best of both, when you have the hardware.
Same task, three homes for the brain. Put them side by side and the tradeoffs stop being abstract — you can read them off the numbers.
The dashboard is a cockpit. Here's what every control does.
Open the dashboard and you'll see the live camera on the left and a stack of controls on the right. Nothing you touch can break it — poke everything.
The live readout is the truth. The two numbers to watch: camera fps (how fast frames arrive) and process fps (how fast the model keeps up). When camera fps is much higher than process fps, the model is the bottleneck — exactly what chapter 2 predicted.
Where this is heading — and the responsibility that comes with it.
For years, “AI” meant shipping your data to a giant server far away. That's changing. Modern robots, phones, and cameras increasingly carry a dedicated NPU (neural processing unit) — a chip built specifically to run neural networks right where the data is. Computing at the edge like this means answers in milliseconds, no internet required, and the video never has to leave the device. Doorbells, cars, and robots are all moving this way.
Running a model on a regular processor works, but it leaves the NPU idle. The next step is compiling a model into the special form an NPU understands, which can make detection several times faster on the very same device — a “regular processor vs. NPU” race you can already start to feel in the numbers.
Use it well
Face technology is powerful and sensitive. Recognition models can be less accurate for some groups than others, so a threshold that's fair for one person can be unfair to another. Recognizing people should happen with their knowledge and consent. As you build, keep asking not just “does it work?” but “who could this get wrong, and who gets to say yes?”
You now know the whole path — from a lens, to a box, to a string of numbers, to a name, to a wave. Go make the robot say hi.
The words you'll hear, plainly.
| detection | Finding that a face is present, and where (a box). Doesn't care who. |
| recognition | Deciding whose face it is, by comparing to people you've added. |
| detector / recognizer | The model that finds faces / the model that identifies them. Two separate, swappable models. |
| faceprint (embedding) | A face turned into a fixed list of numbers — a point in space. Same person, nearby points. |
| match threshold | How close two faceprints must be to count as the same person. Lower = stricter. |
| false positive / negative | Wrongly matching a stranger / wrongly missing someone you know. |
| enrollment | Registering a person by saving their faceprint (click a face, add a name). |
| edge / on-device / NPU | Running the model on the device itself; an NPU is a chip built to do that fast. |