Why my eyes beat my phone at night

Have you ever wondered why you can walk through a dark room that your phone’s screen shows as black grain?

In my last post I left this question open. I thought the electronics must be better than a human eye.

The eye catches less light

My first guess was the rods. Rods in the eye can catch a single photon, the smallest thing there is. A sensor has losses when reading. And I already knew that eyes are better in the dark than a phone, so that seemed to explain it.

Turns out I was wrong! It’s the other way round. Only about 10–20 % of the light that reaches the eye is caught by rods. The rest is lost on the way in: in the cornea, the lens and the fluid inside the eye. A phone sensor’s pixels catch more than half of the light that reaches them.

So the eye works with less light, and still wins. How?

The camera pays for colour

A pixel is colour-blind, just like a rod. To see colour, a phone puts a tiny filter on every pixel: red, green or blue. A green photon that lands on a red filter gets discarded. The filters throw away about two thirds of the light. After the filter, the camera keeps about as much light as my eye. And my eye still wins.

At night, the rods do not filter photons at all, and do not sort them by wavelength (colour). The eye tries to get them all, as many as there are rods available.

Every rod carries the same pigment, rhodopsin, so it can only say “a photon arrived”, never which colour it was. That’s why the night looks grey.

So it is all about tradeoffs here. Camera keeps the colour information, but drops 2/3 of photons when filtering R, G and B. The eye, on the contrary, drops the colour and keeps the light info.

Adding up in space

Many rods, sometimes more than 100, are wired into one nerve cell.

It channels many photons into one destination. So it cannot distinguish what those photons carried one by one. Just the sum.

What the brain loses is the exact location each photon landed on. That’s why night vision is blurry, not only grey.

Cameras can do the same channeling of multiple photons to a single destination. It’s called binning, and there are two types.

digital binning: every pixel is read on its own (charge → number), and every reading adds its own error (any reading adds an error)
charge binning: the charge of a group of pixels is gathered first and read only once. So the error is lower.

Groups of rods are connected to a single nerve cell. That is like charge binning: add first, read once.

Adding up in time

How do you make a noisy count less noisy? My first idea was: take many snapshots one after another, then compose them together.

But the eye doesn’t take snapshots. In the dark it collects light for about 0.1 s before it reports. A video frame gets only 1/30 s.

In this diagram you can see that the same photons fall on all three. The eye catches 4, one video frame catches 1:

But night mode wins

From all the facts above, it would seem that the phone should win here. It collects light 10–30 times longer (0.1 s vs 1–3 seconds). Throw away 2/3, and it still should be more.

And it does. Night mode takes a burst of short frames over a few seconds, aligns them and adds them up. That’s my snapshots idea, done by the phone. A night-mode photo can show colour and detail my eyes never saw.

So my eyes beat the phone’s live screen, where every picture gets a fraction of a second.

Maybe the brain also does some guesswork, based on the experience it has gained over all the years. It does: you “see” your hallway in the dark partly because you know what it looks like.

The camera throws away 2/3 of the light to see colour. How does a colour-blind sensor see colour at all?