When you press a key on a phone keyboard a small popup appears above your finger, and drawing it makes the graphics processor do a different amount of work depending on which letter it is — work that is recorded in counters any app could read without asking permission; this paper (Pittsburgh) shows that those counters identify individual key presses with about 97 percent accuracy on real phones, recovering more than 80 percent of typed passwords, and argues the fix is access control on the counters rather than removing the popups.
The argument in one glance
-
Problem
Hardware leaks facts that software was careful to protect
Software can be fixed by an update, so attacks that matter longest are the ones that exploit hardware behaviour. A side channel is the pure form of this — the secret is not read, it is inferred from something the hardware does while handling it. On a phone this is worth attacking because everything valuable passes through the same small screen, and because a phone is a shared machine in the sense that matters: the apps on it run alongside each other on one processor.
-
What people did
Watch the processor cache, or the motion sensors
Existing phone side channels look at two places. Contention for the processor cache reveals something about what a neighbouring application is doing, since programs competing for the same cache slow each other in measurable ways. Motion sensors reveal something about where a finger landed, since tapping different parts of a screen tilts the device differently. Both have produced real published attacks.
-
Why it fell short
The signals are only loosely tied to what the user did
Neither channel has a crisp relationship to a specific action. Cache contention depends on everything else running; sensor readings depend on how the phone is held. The correlation is weak and noisy, so these attacks infer coarse facts — which application was launched, roughly where the user is, who the user might be — and cannot read a keystroke. Attacks on the graphics processor existed too, but they tracked overall workload, and typing a letter is not enough work to notice.
-
This paper's idea
Watch what is drawn, because what is drawn is what was typed
This paper (Boyuan Yang, Ruirong Chen, Kai Huang, Jun Yang and Wei Gao) starts from a relationship that is not loose at all. Every key press produces visible feedback, all visible feedback is drawn by the graphics processor, and the processor keeps counters describing that drawing at pixel granularity. Different letters produce differently shaped popups, so they produce different counter values, and the link between the secret and the signal is direct rather than statistical.
Why this problem is worth solving
The mechanism turns on a detail of how phone interfaces are drawn.
A
phone screen is composed from layers — a background, an application, a keyboard, and on top of
the keyboard the little popup that appears above the key you are touching. The graphics
processor draws these in order, so a given pixel may be written more than once as higher
layers cover lower ones. That repeated writing is called OVERDRAW, and how much of it occurs
depends on the exact shapes involved, because the hardware skips work where a later layer
completely hides an earlier one. A popup showing a narrow letter occludes a different set of
pixels than one showing a wide letter, and the character drawn inside it has its own shape
too. So the total drawing work for one key press is a function of which key it was. That is
not a statistical tendency, it is geometry, and it is why this channel is sharp where earlier
ones were vague.
The second thing to explain is why any of that is visible to an ordinary application.
PERFORMANCE
counters are hardware registers that tally events — how many pixels were rasterised, how many
tiles were skipped as hidden, how much of a cache was used. They exist for profiling and
debugging, and also so that software can adapt to how loaded the hardware currently is, which
means they are deliberately exposed rather than protected. On desktop graphics hardware there
are managed libraries around them, and vendors have restricted unprivileged access. On the
phones studied here there is no such interface, but there is a device file through which every
graphics library talks to the hardware, and it must be reachable by ordinary applications
because those libraries run inside them. The result is that a counter describing what the whole
device drew can be read by an app that requested no permissions at all, using a system call
that ordinary software makes thousands of times a second.
What people did before, and where it broke
-
Cache-contention attacks. Measure how long your own memory accesses take, and infer what a neighbouring program is doing from how it disturbs the shared cache. A large and successful literature.
-
The gap. The signal depends on everything else running on the machine, so it is noisy and only loosely tied to any particular action. These attacks identify which application is running or roughly where the user is, not what was typed.
-
Motion-sensor attacks. Read the accelerometer and gyroscope, which register the small tilts caused by tapping different parts of the screen, and infer keystrokes from them.
-
The gap. The relationship between a tap and a tilt depends on how the phone is held, on the surface it rests on, and on the user. Accuracy is fragile in exactly the conditions where an attack would be used.
-
Graphics workload attacks. Watch how busy the graphics processor is, through cache timing or interface performance metrics, and infer user activity from the variation.
-
The gap. Sensitivity is bounded by how much work an action causes, and typing one character causes almost none. Applied to phone keyboard input here, the best of these approaches reaches under 14 percent accuracy.
-
Assume the platform’s permission system is the boundary. Android gates sensitive resources behind permissions the user must grant, so the natural expectation is that anything revealing is behind one.
-
The gap. These counters are not gated, because the path to them is the same path every graphics library uses. The permission model has nothing to say about a resource that must be open for the platform to function.
How drawing work becomes typed characters
From a rendered popup to an inferred password
-
-
new
-
new
-
new
-
new
-
new
The idea
Pick a channel where the correlation is causal rather than statistical. The framing that makes this work is a chain of three facts that hold by construction. User input always produces visible feedback, because interfaces are built to confirm what you did. Visible feedback is always drawn by the graphics processor, because that is what draws everything. And on a phone the graphics processor is doing essentially nothing else, so its counters describe that drawing rather than a mixture of unrelated work. Put together, the counters are close to a direct measurement of screen change at pixel granularity, which is why individual key presses are distinguishable here when the same idea applied to overall workload reaches under 14 percent. It is the same lesson as the branch-predictor attack covered elsewhere on this site, where a structure maintained for a completely different purpose turns out to hold a high-resolution record of what a program did.
Most of the work is not the leak but the noise. The counter values for each key are learned once, offline, on a phone the attacker controls, so the online step is a lookup. What takes engineering is that a key press does not produce one clean signal. It produces three screen changes — popup appears, character echoes, popup vanishes — and sampling can split one change across two readings or register one press twice. Measured across 3,485 presses there were 633 duplications, 316 splits, and 21 noisy cases, enough to misread 28 percent of keys if ignored. The fixes are timing arguments rather than signal processing: humans cannot press two keys within 75 milliseconds, so a second candidate inside that window is a duplicate, and adjacent readings can be combined when neither matches a key alone but their sum does. A separate counter turns out to track input length exactly — rising by two per character and falling by two per backspace — which detects corrections, and cursor blinking is excluded because it happens on a half-second cycle. That the practical obstacle is separating signal from ordinary system activity rather than obtaining the signal is the same finding as the cloud cache attack covered elsewhere on this site, whose contribution was showing that noise is not a defence.
The defence has to be access control, because everything else has a hole. The paper works through the obvious mitigations and finds each incomplete. Turning off keyboard popups removes the strong signal, but it is an accessibility feature for people who need to confirm what they pressed, and the length-tracking counter still leaks how many characters a password has. Relying on app-store malware detection fails because the attack makes only ordinary system calls that normal software makes thousands of times a second — demonstrated by publishing a test app that polls the counters and passes every check. Password managers and biometrics avoid typing but still require typing on first login, and depend on adoption. Obfuscating the counter values does work, and the paper notes one bank whose animated login screen incidentally cuts accuracy to 30 percent, but deliberate obfuscation costs graphics work on a battery-powered device — the same tension as the memory-timing defence covered elsewhere on this site, which shapes traffic into a secret-independent pattern and must pay for the shaping. What is left is fine-grained access control, letting profiling tools read counters while ordinary applications cannot.
What one key press looks like
Reading a single character out of a counter trace
Following one key press from the tap to a decision, including the ways it can go wrong.
Step 1 of 5
Three screen changes, not one
Pressing a key produces a popup, then the character echoing into the field when the key is released, then the popup vanishing. Each is a distinct change in the counters, and together they form the signature of one press.
The counters used come from three families — the tiles the hardware skipped because a higher layer hid them, the pixels produced by converting shapes into a grid, and the vertex cache the hardware uses while doing it. All three vary with the geometry of what was drawn.
Why it works
- The channel measures the secret almost directly. Feedback is always drawn, drawing is always done by this processor, and the processor does little else → the counters are close to a per-pixel record of screen change → individual characters are distinguishable, where measuring overall workload reaches under 14 percent.
- Geometry makes the classes separable. Different characters occlude different pixels and contain different shapes → the amount of skipped and rasterised work differs per key → a nearest-match lookup against a pre-recorded table suffices, with no inference at run time.
- Human timing bounds the errors. People cannot press two keys within 75 milliseconds → any second candidate inside that window must be an artefact of one press → the largest error category is removed by a rule that needs no signal processing at all.
- A different counter happens to count characters. One value rises by two per character and falls by two per backspace → corrections are detectable, and cursor blinking is excluded by its half-second period → deleted characters do not corrupt the recovered string.
- Cheapness is what makes it practical. Only the guessed text is transmitted and the lookup is sub-millisecond → battery and network cost are unnoticeable → the app can sit in the background indefinitely, which a heavier attack could not.
What it costs (trade-offs and scope)
- It needs the attacker’s app on the phone. The threat model assumes a malicious app has been installed, disguised inside something benign. That is a real assumption, though the paper strengthens it by showing such an app passes store review.
- Training is per device and per keyboard. The counter signatures depend on the phone model and the keyboard’s layout, so the attacker needs a table for each combination and a step that recognises which one applies.
- Slow typing hurts. Accuracy on whole strings falls to around 60 percent when typing is slow, because more unrelated screen activity falls between presses. Individual key accuracy holds, and errors average under 1.3 per string, which is few enough to brute-force.
- The scope is one vendor’s graphics hardware. The counters, their names, and the interface are specific to one family of mobile graphics processors — admittedly the most widely deployed one — and the work does not establish that other vendors leak the same way.
- It reads what is drawn, not what is meant. A field that does not echo characters, a login flow with animation, or an interface that does not show popups all degrade or remove the signal, which is also why the mitigations that work are the ones that change what is drawn.
The evaluation
| Property | What was measured | The result | What it tells you |
|---|---|---|---|
| Accuracy in controlled conditions | |||
| Accuracy in realistic use | |||
| Against the previous approach | |||
| Breadth | |||
| Cost to the victim |
Check yourself
Check your understanding
1. Why does watching drawing work distinguish individual key presses when watching overall graphics workload does not?
2. Why can this attack read the counters without requesting any permission?
3. Measured across 3,485 key presses, duplicated and split readings would have corrupted 28 percent of inferences. How are duplicates handled?
4. The paper argues that simply disabling keyboard popups is an insufficient mitigation. Why?
Sources
Eavesdropping user credentials via GPU side channels on smartphones. Boyuan Yang, Ruirong Chen, Kai Huang, Jun Yang, Wei Gao. ASPLOS 2022, pages 285 to 299. DOI 10.1145/3503222.3507757. Full text read from the authors’ copy hosted at the University of Pittsburgh. The subsequent CVE assignment and platform patch are recorded on the authors’ project page rather than in the paper, which reports only that disclosure was made and a patch promised.