We build a virtual mirror for hair salons : a tablet camera feed where the customer's hair changes colour in real time while they move their head. Two constraints shaped every decision. The first is privacy: no frame ever leaves the device. Everything runs locally — the model, the fonts, the runtime — with no CDN dependency, because a CDN request would both break offline use and leak the salon's IP to a third party. The second is hardware: salons don't buy flagship tablets. We had to work on whatever mid-range Android is on the counter. This is what we learned making it fast enough. All numbers were measured on real devices with diagnostics built into the engine — none of them are estimates. The bug that looked like slowness Our starting point was bad in a confusing way. A Pixel 8 Pro ran at 9 fps — poor, but working. A Samsung A56 (Exynos 1580, Mali GPU) showed no colour at all . The cause took a while to find, and it's the most useful thing in this article: MediaPipe's ImageSegmenter with the GPU delegate was created without throwing, ran in 5 ms, and returned zero hair pixels. Both the confidence mask and the category mask came back empty. Every time. Our fallback to CPU only triggered on exceptions. There were no exceptions. So it never triggered, and the app simply looked slow instead of broken. A clean comparison, same scene minutes apart: Delegate Mask coverage Time per frame CPU 1.5–2.1 % 483 ms GPU 0.0 % 5 ms Note the trap in that table. The GPU path is 96× faster precisely because it isn't doing anything . If you benchmark by timing alone, the broken path wins. The lesson: don't assume that a delegate which constructs successfully actually works. Check the output, not the exit code. We now measure mask coverage and surface it in the diagnostics. Detecting it at runtime The fix is a fallback that rebuilds the engine on CPU. The rule matters more than it looks: After five consecutive empty masks, having never seen a good one in the whole session , rebuild on CPU. The condition is "never seen a mask", not "empty right now". An empty mask right now is completely normal — it happens whenever nobody is in front of the lens. We had to encode the difference between broken and nobody's there . On the A56 this took us from 2 fps to 14 fps with colour applied. A diagnostic that costs more than what it measures While chasing this, we left outputCategoryMask enabled from an earlier experiment. The diagnostic itself became a significant part of the frame budget. Worth remembering when your measurements start shaping the thing you're measuring. It was never the model With the GPU path honest, we profiled the Pixel 8 Pro's 9 fps. The model inference cost 6 ms. The other 126 ms were ours: 66 ms in a JavaScript loop recolouring pixels one at a time two GPU→CPU readbacks per frame, one of them costing 123 ms That's the whole story of most "the ML model is too slow" complaints we've had since: the model was never the problem. The problem was everything we did around it. Fix 1 — recolour in a fragment shader (66 ms → 0) The per-pixel JavaScript loop moved into a fragment shader. This is the obvious one, and it's free: the pixels are already on the GPU. Fix 2 — decouple the mask from the draw Hair does not move at 60 Hz. We were segmenting every frame for no reason. segmentForVideo is synchronous , and on an Exynos it costs 437 ms of blocked main thread. Now we segment one frame in every PASSO_SEG , and draw the video on all of them using the most recent mask. The step adapts to the device. This is the change that fixed the A56, and it's worth stressing: we expected to need a Web Worker and we didn't. Decoupling cadence was enough. Fix 3 — share the WebGL context with MediaPipe (123 ms → 0) This was the big one. We were copying the mask out of MediaPipe and back into our own pipeline, round-tripping through the CPU. MediaPipe accepts a canvas in its options. Pass it the same canvas you're drawing on, and you can take the mask with getAsWebGLTexture() — it never leaves the GPU. One catch, learned the hard way: MediaPipe recycles that texture under your feet. Binding it directly made the colour flicker. We copy it into our own texture with a shader, which is still enormously cheaper than a CPU round trip. Fix 4 — drop preserveDrawingBuffer (5 ms per frame) preserveDrawingBuffer: true costs real time on every single frame. We only needed it to capture a still of the current look, which is a rare user action. We defer the capture to the next draw call, within the same task, and read it there. Verified on device. Two more things worth knowing MediaPipe will not start inside a type: "module" worker. It dies with ModuleFactory not set , because its WASM loader registers the factory via importScripts() , which doesn't exist in module workers. If you want MediaPipe in a worker, you need a classic worker — or you don't get one. Freeze the frame in a texture before you segment. If you segment first and draw after, the video has already advanced, and the hair lags behind the head permanently. Not occasionally. Always. Pick the specialised model. We were running selfie_multiclass , which segments six classes when we needed one. Switching to the dedicated hair_segmenter : 780 KB instead of 16 MB, and the official Pixel 6 figures are 58 ms vs 217 ms on CPU, 52 ms vs 71 ms on GPU. Our consumer APK dropped from 40.7 MB to 25.8 MB. Where we landed 132 ms → 17 ms per frame. 9 fps → 15 fps on the Pixel 8 Pro, and 2 → 14 on the A56. You'll notice 17 ms per frame should mean roughly 58 fps, and we get 15. That's honest and worth explaining: the remaining ceiling is no longer in our code. It's in how the WebView is composited inside Flutter, and we haven't opened that front yet. We also kept the old CPU 2D path as a fallback for devices without WebGL2. That's not nostalgia — a canvas has exactly one context . If you request WebGL and it fails, you can't fall back to 2D on the same canvas. You have to decide before you commit. The takeaway Four of the five wins here came from removing work we had added ourselves, not from making the model faster. And the single largest discovery — the silently empty GPU delegate — wasn't a performance problem at all. It was a correctness bug wearing a performance costume, and it was invisible until we measured the output instead of the duration . If you're building anything similar: instrument mask coverage, not just milliseconds. The frame that takes 5 ms might be the one doing nothing. The engine described here ships in Miraviso , a consultation tool for hair salons: the customer sees the colour and the haircut before the stylist starts. Happy to answer questions about the WebGL side in the comments.