Published results

What we have measured so far.

Every number on this site comes from a report we have published. Each is listed here with the conditions it was measured under.

Models in production

Model Compared with Result Conditions Source
Pika H3 Flex, generation Its official build, on our GPUs 1.56 to 1.83× After 15 kernel optimizations, F16 softmax and FP8 attention; 1.20 to 1.58× after the 15 optimizations alone. Four launch tasks, same weights, same six denoising steps. November 2026
Pika H3 Flex, end to end Pika's own API 1.24 to 1.70× Four 10-second 768p jobs, from submitting to the finished MP4. Mean of two runs per task, 13 to 14 November 2026. November 2026

Kernels

Kernel Hardware Result Conditions Source
FlashAttention-4 causal forward, merged NVIDIA B200, B300, GB300 up to 1.41× Our longest-first tile ordering, now in the official repository. Measured by its maintainers on grids with a partial section: B200 1.07 to 1.28×, B300 1.05 to 1.23×, GB300 1.04 to 1.41×; other grids within 0.4%. Outputs bitwise unchanged. November 2026
FlashAttention-4 forward NVIDIA B300 1.05 to 1.13× Four shapes, after an agent studied traces of cuDNN's closed-source attention kernel. How: see below. September 2026
FlashAttention-3, agent run NVIDIA H100 3.9× sooner Iteration 14 with MarsTrace, against 54 with the best existing tracer. How: see below. September 2026
FlashAttention-3 forward NVIDIA H200 up to 1.04× Typically 1.01 to 1.03×, on a baseline already above 90% of peak throughput. Submitted to the FlashAttention-3 repository. March 2026
FlashAttention-2 forward NVIDIA A100 up to 1.51× 1.25 to 1.51× at sequence length 1,024 and up with batch size 4 and up, in an automated run. Two small configurations were 7 to 8% slower. March 2026

Behind the numbers

How two of these results were made.

A better trace saves an agent iterations.

We gave a coding agent one task four times: raise the throughput of FlashAttention-3 on an H100. Each session read a different profile. Everything else stayed the same.

All four sessions ended near the same plateau, about 566 TFLOPS. The chart shows the first iteration at which each came within 2.5 TFLOPS of it. The existing traces reported a wait that their own probes had created, which sent the agent in the wrong direction.

Iterations until the agent reached the target

MarsTrace
14
Existing tracer A
54
Existing tracer B
71
Hardware counters only
85

FlashAttention-3 on NVIDIA H100. Source: MarsRover, September 2026.

Learning from a kernel with no source code.

On B300, cuDNN's attention kernel ran faster than open-source FlashAttention-4, and cuDNN ships no source. We traced both on the same inputs and asked an agent to close the gap using only the two traces.

What the trace showed What the agent changed Gain
The kernel ran its thread blocks in four waves, with idle time between them. Made the grid persistent, so each block fetches its next tile in place. 1.09×
Persistent blocks finished up to 1.49× apart. Sorted tiles across all groups, not only within each group. 1.02×
Long tiles were paired with long tiles. Used a fixed schedule on those shapes, pairing long with short. 1.11×
After the last tile, the kernel still waited 544 ns to free a buffer nothing would use. Dropped the final wait. 1.01×

Gains as reported for each change; not every change applies to every shape. Together they make FlashAttention-4 1.05 to 1.13× faster across four shapes. Source: MarsRover, September 2026.

The second change, sorting tiles across all groups, is now part of the official FlashAttention. Its maintainers measured up to 1.41× on the grids it targets (see Kernels, above).

Want numbers like these on your own kernels?

Talk to the optimization team, or see the tools behind them.