mirror of
https://github.com/tailscale/tailscale.git
synced 2026-11-20 08:41:19 -08:00
Profiling a CPU-bound tswg subnet router (EC2, 4 CPUs) found two costs in decrypting merged packets: Copying. Each message was copied out of the received packet and then decrypted in place in the copy, two passes over it. When the message is contiguous in the received packet (in its head or one page frag, as a message received in one datagram is), decrypt straight from it into the new packet instead (tswg_contig). The Poly1305 key. lib/crypto makes it from a separate 32 bytes of ChaCha20 keystream (block 0), and x86's ChaCha20 does anything up to a block with its portable code: a scalar ChaCha20 block per message, 5.5% of the router's CPU. tswg_aead.h has our own ChaCha20-Poly1305 for data messages, from the library's primitives. When decrypting from one buffer to another, it runs ChaCha20 from block 0 over the 64 bytes before the message too (packet headers in the source, headroom in the destination), and recovers block 0's keystream, the Poly1305 key, as the two XORed: one SIMD pass per message (tswg_aead_decrypt_ahead). Otherwise, and for encryption, it asks for two blocks of keystream, which takes the SIMD path, and sets the counter back. The 4-CPU router, 4 clients with 16 flows each, medians of 3 runs: 14.1 to 14.9 Gbps, 2.30 to 2.18 CPU-seconds per GB (wireguard-go: 6.6 Gbps, 3.36). Interop with wireguard-go is unchanged; KASAN is quiet. Also tried, and not kept: zero-copy receive (decrypting in place in the received pages, 7% worse), and a page pool for decrypted packets (its pages didn't come back in time to save allocations). Updates #426 Signed-off-by: Brad Fitzpatrick <[email protected]> Change-Id: I98456d17b82d5ef640c55ca1a7fa7a018dc5a279