mirror of
https://github.com/tailscale/tailscale.git
synced 2026-11-21 01:01:18 -08:00
Handing a packet to a crypt worker, the tx worker or a NAPI kthread on another CPU can wake a halted vCPU, which in a VM is a VM exit and a trip through the host's scheduler. tswg's idle round trip time was 1.5-2x wireguard-go's in VMs because of it. So when the crypt workers look idle, a packet that isn't part of a batch is encrypted or decrypted where it is, after going on the peer's in-order queue as before. To keep TCP ACKs under load from moving crypto onto the CPUs taking the NIC's interrupts (8% off a subnet router), it's limited to 8 packets per CPU per jiffy, and never done for encryption from softirq (forwarding). And peers' NAPI runs in softirq again, not kthreads, unless napi_threaded is set. Idle RTT p50 is now 0.51ms in the lab (wireguard-go 0.60) and 0.75ms on EC2 (0.97), with subnet router throughput unchanged at the same NAPI setting. Threaded NAPI is still 4-5% faster for a CPU-bound router. Updates #426 Signed-off-by: Brad Fitzpatrick <[email protected]> Change-Id: Ib5ebd1edcda1a515dac0ae4e55540bcfe52d1d7f