mirror of
https://github.com/tailscale/tailscale.git
synced 2026-11-20 08:41:19 -08:00
Our UDP sockets now have gro_receive/gro_complete callbacks, as UDP tunnels do. They merge consecutive data messages of a flow that use the same receiving key and are all the same size but perhaps the last, and UDP hands the merged packet (SKB_GSO_UDP_TUNNEL) to encap_rcv whole. It goes through the decrypt ring and the peer's queue as one entry: one worker decrypts each message into a new packet (copying it out and decrypting in place there, rather than linearizing in place), and the peer's NAPI poll delivers them in order. So the IP/UDP input path, the ring and queue handoffs and the worker kick are paid once per merged packet rather than per packet. Upstream gets no outer GRO at all. skb_gro_receive isn't exported to modules, so socket.c has a copy (tswg_gro_merge). gro_receive_network_flush and udp_tunnel_update_gro_rcv are newer than 6.8, so they're probed. The latter matters: 7.x may make the socket's gro_receive a static call, which detaching a user socket must undo, or it would outlive the module. Bulk TCP in the lean harness, now with guest USO off (so the receiving VM sees individual datagrams, as from a physical network): about 9.8 Gbps, against 6.5 before, and within a few percent of wireguard-go (about 10.0), at about 18% less receiver CPU. Making workers poll the ring briefly before going idle (to save wakeup IPIs) was tried and didn't help, so isn't included. Updates #426 Signed-off-by: Brad Fitzpatrick <[email protected]> Change-Id: Ic22482120c6ce75b9a4d896f16b6cf77aca28b4c