Linux 7.3 x86/mm Lands Patches To Greatly Improve Latency-Sensitive Workloads

The highlight of this week's x86/mm pull request of changes for the Linux 7.3 kernel are fixes that reduce the time that the TLB flushing code has interrupts disabled. This helps significantly with latency-sensitive workloads but Intel engineer Dave Hansen noted in the pull request that "it's certainly something to keep an eye on" in looking out for any regressions. Earlier this month I noted the pending patches in "-next" as an imporvement to help reduce latency and improve real-time performance. This work carried out by Bytedance saw up to a 90% latency reduction with these now-merged patches. Bytedance engineer Chuyi Zhou explained in the prior patch series: "The vast majority of smp_call_function*() callers block until remote CPUs complete the IPI function execution. As smp_call_function*() runs with preemption disabled throughout, scheduling latency increases dramatically with the number of remote CPUs and other factors (such as interrupts being disabled). On x86-64 architectures, TLB flushes are performed via IPIs; thus, during process exit or when process-mapped pages are reclaimed, numerous IPI operations must be awaited, leading to increased scheduling latency for other threads on the current CPU. In our production environment, we observed IPI wait-induced scheduling latency reaching up to 16ms on a 16-core machine. Our goal is to allow preemption during IPI completion waiting to improve real-time performance. ... In our production environments, latency-sensitive workloads (DPDK) are configured with the highest priority to preempt lower-priority tasks at any time. We discovered that DPDK's wake-up latency is primarily caused by the current CPU having preemption disabled. ... After applying this patchset, we no longer observe preemption disabled for more than 1ms on the arch_tlbbatch_flush/flush_tlb_mm_range path. The overall P99 of max preemption disabled events in every 30-second interval is reduced to around 1.5ms. The remaining latency is primarily due to lock contention." These patches are now merged for Linux 7.3 after ten rounds of patch review. This should improve wake-up latency and the code has been tested but there appears to be at least some concern to look out for regressions. I'll begin firing up my Linux 7.3 Git kernel benchmarking at large in the coming days as the merge window begins settling down.