Optimizing kernel compilation / alignments for network performance

[Rafał Miłecki]

On 6.05.2022 09:44, Rafał Miłecki wrote:
> On 5.05.2022 18:04, Andrew Lunn wrote:
>>> you'll see that most used functions are:
>>> v7_dma_inv_range
>>> __irqentry_text_end
>>> l2c210_inv_range
>>> v7_dma_clean_range
>>> bcma_host_soc_read32
>>> __netif_receive_skb_core
>>> arch_cpu_idle
>>> l2c210_clean_range
>>> fib_table_lookup
>>
>> There is a lot of cache management functions here. Might sound odd,
>> but have you tried disabling SMP? These cache functions need to
>> operate across all CPUs, and the communication between CPUs can slow
>> them down. If there is only one CPU, these cache functions get simpler
>> and faster.
>>
>> It just depends on your workload. If you have 1 CPU loaded to 100% and
>> the other 3 idle, you might see an improvement. If you actually need
>> more than one CPU, it will probably be worse.
> 
> It seems to lower my NAT speed from ~362 Mb/s to 320 Mb/s but it feels
> more stable now (lower variations). Let me spend some time on more
> testing.

For a context I test various kernel commits / configs using:
iperf -t 120 -i 10 -c 192.168.13.1


I did more testing with # CONFIG_SMP is not set

Good thing:
During a single iperf session I get noticably more stable speed.
With SMP: x ± 2,86%
Without SMP: x ± 0,96%

Bad thing:
Across kernel commits / config changes speed still varies.


So disabling CONFIG_SMP won't help me looking for kernel regressions.