[Tor-BSD] Slow network performance on virtualized OpenBSD 6.6

[George Rosamond]

On 12/1/19 12:46 PM, Winter Paulson wrote:
> Hello all,
> 

Hey hey.  Sorry this post missed me first time around...

> I've been running OpenBSD and FreeBSD bare metal tor exits for some
> time. Now I have some speed issues with a virtualized OpenBSD node where
> I only reach ~ 12.5 MB/s tor throughput.
> 
> The host is running Debian Buster, Intel X550T NICs and AMD EPYC 7xx1
> processor and has 2 x 1 GBit/s uplink (not limited). The VM has 2 GB RAM
> and 4 CPU cores allocated and the tor process is not limited atm by
> single thread performance. Local unbound is doing DNS resolution (2
> threads).
> 
> Running iperf3 to public servers (e. g. ping.online.net) resulted in
> 960 Mbit/s download and more than 120 Mbit/s upload (slow on ramping, so
> there is probably much more available when testing to a private iperf3
> server) while the tor server is running. So in principle there is more
> bandwidth available.
> 
> I changed the network driver from virtio to e1000 and performance
> increased around ~ 1.5 MB/s. I tried to disable hardware offloading but
> couldn't find out how to do so in openbsd. Trying "ifconfig em0 -rxcsum
> -txcsum -tso -lro" results in "bad value" for each of the parameters
> (with virtio and e1000 driver). "ifconfig -m em0" is not available on
> openbsd to show the (enabled) capabilities of the virtual NIC.
> 
> Does anyone know how to check if hardware offloading is active on a
> virtualised openbsd and if applicable how to disable it? Also any other
> tips on how to increase would be much appreciated!


I don't know specifically with the virtualized environment, but in terms
of bare metal, you should tinker with /etc/login.conf and relevant
sysctls.  As you're away, OpenBSD is (thankfully) stingy about network
resources.

Virtualized environments are much harder to optimize IMHO since you
aren't just dealing with an operating system and hardware, but an OS on
top of an OS.

I'm not getting great numbers on bandwidth, but I'm partially restricted
by our shared 100mbit connection being close to saturation.

Some of my sysctl knobs on a few OpenBSD boxes include:

kern.maxfiles=13000             # default 7030
kern.somaxconn=2500             # 128
kern.maxproc=3000               # 1310
kern.maxlocksperuid=2000        # default 1024

I have a lot more to do in this area, and I haven't even put much
thought into the network card layer... so it would be greatly
appreciated if you shared anything you discover, including in the
virtualized environment.

g