[Tor-BSD] Performance loss migrating Linux -> FreeBSD

[Mark Saad]

---
Mark Saad | mark.saad at ymail.com
> On Apr 9, 2016, at 5:32 AM, failure <failure at openmailbox.org> wrote:
> 
> Hello,
> 
> to add some diversity (and learn some new stuff) I migrated my VIA Nano
> U2250 1.6Ghz (no hw accel. for encryption) powered dedicated exit node
> from Linux to FreeBSD.
> The throughput limit is the CPU which was able to push around 82 Mbit/s
> max with linux but is only able to push 67 Mbit/s max with FreeBSD.
> Memory usage is much lower on FreeBSD.
> 
> Linux was: Ubuntu, 4.3 kernel, haveged, postfix, ntp, munin-node,
> IPtables rules (as suggested by torservers.net[1])
> 
> FreeBSD 10.3, std kernel, munin-node, sendmail, ntp, pf
> 
> pf.conf:
> 
> set skip on lo
> set optimization aggressive
> set limit states 13000
> set block-policy drop
> scrub on em0 reassemble tcp no-df random-id
> antispoof for em0
> block in proto tcp
> block in proto icmp
> pass in on em0 proto tcp from any to [IP-of-exit] port 443
> pass in on em0 proto tcp from any to [IP-of-exit] port 80
> pass in on em0 proto tcp from any to [IP-of-exit] port [SSH-port]
> pass in on em0 proto tcp from [IP-of-munin-server] to [IP-of-exit] port 4949
> pass in inet proto icmp all icmp-type echoreq
> pass out all
> 
> sysctl.conf:
> 
> net.inet.ip.random_id=1
> net.inet.ip.portrange.reservedhigh=0
> net.inet.tcp.blackhole=2
> net.inet.udp.blackhole=1
> 

Two other sets of changes I would recommend 

In sysctl.conf 

net.inet.ip.fastforwarding=1

#this enables the TCP fast path which is disabled
By default as is mutually exclusive with IPSec

In boot/loader.conf

net.inet.tcp.tso=0
kern.ipc.nmbclusters="1000000"
hw.em.fc_setting=0

#this turns off TCP segment offload , flow control
And adds more network cluster buffers .

You will need to bounce your nose after making this set of changes . Loader.conf is only read at boot.



> Would migrating from openssl to libressl maybe improve things? Are there
> any other things I might have missed or is there nothing to do about? A
> 18 % loss of network max speed (which correlates to the loss of the
> average speed) seems to be a lot for me.
> 
> Thanks for any help!
> 
> [1] https://www.torservers.net/wiki/setup/server
> 
> _______________________________________________
> Tor-BSD mailing list
> Tor-BSD at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/tor-bsd


---
Mark Saad | mark.saad at ymail.com