[Tor-BSD] Recognizing Randomness Exhaustion

[Greg Troxel]

Libertas <libertas at mykolab.com> writes:

> Some of the people at tor-bsd at lists.nycbug.org and I are trying to
> figure out why Tor relays under-perform when running on OpenBSD. Many
> such relays aren't even close to being network-bound,
> file-descriptor-bound, memory-bound, or CPU-bound, but relay at least
> 33-50% less traffic than would be expected of a Linux machine in the
> same situation.

I'm more familiar with NetBSD, but hopefully my comments are helpful.

> For those not familiar, a Tor relay will eventually have an open TCP
> connection for each of the other >6,000 active relays, and (if it allows
> exit traffic) must make outside TCP connections for the user's requests,
> so it's pretty file-hungry and crypto-intensive.

It may also have something to do with TCP.  A few thoughts:

* run netstat -f inet and look and the send queues.  That's not really
  cleanly diagnostic, but if they are all huge, it's a clue

* run netstat -m and vmstat -m (not sure those map from NetBSD).  Look
  for runnig out of mbufs and mbuf clusters.   Perhaps bump up
  NMBCLUSTERS in the kernel if it's not dynamic.

* Take a critical look at your TCP performance.  This is not that easy,
  but it's very informatve.   Get and install xplot:
    http://www.xplot.org/
  Take traces of v4 tcp trafffic with 
    tcpdump -wTCP -i wm0 ip and tcp
  and then
    tcpdump -r TCP -tt -n -S | tcpdump2xplot
  Then you'll need to read all the xplot READMEs (see the source).  This
  will show you tcp transmitted segments, sack blocks, the ack line, dup
  acks, and other TCP behavior.  It's not that easy to follow, but if
  you understand TCP you'll be able to spot odd behavior far faster than
  reading text traces.   It's possible that tcpdump2xplot may mishandle
  OpenBSD's tcpdump output - it's perl to turn text back into bits, and
  it's broken over the years with tcpdump upgrades.

  You may well not want to send me a trace, but if you send me the
  binary pcap, the text version above, or the tcpdump2xplot files, I can
  take a look.

> One possible explanation is that its randomness store gets exhausted. I
> once saw errors like this in my Tor logs, but I don't know how to test
> if it's a chronic problem. I also couldn't find anything online. Is
> there any easy way to test if this is the bottleneck?

On NetBSD, there is "rndctl -s".  I would expect the same or similar in
OpenBSD, and you can look every second to see if there are bits still in
the pool.  I don't think this will turn out to be the issue, though, if
you're seeing 30% of what you think you should - I would expect the
performance hit due to running out of bits to be much bigger.

Greg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 180 bytes
Desc: not available
URL: <http://lists.nycbug.org/pipermail/tor-bsd/attachments/20141231/2cad9c5e/attachment.bin>