[Tor-BSD] attic: randomness exhaustion

[Richard Johnson]

On 2015-03-15 02:26, Libertas wrote:
> That said, it seems that updating to Tor 0.2.6.3 has significantly
> increased my throughput.

I cleaned up our state limit blockage by boosting them a while ago and noticed 
a slight throughput increase.  Now I've noted an additional 1.7x throughput 
increase about a week after upgrading to 0.2.6.3-alpha.  (I don't know that 
0.2.6.3-alpha is the sole cause of the increase.  I will switch up to 
0.2.6.4-rc this week.)  Numbers from nearly 3 weeks on 0.2.6.3-alpha are below 
for comparison.

There are 6 total relays (1 exit, 2 early in their lifecycle, 3 older 
middle/guards) on our 2-CPU (AMD Opteron 2435, 2.6GHz, non-AESNI) 12-core 
OpenBSD 5.6-stable system running tor 0.2.6.3-alpha.  The currently highest 
bandwidth relay is averaging 1.7GB/sec per arm, perhaps because it peaked over 
6GB/sec for a few days [0].  Average throughput achieved by all 6 together is 
now 4.6MB/sec per arm, up from 2.6MB/sec on 0.2.5.10.

As that throughput grows, we've started peaking over 14k current state entries 
in pf for Tor traffic [1].  The state entry count would be even higher but for 
"set optimization aggressive" in pf.conf expiring TCP leavings faster than the 
norm (which naturally raises the state-mismatch events as later client packets 
arrive).

One isolated congestion event showed with a filled ifq two weeks ago, even 
with ...ifq.maxlen boosted to 1024, but otherwise no ifq problems [2].  As 
well, our pf state table allocation is large enough that we're not running out 
of memory for states [1].


Richard

-------
[0]
https://globe.torproject.org/#/relay/1CC39E06101B0DBFA103A18A2032C4A0FE0503C8
https://atlas.torproject.org/#details/1CC39E06101B0DBFA103A18A2032C4A0FE0503C8

[1] pfctl -si output from a near peak time:

State Table                          Total             Rate
   current entries                    14226
   searches                     10297827608         9138.3/s
   inserts                         34574653           48.7/s
   removals                        34560427           48.7/s
Counters
   match                           51758879           59.9/s
   bad-offset                             0            0.0/s
   fragment                             968            0.0/s
   short                                845            0.0/s
   normalize                            200            0.0/s
   memory                                 0            0.0/s
   bad-timestamp                          0            0.0/s
   congestion                          2023            0.0/s
   ip-option                              0            0.0/s
   proto-cksum                          118            0.0/s
   state-mismatch                    708205            0.6/s
   state-insert                           0            0.0/s
   state-limit                            0            0.0/s
   src-limit                              0            0.0/s
   synproxy                               0            0.0/s
   translate                              0            0.0/s

[2] select sysctl output:

kern.nfiles=13058
net.inet.ip.ifq.maxlen=1024
net.inet.ip.ifq.drops=3220
kern.netlivelocks=76