[Tor-BSD] high-bandwidth relays

[Christian Sturm]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 06/30/2015 03:24 PM, teor wrote:
> 
>> On 30 Jun 2015, at 03:33 , Christian Sturm <reezer at reezer.org>
>> wrote:
>> 
>> Signed PGP part On 06/29/2015 07:26 PM, George Rosamond wrote:
>>> The question I have is running multiple Tor instances.  At
>>> what point is it necessary?
>> 
>> I'd do that incrementally. Just set up your Tor relay and wait
>> for one CPU to be exhausted.
>> 
> 
> Tor's crypto is mostly multithreaded. But some other operations are
> single-threaded. So it's possible for Tor to exhaust one CPU, but
> only partially use other CPUs. Another Tor instance can fully
> utilise another CPU, and the crypto threads can use the remaining
> CPUs.
> 
> So if you have 4 or more cores, I'd start with 2 Tor instances.
> 
> The current Tor network limits you to 2 instances per public IPv4
> address. This is a feature that mitigates sybil attacks, by tying
> the number of instances to the scarcity of IPv4 addresses. If 2
> instances don't utilise all your cores after several months, then
> you may want to consider another IPv4 address and another instance
> or two.


I said one probably do it incrementally (so start out with one
instance before adding others), because it allows you to measure the
throughput you are able to achieve with one CPU and see how that
relates to your available bandwidth.

There are various factors influencing that, such as as raw CPU
processing power, choosing an OpenSSL implementation with fast ECC
(which is not enabled per default on FreeBSD), AES-NI (which you also
have to manually enable by loading a kernel module).

Another factor how fast clients switch over to newer code that allows
faster code to actually be used. I didn't think through that yet, but
if you have high bandwidth and your node is stable you will be a guard
node that certain clients will use, meaning that over time the clients
who chose your node as a guard will maybe become older over time,
which when there is protocol changes would maybe mean that setting up
a new relay (keys/fingerprint) would result in a speed up as the
amount of people who for example use TAP instead of the ntor would be
reduced. However I don't know how big such an effect would be.

>> When your CPU on Tor averages to close to 100% (will take a
>> while) you can set up a new instance.
> 
> Please see 
> https://blog.torproject.org/blog/lifecycle-of-a-new-relay to
> understand the growth rate that you should expect, based on how the
> network uses new relays over time.

The question is also whether you will eventually end up maxing out
your one CPU core or if you will max out your available bandwidth
first. It would probably be interesting to know what high-bandwidth
means and what the server would look like, hardware-wise.

I just think finding this out is easier if you start with one instance
and see how that works, also so you don't end up having overhead,
should you be able to max out your CPUs, also depending on other stuff
potentially running on those systems. It's maybe also good to have
some resources spare.

OTOH I really hope Tor will eventually end up being able to even in a
single instance make better use of the available CPUs, cause as
systems get more CPUs this will become a bigger issue and I think
there might be overhead and potential other potential side effects
coming from this.

>> You will then see how much throughput a single instance can
>> handle.
> 
> I'm working on a way of load testing a tor relay's CPU, by running
> a local Tor test network. Because all the network communication is
> on localhost, this helps you work out the maximum CPU-bound
> throughput of your relay. There's a draft implementation, but it's
> not quite ready for release yet. So waiting a few months for the
> network ramp-up is your best option.
> 
> It should be ready some time in the next few months, watch this
> Trac ticket for details: 
> https://trac.torproject.org/projects/tor/ticket/14175

That's really cool! Will try it out in future. Thanks! :)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQQcBAEBCAAGBQJVkvBSAAoJEGTx4c0EAjoaRSgf/3+RIHHp6FkYRi/fS/+p3j7q
ikoI317xHNPKK94z/5dzsa1FpPimi3+VLa/q5oan6x4koYZx8JeM1vpw/JtM5f38
1xCteVK/kQb/ibYx8gqY7M608gCN79kfgSBuFGOpRdVZpZz5yHth6anEX1Nq9dL4
1Xivi9O68+zGeCR3N7SYF9Xz1i4Y4zzl8xjLY11rAZUo/DeZEd3o+rJHFJKqWY8k
huVLyAGlKLdEAbFOLaWb4vGSnpr3Cnca9RaDvO0bMA/NE3Cf0WqBZKCMO1JQoJBR
senLE4yfYlhOLjScey8lFhirliEmMFPpXYZRUnvFKiyPpshZRsmW69V1nei2IMAx
3Ty8QOhVf7fv34KikUN9A17kJOwbsrlnY17mZyAgTQ7XSqC5dRHeqQhvr9bWjtJV
kgGXBqHJBaiA/UNAXTJGMb4STXIQsK5neiZ6ZPnDxUnMW9A1lAoBCAWFLr7oXqzt
tsrx0YQE3HBZUlUiRbZf6Y/qx80Z2qtRcK+zlr7xfC5bcR/Zo83yluwem1jsDZw+
Uy0h3k29PpNWgk2Z8KtHlPB5TGF3+eCULHIZ3xqeeEHna4AJgqdDLw1NG33bsgU+
ec6kxm2DzrwWFGRDAhJjWjDVM+H70GsO7NzHhIx0OXngvYIFLnGXrPAcpX1C3jCx
mpvBb16WoStfKrK38gt5/VGTY1PqO1yf4wicG8yAxXGxEFlK/8FWr4PRuwVIrXtE
H0JJeCwifVPLhUu+hSeK+nLH9GnSjLtDw14nM4/i+CdF+tcCZAcN/kO28kqRmItM
TzTPdA5HK/1Fe9tJPulFRDLbAxdCFSq2+c1dDjJEZLOjpxBqdKWSGH2BTWShOq8x
EISgLCDopYb0GrHCy9JzAC6TLrH37F3rwfc+sQRf4wryasV+LCPa5FFfakv7fKLt
FTj51hevgiQMcurCAXh7D9qhTVpNrEJAJNRvW6Qag3WEdaQ0Ehr6dl/wO1xjpmR6
LVRPcJ3xhu65DabuPLbGjRdmA4S3iuw4BAVQtGGANdXQg3BNmW+zkX5VZvEYRJKa
QTkAioS8WvLPxjcBSvmSy+4m5H/JztR/4JTwSOOt9PKS169G134x3NWFgu291I/u
Hd4rzU7H7gdBA9A6s6Xv+awlyzX7E5u6UPR1AUnsfJ/fxbQQFYRgtVgLZSbd4DRh
zZrZdRyJp34PmTIUy+UKPsl2lKRDeVAT4GLeSlXWKZIqYXRmrki0FPCGwf63xGFm
IFo0qQVL1zXSy2UI9hGeFf3mmmfh+BbW3nr7Skr6D+cd2MXkil4Vn8Bnnb94k27K
80V7nEjqakldm7xfhvovaGXndtFI5XVUZVoP1dYn5PL7vj8kflNgq6hxQeGMZuA=
=v8KT
-----END PGP SIGNATURE-----