[nycbug-talk] Serving High Performance Web Sites --- UNIGROUP Meeting 22-MAY-2008 (Thu):

[Jonathan Vanasco]

On May 15, 2008, at 1:52 PM, forest mars wrote:
> Question for the list, who has made the jump from Apache to a  
> faster web server?

I mostly dropped Apache about 4 years ago.

Most of my work since 2003 has been under mod_perl - which for the  
unknowing, is a perl interpreter compiled into Apache with various  
hooks to turn it into an Application Server.

It works *really* well, and is a pure joy to use -- but speed  
improvements are gained with memory use.  The 'default' recommended  
setup is to run 2 versions of Apache -- one 'Vanilla' on port 80, and  
another one (or more) with mod_perl compiled in on the 8*** ports.

While working on an earlier project I worked on ( RoadSound.com ) I  
became an early adopter of lighttpd.  It was f*ing amazing.  Aside  
from being tiny and more robust than the other micro-servers, it was  
really portable.  You could run one lighttpd, proxy that to multiple  
ports on the same box with little overhead, and have tons of little  
lightys running php apps or whatever in isolated dirs or even jails.

Unfortunately, lighty had a *really* bad memory leak on the proxy  
module circa 2006.  It didn't close connections or something stupid,  
and your process would just grow and grow.  Fixing it was de- 
prioritized to develop new features and seemingly cater to youtube  
and other high profile sites.  I got frustrated and looked for new  
options.

I found nginx as a joke -- I looked to see what all the russian mp3  
porn warez & hacker sites were using.  i figured the russians are  
resourceful, seem to breed BRILLIANT mathematicians and engineers,  
write lean code, and tend to use older tech than us.  I also figured  
that a lot of those sites run on compromised boxes -- so a server  
would have to be small.   I rationalized that they might have some  
super-fast server systems - and was right.  I found nginx by checking  
header info for all the 'grey' russian sites -- they all used it.

I wasn't ready to make the jump, but Bob ( Ippolito , Mochimedia )  
was -- lighttpd had to be restarted every hour or so on their  
cluster.  He jumped and raved about it "it just fucking works like it  
should!" or something like that.  I switched onto it too , and have  
been ecstatic ever since.  it works like it should, and there just  
aren't bugs all that often.

On one of my typical boxes, i have
	80 nginx
		static content + php fcgi
	8080-8089 apache::mod_perl
		mod_perl apps ; usually 1-4 different 'master' apaches, and a  
handful of children

for my concerns, speed/throughput isn't as important as memory.   
nginx takes up no cpu/mem -- which means I can push that to mod_perl  
or php.  under apache, its something like 20 MB for the parent  
process and ~ 10 for each child once they respawn ( they all start  
out at 5MB, but seem to max at 20/10 after a few thousand requests )  
-- which comes out to 80MB total.  with nginx, i get another 70MB of  
usable memory for apps.  I like that.


in terms of throughput -- IIRC, there are some kernel limits to  
FreeBSD in terms of connections, and you need to jump to linux in  
order to really get a fast/high octane server.