[nycbug-talk] cfengine book

[Brian Cully]

On 3-Jul-2008, at 16:55, Jesse Callaway wrote:
> thanks for the suggestion. My takeaway from last night's meeting was
> that cfengine is entirely inappropriate for use where I work... too
> diverse of a base of computers. Too bad!

	If you're only using one OS, cfengine is a great tool for  
distribution even among a diverse set of workloads. If you're using  
more than one OS, it's not worth the headache to try and cram it all  
onto one master cfengine box. Just keep one cfengine box per OS  
install and you'll still be doing pretty good.

	I used to maintain about 16 servers with cfengine, doing a variety of  
tasks with different requirements. Each of the other groups I worked  
with did the same, and when I left that job, my cfengine box was  
rolled into a much larger one from my parent group, which ended up  
servicing over 50 machines across the country.

	Making this all work well depends on three things, really:

	1) graft (stow, or equivalent), so you can easily maintain various  
versions of software around (say, one for test, beta, and prod, or for  
app compatibility, or any number of reasons),
	2) a clean hierarchy to work with. We used to break stuff up into / 
local and /shared, where /local is machine-local, /shared has all our  
customizations (for everything, always, and this is a good thing). One  
of the other important things about this is that /everything/ in / 
local is either a symlink to something in /shared or auto-generated by  
script. And,
	3) A well designed heirarchy of cfengine configs. At this job, we  
used three levels: global, role, and host. Roles were also composable.  
Scripts and symlinks were used to change host/role assignment, such  
that it could all be accomplished from within cfengine (including  
ifconfig, restarts of services, or even the host itself).

	With the acknowledgement that Disk Space is Cheap, we clone the  
entire cfengine box to every host managed by it (sans a couple things  
in /etc, natch). This gives us network wide backups and (close- 
to-)instant-on solutions in the event of failure. It also vastly  
simplifies maintenance assuming you follow the above rules.

	The reason I don't like using a single cfengine box for multiple OSes  
is that I also like to use it as a build host and cross-compiling is a  
nightmare. Money was also never so tight I couldn't just get another  
box for it.

	Getting the system up, running, and humming can take quite a bit of  
work. It gets paid back very quickly, however. I can say without  
hesitation that cfengine was the only reason I was able to manage as  
many servers single-handedly as I did.

	One of the things you need to submit to, in order to fully utilize  
cfengine, is that you have to give it complete control of your hosts.  
Frankly, I've always thought my brain was too important for  
micromanaging hosts, so this wasn't a big deal for me. YMMV.

-bjc

P.S.
At the job previous, I set up rsync to do pretty much what I was  
shooting for cfengine to do later. rsync was substantially easier to  
comprehend and get working, but it is absolutely nowhere near as  
powerful. cfengine is a bit baroque, has tons of useless (or at least  
questionable) features, but does a bang-up job at almost anything you  
want it to do.