[nycbug-talk] version control for config files

[Isaac Levy]

On Mar 8, 2009, at 12:59 AM, David Lawson wrote:

> Puppet is a really good option for config management, I looked really
> seriously at it a while back and liked what I saw a lot, but I had two
> major problems with it.  First, it's written in Ruby and I'm not a
> Ruby programmer, so that kind of bites if I wanted to extend it.
> Second, and nearly every configuration management system I've looked
> at suffers from this problem, there's a pretty serious bootstrap cost
> to implementing it.  It's a reasonably complex system, it's non-
> trivial to understand how to make it do what you want it to do, and
> when you do get a good handle on it, converting all your systems to
> use it can be a pretty serious undertaking, depending on how many you
> have.  At the time, I was looking at a couple hundred, with
> configurations drifted all over hell and gone, so that was a time sink
> I couldn't afford.
>
> What I ended up doing is writing a stupid simple configuration manager
> in python, we call it ghetto-config in the office.  I've actually been
> thinking about asking to open source it, I'll talk to my boss about it
> on Monday, but the basic concepts are simple, and it didn't take me
> more than an afternoon to implement a first cut.
>
> Ghetto-config works as a very simple templating engine, you define a
> number bunch of key value pairs, and you can use those keys in a file
> and have ghetto-config substitute in the appropriate value for you
> when it parses it.  It also understands how to set file modes, create
> symlinks, manage owners and groups, and how to diff version of a file
> if it's changed.
>
> Basically, you assign each machine a unique ID of some sort at install
> time (or later, if you want), we used the MAC address of the interface
> that was used to PXE boot the machine for kickstart, but you could use
> hostname or whatever if that was easier.  That serves as the unique
> identifier for the system.  Ghetto-config takes that information and
> uses it to build a URL to fetch configuration information from a
> central HTTP server storing config data.  It fetches the URL it builds
> and gets back a file in config parser syntax (basically an .ini file)
> with a couple special sections.  The first section is includes, so it
> can include in other config parser syntax files.  The second is
> definitions, it allows to set up key-value pairs, like $eth0-ip$ =
> 192.168.1.15, and the third set of managed file sections.
>
> Each managed file section gives the URL to fetch the template from (so
> you can use the same template file for multiple machines by just
> pointing to the URL of a canonical version) and the location on the
> file system to write the rendered template to once the substitutions
> have been performed.  It optionally includes a file mode, owner,
> group, and the location of a symlink to make to the file location.
>
> There's some additional detail in structuring the central config
> server and doing some other stuff that makes it simpler to manage, but
> that's the gist of it.  It's about two hundred lines of python, it
> supports doing diffs between the central configuration and the local
> reality for any attribute it understands (so file contents, owner,
> group, mode, etc.) as well as a programmatic mode intended to be run
> from cron that'll tell you whether anything has changed on the machine
> so you can make sure none of your machines have drifted nightly, etc.
> We've started checking the central config tree into SVN so we have an
> audit trail of who did what to what file.
>
> Does it do everything puppet does?  God no.  Does it do everything
> cfengine does?  Again, god no.  It's an 80/20 solution, it took me an
> afternoon to write the first version of it, and maybe a week of total
> development time to get it doing what it does and full test coverage
> for it.  It happily manages several hundred machines, it makes
> installing and provisioning a new machine a thirty second job instead
> of a half hour or so, and it makes managing software installs over
> multiple machines much, much easier and more deterministic.
>
> Like I said, I've been planning on asking about open sourcing it
> anyway, but if I don't get to do that, I'll be happy to answer
> questions or give pointers where I can.
>
> --Dave


This sounds hot- I wanna try it :)

Rocket-
.ike