[nycbug-talk] version control for config files

[Pete Wright]

On 7-Mar-09, at 10:32 PM, Charles Sprickman wrote:

> On Sun, 8 Mar 2009, Isaac Levy wrote:
>
>> On Mar 8, 2009, at 12:59 AM, David Lawson wrote:
>>
>>> Puppet is a really good option for config management, I looked  
>>> really
>>> seriously at it a while back and liked what I saw a lot, but I had  
>>> two
>>> major problems with it.  First, it's written in Ruby and I'm not a
>>> Ruby programmer, so that kind of bites if I wanted to extend it.
>>> Second, and nearly every configuration management system I've looked
>>> at suffers from this problem, there's a pretty serious bootstrap  
>>> cost
>>> to implementing it.  It's a reasonably complex system, it's non-
>>> trivial to understand how to make it do what you want it to do, and
>>> when you do get a good handle on it, converting all your systems to
>>> use it can be a pretty serious undertaking, depending on how many  
>>> you
>>> have.  At the time, I was looking at a couple hundred, with
>>> configurations drifted all over hell and gone, so that was a time  
>>> sink
>>> I couldn't afford.
>>>
>>> What I ended up doing is writing a stupid simple configuration  
>>> manager
>>> in python, we call it ghetto-config in the office.  I've actually  
>>> been
>>> thinking about asking to open source it, I'll talk to my boss  
>>> about it
>>> on Monday, but the basic concepts are simple, and it didn't take me
>>> more than an afternoon to implement a first cut.
>>>
>>> Ghetto-config works as a very simple templating engine, you define a
>>> number bunch of key value pairs, and you can use those keys in a  
>>> file
>>> and have ghetto-config substitute in the appropriate value for you
>>> when it parses it.  It also understands how to set file modes,  
>>> create
>>> symlinks, manage owners and groups, and how to diff version of a  
>>> file
>>> if it's changed.
>>>
>>> Basically, you assign each machine a unique ID of some sort at  
>>> install
>>> time (or later, if you want), we used the MAC address of the  
>>> interface
>>> that was used to PXE boot the machine for kickstart, but you could  
>>> use
>>> hostname or whatever if that was easier.  That serves as the unique
>>> identifier for the system.  Ghetto-config takes that information and
>>> uses it to build a URL to fetch configuration information from a
>>> central HTTP server storing config data.  It fetches the URL it  
>>> builds
>>> and gets back a file in config parser syntax (basically an .ini  
>>> file)
>>> with a couple special sections.  The first section is includes, so  
>>> it
>>> can include in other config parser syntax files.  The second is
>>> definitions, it allows to set up key-value pairs, like $eth0-ip$ =
>>> 192.168.1.15, and the third set of managed file sections.
>>>
>>> Each managed file section gives the URL to fetch the template from  
>>> (so
>>> you can use the same template file for multiple machines by just
>>> pointing to the URL of a canonical version) and the location on the
>>> file system to write the rendered template to once the substitutions
>>> have been performed.  It optionally includes a file mode, owner,
>>> group, and the location of a symlink to make to the file location.
>>>
>>> There's some additional detail in structuring the central config
>>> server and doing some other stuff that makes it simpler to manage,  
>>> but
>>> that's the gist of it.  It's about two hundred lines of python, it
>>> supports doing diffs between the central configuration and the local
>>> reality for any attribute it understands (so file contents, owner,
>>> group, mode, etc.) as well as a programmatic mode intended to be run
>>> from cron that'll tell you whether anything has changed on the  
>>> machine
>>> so you can make sure none of your machines have drifted nightly,  
>>> etc.
>>> We've started checking the central config tree into SVN so we have  
>>> an
>>> audit trail of who did what to what file.
>>>
>>> Does it do everything puppet does?  God no.  Does it do everything
>>> cfengine does?  Again, god no.  It's an 80/20 solution, it took me  
>>> an
>>> afternoon to write the first version of it, and maybe a week of  
>>> total
>>> development time to get it doing what it does and full test coverage
>>> for it.  It happily manages several hundred machines, it makes
>>> installing and provisioning a new machine a thirty second job  
>>> instead
>>> of a half hour or so, and it makes managing software installs over
>>> multiple machines much, much easier and more deterministic.
>>>
>>> Like I said, I've been planning on asking about open sourcing it
>>> anyway, but if I don't get to do that, I'll be happy to answer
>>> questions or give pointers where I can.
>>>
>>> --Dave
>>
>>
>> This sounds hot- I wanna try it :)
>
> Ditto.
>
> Specifically, it sounds simple enough to actually sit down and  
> implement,
> which is what has scared me away from everything else.
>


should check out cobbler if you all are interested in a provisioning  
mgt system:

https://fedorahosted.org/cobbler/

yea, it's currently RHEL/Fedora based, but i've used it for a while  
now to manage other OS's.  it's written in python, and it supports  
Cheeta templates inside kickstart/preseed/<other-install-auto- 
responder-here> files.  it also manages dhcpd and bind configs pretty  
well and has a webUI.

the developer works for redhat - but he's a super nice guy and really  
wants better BSD and non-rhel support for his project.

it integrates with puppet quite well too...

-p