[nycbug-talk] 5.4 Jails, nullfs or unionfs?

[Charles Sprickman]

On Tue, 19 Apr 2005, Isaac Levy wrote:

> Turning on the -v flag here,

and how!  Thanks for the excellent overview.  That should all be on a 
webpage somewhere.

[big snip]

> In FreeBSD 5.x devices are mounted using mount_devfs.  This is terrific for 
> jailing, insomuch as the start scripts for a given jail can contain flags to 
> mount_devfs to hide various devices, and the jail never gets them- it's that 
> simple.

Just to add to this, /etc/defaults/devfs.rules has a ruleset (#4, at the 
bottom) that is given as an example for a jailed system.

> Now, with regard to nullfs and unionfs, I have *no idea* what the state of 
> these are for FreeBSD, or for jails- but I'm not personally aware of any 
> manditory use cases for these in jails to begin with- though I can think of 
> things which would become nicer to manage, (a single update to a ports tree 
> for all systems perhaps, or a single user-land image template for massively 
> parallel jailed clusters, etc...), but these cases, to me, seem to be better 
> suited to chrooted enviornments, because of the implied homogeneity- so I'm 
> stumped, (and looking for a reason to get exited about nullfs or unionfs!)...

I currently don't do anything with jails, or 5.x for that matter.  But in 
looking for new services to offer and new business models as far as 
"dedicated/shared" web hosting and all the permutations of that that stem 
from multiple "virtual hosts" on one big box, I'm starting to wonder about 
what the easiest way to manage all this is.

It seems like the dumb-simple way is to come up with a set of configs for 
each jail and then just do an installworld into the jail from the main 
host.  What interested me about nullfs or mount_union is that you could 
really save a ton of space and work if you had all of the stuff needed by 
all jails shared read-only off of the "host" system.  From looking at the 
archives, people do this with nullfs quite a bit.  The gotcha here of 
course is that both nullfs and unionfs have rather stern warnings in their 
manpages about actually using them for anything important.

It's odd that there's not more info out there on the subject, as I have to 
imagine that many of these $30/month "dedicated server" folks rely on 
FreeBSD and jails to make their business work.  I find it hard to believe 
they do 100 installworlds to update a box full of jails. :)

Maybe we should lure "scrappy at hub.org" down for a talk at one of our 
meetings.  He seems to be the main bugfinder in the area and it would be 
interesting to hear how he goes about all this stuff.  It seems one of his 
long-running issues that occured as a result of "stacking" filesystems was 
that his machines would run out of vnodes, or worse, something was leaking 
vnodes...

My first foray into 5.x and jails is going to be a new shell server for a 
client.  If you believe that security comes from layering protection, 
putting the shell users in a jail makes sense.  And then when I think of 
what else I can do with this thing that will also be a repository for lots 
of shareable content (personal pages, blogs, possibly an 
AFP/SMB-accessible backup service, etc...) segregating some of these 
services sounds like a good idea.

Anyhow, thanks for all the info, that message goes in the permanent 
collection.  You should think about doing a DN on jail changes in 5.x. 
5.4 looks like it's getting to the point where more people may entertain 
running it in a production environment.

Thanks,

Charles

> Rocket-
> .ike
>
>