[nycbug-talk] shared hosting

[Isaac Levy]

Wordup Sunny,

On Jan 26, 2005, at 8:25 PM, Sunny Dubey wrote:

> On Wednesday 26 January 2005 18:47, Agent Smith wrote:
>> Hi Sunny, All,
>>
>> On Jan 25, 2005, at 8:25 AM, Sunny Dubey wrote:
>>> On Monday 24 January 2005 14:33, Isaac Levy wrote:
>>>> Noo!  Please respond on-list, I'd like to know too!
>>>
>>> I can provide virtualized NetBSD hosting.  You log in as root do
>>> whatever you
>>> want, run your own networking firewalls, etc etc etc.  This is *not*
>>> jail()
>>> or any jail() derivative, it is something more powerful/flexible
>>> called XEN.
>>
>> I don't mean to be confrontational with what folows, seeing as you and
>> I have a decent dialouge with some lively banter, but I gotta ask you
>> to put your money where your mouth is with XEN here man.
>>
>
> nothing is confrontational .... (not until we have our Matrix style 
> fight
> involving flying, guns, and the 'truth' .. lol)

Next nyc*bug meeting- flying 30 ft. above the stairway at apple...  
lets rumble...

>
>> I never think it's fair to say one thing is more or less
>> powerful/flexible than another, and of course, your bound to meet
>> resistance pimping a Linux based product on a BSD mailing list <g>.
>>
>
> Linux based yes, but the end result is a pure NetBSD experience for 
> the end
> user.  The user will watch his/her the NetBSD kernel boot up, the user 
> will
> log into a NetBSD system with NetBSD libraries and NetBSD tools.

This is pretty darned fascinating, I gotta say.

>
> In that sense, its very much like IBM's VM product.  IBM sells their 
> z-Series
> with the ability to run many linux instances all at once.  (Well, its 
> a bit
> more complicated than that, but this is long story short).  People 
> don't care
> for the s390 architecture too much, nor do they care about VM ... they 
> just
> want to run as many linux'es as they want for whatever reason they 
> want (Once
> again, we'd have to start talking about things liked "partitioned 
> hardware"
> to get any real meat into this XEN/VM comparsion discussion)

Right.

>
> XEN is essentially a linux machine.  If you lock down the linux 
> machine good
> enough (IE: no open ports, except for SSH on an off port), it becomes 
> just as
> secure as any other machine from an external point of view.  (Internal
> security is an entire subject of its own.  But unlike UML, exploiting 
> the
> virtualized kernel in XEN does not give you privileges on the host.)

Ok- that's quite cool and interesting as well.  How does it work?  I.E. 
if a cracker exploits a virtualized kernel, does XEN keep some kind of 
jail-like process tree?  (i.e. does the kernel itself then die as a 
chrooted process or somesuch?)  Or does a process have a life beyond 
the XEN'd process tree?

>
> So in the end, whether is a linux based product or not is somewhat 
> irrelevant.

Well, that's a matter of preference and opinion, (i.e. not worth you 
and I going there now <g>), but your point is well taken...

> Its all about the stability of the guest OS and the dedication of the
> developers of the guest OS to maintain and support the port of 
> whatever guest
> OS.  (in this case the guest OS is Netbsd:  
> http://netbsd.org/Ports/xen/ )

Gotcha.

>
>>
>> --
>> Question:
>>
>> Sunny- for the purposes of the archive for this BSD oriented list, can
>> you please explain *why* XEN is more powerful/flexible than jail(8)?
>>
>
> Because you are booting the kernel, and have full control of *almost
> everything* once inside the kernel.
>
> I could easily compile 13 different NetBSD kernels each with a 
> different patch
> and boot them all at once to do mass testing.  It really doesn't get 
> easier
> than that.

Ok- again, like our conversations before re. UML compared to jail(8), 
I'd come to similar conclusions here?  I.E. each system has strengths 
and weakness in a given context.  That said, I can't think of many 
applications where I'd *want* different kernels on a VM cluster for 
webservers?

Sunny, you do a lot of this, have you run into any specific instances 
of applications/requirements where you need different kernels running 
on the net from the same box?  (Aside from development which is all 
about some core kernel-level dev, I'm talking about Application here).  
I'm really trying to come up with a use case that challenges my own 
practices here...  (which leads to the stripped-down homogeny of 
jailing)...

>
> Additionally (as Marc had suggest) I could booth 13 differnet NetBSD 
> installs
> each with a different ipf/pf/etc script and try tinkering with the 
> networking
> till my face was blue, heh.  This includes control of ICMP, etc

Ok- this is one use case, but still on the dev. side- I'm talking 
micro-complexity in production baby...

>
>> I'm not sure, but for a BSD based mailing list, I'd think that it
>> should be clearly stated that you'd be running a NetBSD Virtual 
>> Machine
>> image, running inside of XEN, on a Linux distro?
>>
>
> its inside XEN with is running on a linux distro.  But as stated 
> above, its
> not entirely important.

Right, not from the hostee's perspective, somewhat irrelevant.  But 
from the .ike perspective, (licensing, stability, security, man page 
gripes) I'll choose not to comment until we can both come at each other 
matrix style etc... <g>

>
>> --
>> Additional Questions:
>>
>> - Does XEN support VM's of other OS's, or does only NetBSD fit the 
>> bill
>> due to it's rep. for hardware compatability?  (i.e. to meet some funky
>> stuff in the VM?)
>
> XEN can support any OS, but that OS must be ported to the 'XEN 
> architecture'.
> Currently this includes linux and NetBSD, with FreeBSD support coming 
> along
> the way.  And being that OpenBSD folks don't take it seriously.

/me waits for the day when I can run jail(8)s inside of one of Sunny's 
Xen boxen...

>
> XEN itself is currently limited to x86 and amd64.  There are no plans 
> of
> changing this.  (intel and AMD apparently have people looking into XEN 
> to
> ensure that it will work with whatever emt64/amd64 hardware 
> partitioning
> exists in the future.  I guess they want to compete with IBM's
> VM/s390/zSeries combination *shrug*)
>
> The following is a bit on the super technical side, and so I could 
> easily be
> wrong ....
>
> buuuutt I'm going to try anyways
>
> lets suppose NetBSD has a foobar() syscall.  Implementing foobar() on 
> x86
> requires different code than ppc64, obviously.  on x86 foobar() pretty 
> much
> needs MMU support and nothing more.  But on ppc64 foobar() not only 
> needs MMU
> support, but it can take advantage of other ppc64 target features.  So 
> two
> sets of code is written for foobar().
>
> XEN is the same sorta target.  If I as a NetBSD developer wanted 
> foobar() on
> XEN, I would have to write code to this 'XEN architecture'.
>
> I believe this is the way it works.

Makes sense, more or less- and really pretty damn interesting.

>
>>
>> - Are there any fundamental differences in how the VM accesses 
>> devices,
>> memory, networking, etc... which is different than in jail(8)'ing or
>> User Mode Linux (UML)?
>>
>
> I'm not too keen on the details with jail(8), and UML to some extent 
> is a bit
> different than XEN.  But suffice to say, Alex's answer is some what on 
> the
> mark.  XEN creates a quasi-architecture, that the host needs to provide
> support for, as well as the guest OSes need to support.

Dig- dug.

>
>> - How long has XEN existed, and beyond the commercial backing, what
>> kind of history does XEN have for stability and maturity in
>> production-level environments, especially large-scale systems running
>> on the public internet?
>>
>
> apparently XEN has existed for 3 years, but I have to admit a whole 
> bunch of
> us didn't give it much thought till sometime mid-last year.  Redhat 
> and SuSE
> are working to integrate XEN into their distros.  (Mandrake has me! 
> haha)
> IBM stands behind it (I suppose they too are aware that efficient
> virtualization will soon be a commoddity product).  Intel and AMD stand
> behind it, largely for their own CPU partitioning schemes.
>
> There is also XenSource which is a company that supports the code base 
> and
> such.

That's very cool.

>
>> - How does XEN fundamentally work?  I have read the XEN virtual 
>> machine
>> papers, available here,
>>
>> http://www.cl.cam.ac.uk/Research/SRG/netos/xen/performance.html
>>
>> but it only seems to cover like systems- and seems to be focused
>> primarily on performance (an issue with classical emulation systems).
>>
>
> XEN works by creating a quasi-architecture on top x86.  So the amount 
> of work
> needed to port an x86 friendly OS is very little.  And none of the 
> userland
> libraries and tools have to be touched at all to work out-of-the-box.

'Nod.

>
>> jail(8) is fundamentally ridiculously simple by design, which I see as
>> an important factor when working with the complexity which arises, in
>> the context of virtualizing services as complex as Operating Systems.
>> How does XEN help an administrator manage the complexity,
>
> xen comes with a control panel app and was built with mass hosting in 
> mind.
> This I suppose is the single and biggest reason why people are so much 
> more
> interested in it than the traditional UML.  The traditional UML was a 
> more of
> hack'ish sorta thing in which everyone wrote their own tools for and 
> such.

EEEWwwwww... Like GUI?  :/

>
>> and how
>> complex is the actual virtualization mechanism itself?  (i.e. is it a
>> 'large' software like VMWare and the like? [I'm asking in the context
>> of the relatively few lines of kernel code that make up the whole of
>> Jail(8)])
>
> virtualization can be fairly complex (vmware, VM) or just complex.  
> XEN is
> half and half in a way the allows for minimal performance loss.  I 
> hope what
> i've typed above gives one a sense of what XEN is in relation to Hosts 
> and
> Guests.
>
> HTH
>
> Sunny Dubey

Yo Sunny, thanks for the informative and thoughtful answers.  I'm still 
not giving up jail(8) any time soon for my needs in production-level, 
redundancy oriented, internet service; actually makes me cling to 
jailing even harder, HOWEVER, this is some insanely cool tech. man- and 
I believe I'm now a sideline fan of Xen alltogether- for whatever 
that's worth as we all proceed doing our thing...

Rocket-
.ike