[nycbug-talk] hadoop - sharing a server?

[Pat McEvoy]

On Thu, May 13, 2010 at 10:27 AM, Mark Saad <mark.saad at ymail.com> wrote:
> Pat
>  Ed the guy who gave the hadoop talk, uses Hadoop on VServer  on  Debian .
>
>  --
> Mark Saad
> mark.saad at ymail.com

Hey Mark,
Cheers for the info. I put together that video to practice knitting
slides and audio together in order to make a video. It took ages but I
am happy with the outcome. The video happened because the raw
materials for it were easily at hand at the time. Future content will
be much more BSD-centered, I promise.
P



>
>
>
> ----- Original Message ----
>> From: Pat McEvoy <mcevoy.pat at gmail.com>
>> To: Charles Sprickman <spork at bway.net>
>> Cc: talk at lists.nycbug.org
>> Sent: Wed, May 12, 2010 1:18:03 AM
>> Subject: Re: [nycbug-talk] hadoop - sharing a server?
>>
>> On Tue, May 11, 2010 at 11:55 PM, Charles Sprickman <
>> ymailto="mailto:spork at bway.net"
>> href="mailto:spork at bway.net">spork at bway.net> wrote:
>> Hi
>> all,
>>
>> I just recently went back and listened to the hadoop
>> presentation from a few
>> months ago.  The timing was great, as I've been
>> tasked with setting up a
>> basic hadoop environment for pulling some stats
>> out of a ton of mail logs.
>>  We'll likely be using HBase, but will be
>> looking at Pig as well.
>>
>> I have a 3-node test setup running on
>> FreeBSD 8.0 in VMWare.  I was
>> pleasantly surprised that Java was not a
>> real pain to get going.  In short,
>> this all looks good, and it looks
>> like it would be easy enough to copy one
>> of these nodes to a jail,
>> archive that jail, and then deploy a bunch of
>> these things all over the
>> place.
>>
>> So my question...  What we're looking to do with Hadoop
>> does not yet justify
>> going out and buying a half dozen or so servers.
>>  I'd like to jail it on a
>> bunch of our existing servers.  The nature of
>> the load on these things is
>> that they have widely varying workloads with
>> many lulls during the day.  The
>> nature of the jobs we want to run on the
>> hadoop cluster is that basically we
>> can wait as long as it takes for
>> now.  So is anyone running hadoop nodes on
>> servers not dedicated to this
>> task?  Does it respond to being niced down?
>>  Are there some resource
>> utiliztion knobs I've missed in all the quicky
>> howto's I've
>> read?
>>
>> Thanks,
>>
>> Charles
>>
>>
>> _______________________________________________
>> talk mailing
>> list
>>
>> href="mailto:talk at lists.nycbug.org">talk at lists.nycbug.org
>>
>> http://lists.nycbug.org/mailman/listinfo/talk
>>
>
> You can also watch
>> the Hadoop presentation
>> here:
> http://blip.tv/file/3276543
> P
> _______________________________________________
> talk
>> mailing list
>
>> href="mailto:talk at lists.nycbug.org">talk at lists.nycbug.org
>
>> href="http://lists.nycbug.org/mailman/listinfo/talk" target=_blank
>> >http://lists.nycbug.org/mailman/listinfo/talk
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>