From george at ceetonetechnology.com  Fri Dec  2 10:05:27 2011
From: george at ceetonetechnology.com (George Rosamond)
Date: Fri, 02 Dec 2011 10:05:27 -0500
Subject: [nycbug-talk] Crypto Class
Message-ID: <4ED8E937.8070309@ceetonetechnology.com>

 From Schneier's blog:

www.crypto-class.org

It's been going on for years, but it's still remarkable that such things 
are available online, and for free.  And from Stanford, no less.

It's particularly surprising considering that university education 
inflation is only beaten by health care costs in the US.

g


From akosela at andykosela.com  Fri Dec  2 11:54:57 2011
From: akosela at andykosela.com (Andy Kosela)
Date: Fri, 2 Dec 2011 17:54:57 +0100
Subject: [nycbug-talk] Crypto Class
In-Reply-To: <4ED8E937.8070309@ceetonetechnology.com>
References: <4ED8E937.8070309@ceetonetechnology.com>
Message-ID: <CALMnNGibGd+2NzXPYM-66Zfp-Qe9C7+im0tk+nDCZjv+EZE7Mg@mail.gmail.com>

On Fri, Dec 2, 2011 at 4:05 PM, George Rosamond
<george at ceetonetechnology.com> wrote:
> From Schneier's blog:
>
> www.crypto-class.org
>
> It's been going on for years, but it's still remarkable that such things are
> available online, and for free. ?And from Stanford, no less.
>
> It's particularly surprising considering that university education inflation
> is only beaten by health care costs in the US.

George, thanks for this.  It seems there exist also other very
valuable courses there.  This one looks also promising:

 http://www.algo-class.org/

--Andy


From branto at branto.com  Sat Dec  3 09:32:08 2011
From: branto at branto.com (Brant Ian Stevens)
Date: Sat, 3 Dec 2011 09:32:08 -0500
Subject: [nycbug-talk] Crypto Class
In-Reply-To: <CALMnNGibGd+2NzXPYM-66Zfp-Qe9C7+im0tk+nDCZjv+EZE7Mg@mail.gmail.com>
References: <4ED8E937.8070309@ceetonetechnology.com>
	<CALMnNGibGd+2NzXPYM-66Zfp-Qe9C7+im0tk+nDCZjv+EZE7Mg@mail.gmail.com>
Message-ID: <7B1930E9-6409-4D67-B181-0FFA915F298A@branto.com>

OCW has been one of the great achievements of the Internet Era, IMO.

On Dec 2, 2011, at 11:54 AM, Andy Kosela wrote:

> On Fri, Dec 2, 2011 at 4:05 PM, George Rosamond
> <george at ceetonetechnology.com> wrote:
>> From Schneier's blog:
>> 
>> www.crypto-class.org
>> 
>> It's been going on for years, but it's still remarkable that such things are
>> available online, and for free.  And from Stanford, no less.
>> 
More Stanford Content:
http://see.stanford.edu/

MIT Content:
http://ocw.mit.edu/index.htm

>> It's particularly surprising considering that university education inflation
>> is only beaten by health care costs in the US.
> 
> George, thanks for this.  It seems there exist also other very
> valuable courses there.  This one looks also promising:
> 
> http://www.algo-class.org/
> 
> --Andy
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk



From akosela at andykosela.com  Sat Dec  3 16:55:35 2011
From: akosela at andykosela.com (Andy Kosela)
Date: Sat, 3 Dec 2011 22:55:35 +0100
Subject: [nycbug-talk] Crypto Class
In-Reply-To: <7B1930E9-6409-4D67-B181-0FFA915F298A@branto.com>
References: <4ED8E937.8070309@ceetonetechnology.com>
	<CALMnNGibGd+2NzXPYM-66Zfp-Qe9C7+im0tk+nDCZjv+EZE7Mg@mail.gmail.com>
	<7B1930E9-6409-4D67-B181-0FFA915F298A@branto.com>
Message-ID: <CALMnNGhOM1sMp++z1hbp_cFEc8j6=hy+1Sop3gZaXq1LfUvztQ@mail.gmail.com>

On Sat, Dec 3, 2011 at 3:32 PM, Brant Ian Stevens <branto at branto.com> wrote:
> OCW has been one of the great achievements of the Internet Era, IMO.

Yes, OCW is a remarkable achievement.  I checked some of their online
courses on CS and it's definetly a top quality material.  It shows a
great potential of the Internet in educating.  I particularly very
much enjoy video sessions where you can almost feel like you are
there...

That's why I also love very much BSD Conferences and similar channels
on youtube :)

--Andy


From george at ceetonetechnology.com  Sun Dec  4 11:13:51 2011
From: george at ceetonetechnology.com (George Rosamond)
Date: Sun, 04 Dec 2011 11:13:51 -0500
Subject: [nycbug-talk] FreeBSD logo
Message-ID: <4EDB9C3F.5030505@ceetonetechnology.com>

Assume some others caught the recent thread about the FreeBSD logo on 
FBSD chat.  Basically, some guest at a Wyoming hotel were horrified by 
the FBSD logo on the wireless device.

So for all those haunted by daemons:

http://box.jisko.net/i/de9679fd.png

Now someone has to rewrite all the daemons into angels.

/etc/rc.d needs to be changed to /etc/rc.a

g


From jamex1642 at gmail.com  Mon Dec  5 16:49:52 2011
From: jamex1642 at gmail.com (James Reynolds)
Date: Mon, 5 Dec 2011 15:49:52 -0600
Subject: [nycbug-talk] UNIX TIPS - Can you ever have enough?
Message-ID: <CALCVq+GNDm-bZqheedaVfYvN9YqJhyyhqkEzXi8st0rRLQ8PMw@mail.gmail.com>

http://www.in-ulm.de/~mascheck/


From submodd at gmail.com  Wed Dec  7 10:59:46 2011
From: submodd at gmail.com (gm)
Date: Wed, 7 Dec 2011 10:59:46 -0500
Subject: [nycbug-talk] I look forward to the holiday party tonight
Message-ID: <CAOYiD9MQRqycJz1YQtJeyZi+oij4tagd_KNMd70z2byf7nqS+w@mail.gmail.com>

FYI if you want to see me expose heartless Georgia Tech and ACM on NY1 as I
endlessly look for software engineering work you can watch it at
http://www.ny1.com/content/150699/startups-provide-non-traditional-spin-to-job-fairs

George
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111207/23ca750e/attachment.htm>

From ike at blackskyresearch.net  Thu Dec  8 08:54:48 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Thu, 8 Dec 2011 08:54:48 -0500
Subject: [nycbug-talk] Presentation Slides
Message-ID: <201112081355.pB8Dt4X8000319@rs134.luxsci.com>

Hi All,

My presentation slides from last night are up here:
http://blackskyresearch.net/presentations/2011_12_07-A_Footnote_on_Inappropriate_Cloud_Use/

http://blackskyresearch.net/presentations/2011_12_07-A_Footnote_on_Inappropriate_Cloud_Use/misc/flav-unixdead.png

Rocket-
.ike




From nikolai at fetissov.org  Thu Dec  8 09:57:21 2011
From: nikolai at fetissov.org (Nikolai Fetissov)
Date: Thu, 8 Dec 2011 09:57:21 -0500
Subject: [nycbug-talk] December 2011 meeting audio
Message-ID: <658341e145166233e718e3eb20d812d0.squirrel@geekisp.com>

Folks,

Audio recording of 2011 holiday meeting is in two parts:

http://www.fetissov.org/public/nycbug/nycbug-12-07-11.1.mp3
http://www.fetissov.org/public/nycbug/nycbug-12-07-11.2.mp3

The quality is not the greatest because of the microphone effects.

Cheers,
--
 Nikolai



From george at ceetonetechnology.com  Fri Dec  9 10:40:07 2011
From: george at ceetonetechnology.com (George Rosamond)
Date: Fri, 09 Dec 2011 10:40:07 -0500
Subject: [nycbug-talk] hardware testing
Message-ID: <4EE22BD7.4050508@ceetonetechnology.com>

How do others deal with hardware testing?

You have the parts, the boxes built, new or used, how do you go about 
testing the RAM, drives, etc?

I've used FreeBIE and other types of utilities, but I really would like 
to use or build some sort of self-contained, scripted, cd or USB 
bootable tool.  Boot it off the removable media, and get a full review 
the next day, if necessary.

Anyone?

g


From edlinuxguru at gmail.com  Fri Dec  9 10:58:19 2011
From: edlinuxguru at gmail.com (Edward Capriolo)
Date: Fri, 9 Dec 2011 10:58:19 -0500
Subject: [nycbug-talk] hardware testing
In-Reply-To: <4EE22BD7.4050508@ceetonetechnology.com>
References: <4EE22BD7.4050508@ceetonetechnology.com>
Message-ID: <CAENxBwxVDeuYRH=ZffKom87+-FHqJxUWPnZR6og5Eck13Q_Y8A@mail.gmail.com>

I do not always get the chance to burn in hardware as I would like. I am
not as much looking for a performance report, as I am looking to ensure the
machine will not fall over as soon as load hits it.
Flaky NIC driver
Bad disks in RAID set or smart type errors
Since I am in to Apache Cassandra these days, I just use a NoSQL benchmark
tool. https://github.com/brianfrankcooper/YCSB/wiki

NoSQL is good at benchmarking disk, memory, network, and creates some solid
processing load. So I just beat up the node for hours also you can compare
the stats vs other classes of hardware and see how the system does.

Edward

On Fri, Dec 9, 2011 at 10:40 AM, George Rosamond <
george at ceetonetechnology.com> wrote:

> How do others deal with hardware testing?
>
> You have the parts, the boxes built, new or used, how do you go about
> testing the RAM, drives, etc?
>
> I've used FreeBIE and other types of utilities, but I really would like to
> use or build some sort of self-contained, scripted, cd or USB bootable
> tool.  Boot it off the removable media, and get a full review the next day,
> if necessary.
>
> Anyone?
>
> g
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111209/606ae14d/attachment.htm>

From pete at nomadlogic.org  Fri Dec  9 16:24:11 2011
From: pete at nomadlogic.org (Pete Wright)
Date: Fri, 09 Dec 2011 13:24:11 -0800
Subject: [nycbug-talk] hardware testing
In-Reply-To: <4EE22BD7.4050508@ceetonetechnology.com>
References: <4EE22BD7.4050508@ceetonetechnology.com>
Message-ID: <4EE27C7B.3060201@nomadlogic.org>

On 12/9/11 7:40 AM, George Rosamond wrote:
> How do others deal with hardware testing?
>
> You have the parts, the boxes built, new or used, how do you go about 
> testing the RAM, drives, etc?
>
> I've used FreeBIE and other types of utilities, but I really would 
> like to use or build some sort of self-contained, scripted, cd or USB 
> bootable tool.  Boot it off the removable media, and get a full review 
> the next day, if necessary.
>
> Anyone?
i find using pgbench will give you a good burn-in on most hardware 
types.  it'll exercise two types of disk i/o - pg_xlog sync writes, and 
normal db writes.  it'll also allow you to exercise your cpu/cores and 
utilize memory.  the only missing component is stressing your network i/o.

having said that - i don't know of any off-the-shelf benchmarking suites 
for freebsd...might be a good time to make one :)

-pete


-- 
Pete Wright
pete at nomadlogic.org
www.nomadlogic.org



From matt at tablethotels.com  Fri Dec  9 20:08:31 2011
From: matt at tablethotels.com (Matthew Story)
Date: Fri, 9 Dec 2011 20:08:31 -0500
Subject: [nycbug-talk] anyone know why install functions this way?
Message-ID: <34B61D8A-8013-454A-8FF4-7E7A069FC02B@tablethotels.com>

Know this isn't a bug list or anything like that, but figured I'd throw this out there ... the only thing I found on the interweb was regarding RPMs on CentOS ... so this is apparently not even a BSDism, but it makes writing my installer scripts vastly less reliable.  Lots more below, but the gist is this:

install -o root -d foo/

Will create the directory with my user and effective group id, and my umask ... and yield an error message ... but still exit 0 (success).  At the very least I would expect the directory to linger with these permissions and exit non-zero (failure) ... this is also not consistant with the behavior of files.

Anyway ... anyone know a historical reason for this, or have a good hack around this ... getting patches into section 1 to trickle down into my world takes a while, assuming the behavior is even a bug ...

------------------------lots of copy-paste below----------------------------------

[matt ~]$ mkdir install-showcase
[matt ~]$ cd install-showcase/
[matt ~/install-showcase]$ who am i
matt             14       Dec  9 19:52 
[matt ~/install-showcase]$ install -d ./test.dir; echo $? # install a directory as me, success!
0
[matt ~/install-showcase]$ ls -d ./test.dir; echo $? # see, success!
./test.dir
0
[matt ~/install-showcase]$ install -o root -d ./test.dir.1; echo $? # install a directory as root, success?!                                                                   
install: chown 0:4294967295 ./test.dir.1: Operation not permitted
0
[matt ~/install-showcase]$ ls -d ./test.dir.1 # see, success!
./test.dir.1
[matt ~/install-showcase]$ ls -d ./test.dir.1; echo $? # see, success!                                                                                                         
./test.dir.1
0
[matt ~/install-showcase]$ install -o doesnotexist -d ./test.dir.2; echo $? # install directory as a user that doesn't exist, failure!
install: doesnotexist: Invalid argument
67
[matt ~/install-showcase]$ touch filetobeinstalled; ls filetobeinstalled; echo $? # but does it work with files?                                                               
filetobeinstalled
0
[matt ~/install-showcase]$ install filetobeinstalled test.file; echo $? # install a file as me, success!                                                                       
0
[matt ~/install-showcase]$ ls filetobeinstalled; echo $?
filetobeinstalled
0
[matt ~/install-showcase]$ install -o root filetobeinstalled test.file.1; echo $? # install a file as root, success?                                                           
install: test.file.1: chown/chgrp: Operation not permitted
71
[matt ~/install-showcase]$ ls test.file.1; echo $? # not even a lingering file                                                                                                 
ls: test.file.1: No such file or directory
1
[matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode?                                                                     
install: chown 0:4294967295 test.dir.2: Operation not permitted
0
[matt ~/install-showcase]$ ls -d test.dir.2; echo $? # yup ... safe mode too ... :(
test.dir.2
0
[matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode ... with an existing target
install: chown 0:4294967295 test.dir.2: Operation not permitted
0
[matt ~/install-showcase]$ ls -ld test.dir.2; echo $? # yup ... safe mode too ... :(
drwxr-xr-x  2 matt  matt  2 Dec  9 20:00 test.dir.2
0
[matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode ... with an existing target
install: chown 0:4294967295 test.dir.2: Operation not permitted
0

From matt at tablethotels.com  Mon Dec 12 09:41:58 2011
From: matt at tablethotels.com (Matthew Story)
Date: Mon, 12 Dec 2011 09:41:58 -0500
Subject: [nycbug-talk] anyone know why install functions this way?
In-Reply-To: <34B61D8A-8013-454A-8FF4-7E7A069FC02B@tablethotels.com>
References: <34B61D8A-8013-454A-8FF4-7E7A069FC02B@tablethotels.com>
Message-ID: <4556A9E5-7168-4A4D-8EAC-D53F115ED14F@tablethotels.com>

Some additional information, install seems to fail when you specify a non-existant user on lookup, not chmod, if you specify a non-existant UID it succeeds again:

[matt at matt0 ~/install-showcase]$ install -o 999 -d test.dir.7; echo $? # see, success
install: chown 999:4294967295 test.dir.7: Operation not permitted
0
[matt at matt0 ~/install-showcase]$ ls -ld test.dir.7
drwxr-xr-x  2 matt  matt  2 Dec 12 09:28 test.dir.7
[matt at matt0 ~/install-showcase]$ id 999
id: 999: no such user

A couple of work-arounds I'm using for now below (if anyone is interested) ...

# assume success until failure method ...
shout() { echo "$0: $*" >&2; }
barf() { shout "$*"; exit 111; }
cleanup() {
    rc="$?"
    [ ! -z "$working" -a -e "$working" ] && rm -r "$working"
    exit "$rc"
}
dir_install() {
    eval working=\"\$$#\"
    trap cleanup INT TERM EXIT
    "$@" 2>&1 | { 
        read err 
        [ -z "$err" ] || barf "cannot install dir: $working"
    } || exit $?
    trap "" INT TERM EXIT
}
dir_install -o foo -d "some.dir"

# test for failure up front, with a file
test_file="test_install-d.$$.`hostname`.`date +%s`"
install -o "user" -g "group" "$test_file"|| rc=$?
[ -e "$test_file" ] && rm "$test_file"
[ 0 -ne "$rc" ] && exit "$rc"
install -o "user" -g "group" -d "$dir" || exit $?


On Dec 9, 2011, at 8:08 PM, Matthew Story wrote:

> Know this isn't a bug list or anything like that, but figured I'd throw this out there ... the only thing I found on the interweb was regarding RPMs on CentOS ... so this is apparently not even a BSDism, but it makes writing my installer scripts vastly less reliable.  Lots more below, but the gist is this:
> 
> install -o root -d foo/
> 
> Will create the directory with my user and effective group id, and my umask ... and yield an error message ... but still exit 0 (success).  At the very least I would expect the directory to linger with these permissions and exit non-zero (failure) ... this is also not consistant with the behavior of files.
> 
> Anyway ... anyone know a historical reason for this, or have a good hack around this ... getting patches into section 1 to trickle down into my world takes a while, assuming the behavior is even a bug ...
> 
> ------------------------lots of copy-paste below----------------------------------
> 
> [matt ~]$ mkdir install-showcase
> [matt ~]$ cd install-showcase/
> [matt ~/install-showcase]$ who am i
> matt             14       Dec  9 19:52 
> [matt ~/install-showcase]$ install -d ./test.dir; echo $? # install a directory as me, success!
> 0
> [matt ~/install-showcase]$ ls -d ./test.dir; echo $? # see, success!
> ./test.dir
> 0
> [matt ~/install-showcase]$ install -o root -d ./test.dir.1; echo $? # install a directory as root, success?!                                                                   
> install: chown 0:4294967295 ./test.dir.1: Operation not permitted
> 0
> [matt ~/install-showcase]$ ls -d ./test.dir.1 # see, success!
> ./test.dir.1
> [matt ~/install-showcase]$ ls -d ./test.dir.1; echo $? # see, success!                                                                                                         
> ./test.dir.1
> 0
> [matt ~/install-showcase]$ install -o doesnotexist -d ./test.dir.2; echo $? # install directory as a user that doesn't exist, failure!
> install: doesnotexist: Invalid argument
> 67
> [matt ~/install-showcase]$ touch filetobeinstalled; ls filetobeinstalled; echo $? # but does it work with files?                                                               
> filetobeinstalled
> 0
> [matt ~/install-showcase]$ install filetobeinstalled test.file; echo $? # install a file as me, success!                                                                       
> 0
> [matt ~/install-showcase]$ ls filetobeinstalled; echo $?
> filetobeinstalled
> 0
> [matt ~/install-showcase]$ install -o root filetobeinstalled test.file.1; echo $? # install a file as root, success?                                                           
> install: test.file.1: chown/chgrp: Operation not permitted
> 71
> [matt ~/install-showcase]$ ls test.file.1; echo $? # not even a lingering file                                                                                                 
> ls: test.file.1: No such file or directory
> 1
> [matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode?                                                                     
> install: chown 0:4294967295 test.dir.2: Operation not permitted
> 0
> [matt ~/install-showcase]$ ls -d test.dir.2; echo $? # yup ... safe mode too ... :(
> test.dir.2
> 0
> [matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode ... with an existing target
> install: chown 0:4294967295 test.dir.2: Operation not permitted
> 0
> [matt ~/install-showcase]$ ls -ld test.dir.2; echo $? # yup ... safe mode too ... :(
> drwxr-xr-x  2 matt  matt  2 Dec  9 20:00 test.dir.2
> 0
> [matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode ... with an existing target
> install: chown 0:4294967295 test.dir.2: Operation not permitted
> 0
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
> 



From matt at tablethotels.com  Mon Dec 12 09:55:39 2011
From: matt at tablethotels.com (Matthew Story)
Date: Mon, 12 Dec 2011 09:55:39 -0500
Subject: [nycbug-talk] anyone know why install functions this way?
In-Reply-To: <4556A9E5-7168-4A4D-8EAC-D53F115ED14F@tablethotels.com>
References: <34B61D8A-8013-454A-8FF4-7E7A069FC02B@tablethotels.com>
	<4556A9E5-7168-4A4D-8EAC-D53F115ED14F@tablethotels.com>
Message-ID: <EA1F6346-C878-4260-B7E9-D56609ECFFCC@tablethotels.com>

one correction below ...

On Dec 12, 2011, at 9:41 AM, Matthew Story wrote:

> [snip...]
> A couple of work-arounds I'm using for now below (if anyone is interested) ...
> 
> # assume success until failure method ...
> shout() { echo "$0: $*" >&2; }
> barf() { shout "$*"; exit 111; }
> cleanup() {
>    rc="$?"
>    [ ! -z "$working" -a -e "$working" ] && rm -r "$working"
>    exit "$rc"
> }
> dir_install() {
>    eval working=\"\$$#\"
>    trap cleanup INT TERM EXIT
     install "$@" 2>&1 | {
>        read err 
>        [ -z "$err" ] || barf "cannot install dir: $working"
>    } || exit $?
>    trap "" INT TERM EXIT
> }
> dir_install -o foo -d "some.dir"
> 
> # test for failure up front, with a file
> test_file="test_install-d.$$.`hostname`.`date +%s`"
> install -o "user" -g "group" "$test_file"|| rc=$?
> [ -e "$test_file" ] && rm "$test_file"
> [ 0 -ne "$rc" ] && exit "$rc"
> install -o "user" -g "group" -d "$dir" || exit $?
> 
> 
> On Dec 9, 2011, at 8:08 PM, Matthew Story wrote:
> 
>> Know this isn't a bug list or anything like that, but figured I'd throw this out there ... the only thing I found on the interweb was regarding RPMs on CentOS ... so this is apparently not even a BSDism, but it makes writing my installer scripts vastly less reliable.  Lots more below, but the gist is this:
>> 
>> install -o root -d foo/
>> 
>> Will create the directory with my user and effective group id, and my umask ... and yield an error message ... but still exit 0 (success).  At the very least I would expect the directory to linger with these permissions and exit non-zero (failure) ... this is also not consistant with the behavior of files.
>> 
>> Anyway ... anyone know a historical reason for this, or have a good hack around this ... getting patches into section 1 to trickle down into my world takes a while, assuming the behavior is even a bug ...
>> 
>> ------------------------lots of copy-paste below----------------------------------
>> 
>> [matt ~]$ mkdir install-showcase
>> [matt ~]$ cd install-showcase/
>> [matt ~/install-showcase]$ who am i
>> matt             14       Dec  9 19:52 
>> [matt ~/install-showcase]$ install -d ./test.dir; echo $? # install a directory as me, success!
>> 0
>> [matt ~/install-showcase]$ ls -d ./test.dir; echo $? # see, success!
>> ./test.dir
>> 0
>> [matt ~/install-showcase]$ install -o root -d ./test.dir.1; echo $? # install a directory as root, success?!                                                                   
>> install: chown 0:4294967295 ./test.dir.1: Operation not permitted
>> 0
>> [matt ~/install-showcase]$ ls -d ./test.dir.1 # see, success!
>> ./test.dir.1
>> [matt ~/install-showcase]$ ls -d ./test.dir.1; echo $? # see, success!                                                                                                         
>> ./test.dir.1
>> 0
>> [matt ~/install-showcase]$ install -o doesnotexist -d ./test.dir.2; echo $? # install directory as a user that doesn't exist, failure!
>> install: doesnotexist: Invalid argument
>> 67
>> [matt ~/install-showcase]$ touch filetobeinstalled; ls filetobeinstalled; echo $? # but does it work with files?                                                               
>> filetobeinstalled
>> 0
>> [matt ~/install-showcase]$ install filetobeinstalled test.file; echo $? # install a file as me, success!                                                                       
>> 0
>> [matt ~/install-showcase]$ ls filetobeinstalled; echo $?
>> filetobeinstalled
>> 0
>> [matt ~/install-showcase]$ install -o root filetobeinstalled test.file.1; echo $? # install a file as root, success?                                                           
>> install: test.file.1: chown/chgrp: Operation not permitted
>> 71
>> [matt ~/install-showcase]$ ls test.file.1; echo $? # not even a lingering file                                                                                                 
>> ls: test.file.1: No such file or directory
>> 1
>> [matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode?                                                                     
>> install: chown 0:4294967295 test.dir.2: Operation not permitted
>> 0
>> [matt ~/install-showcase]$ ls -d test.dir.2; echo $? # yup ... safe mode too ... :(
>> test.dir.2
>> 0
>> [matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode ... with an existing target
>> install: chown 0:4294967295 test.dir.2: Operation not permitted
>> 0
>> [matt ~/install-showcase]$ ls -ld test.dir.2; echo $? # yup ... safe mode too ... :(
>> drwxr-xr-x  2 matt  matt  2 Dec  9 20:00 test.dir.2
>> 0
>> [matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode ... with an existing target
>> install: chown 0:4294967295 test.dir.2: Operation not permitted
>> 0
>> _______________________________________________
>> talk mailing list
>> talk at lists.nycbug.org
>> http://lists.nycbug.org/mailman/listinfo/talk
>> 
> 
> 



From ike at blackskyresearch.net  Mon Dec 12 11:57:41 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Mon, 12 Dec 2011 11:57:41 -0500
Subject: [nycbug-talk] Measuring Disk Performance
Message-ID: <201112121658.pBCGw5A1032042@rs134.luxsci.com>

Hi All,

Disks: I need to measure disk performance across heterogeneous UNIX boxes, and I'm sick of bonnie++ for a bunch of reasons I won't' waste time on here.

I know from conversations that a few folks here have their own ways of testing disks-

I'd really like to know what people do to measure general disk performance?  e.g. really simple tests:

  - r/w/d large files
  - r/w/d small files
  - disk performance when directories contain large numbers of files

I commonly have need to test things like:
  - different block sizes
  - different inode allocations (UFS/ext3
  - different filesystem partition layouts
  - different filesystem features (think ZFS fun)

Any thoughts, experiences, urls or shell utils to share?

Rocket-
.ike




From matt at tablethotels.com  Mon Dec 12 11:58:56 2011
From: matt at tablethotels.com (Matthew Story)
Date: Mon, 12 Dec 2011 11:58:56 -0500
Subject: [nycbug-talk] anyone know why install functions this way?
In-Reply-To: <EA1F6346-C878-4260-B7E9-D56609ECFFCC@tablethotels.com>
References: <34B61D8A-8013-454A-8FF4-7E7A069FC02B@tablethotels.com>
	<4556A9E5-7168-4A4D-8EAC-D53F115ED14F@tablethotels.com>
	<EA1F6346-C878-4260-B7E9-D56609ECFFCC@tablethotels.com>
Message-ID: <7A1CAC66-43DA-4511-B278-31FE6FDD5B2D@tablethotels.com>

For anyone interested, there does actually seem to be a valid reason for this behavior, the reason is that you can `install' an existing directory with owner/group flags set.  This means that if you are installing to an existing directory, and it fails ... unlink would remove the directory and any contents ... hardly desirable.  install -d does the following:

1. byte-wise walk path, stopping at each '/' character
   +-> for each sub-path
   +----> stat sub-path
   +--------> doesn't exist (ENOENT)
   +-------------> mkdir with file mode 755 || exit error
   +--------> exists but is not a directory && exit error
   
2. if gid or uid is specified and not equal to effective uid/gid:
   +-> chown leaf directory || warn
3. chmod leaf directory || warn

Might be interesting to make this a little more robust for "new" directories, and to exit non-0 even for existing dirs, while preserving content.  Will take this up on bugs list for FreeBSD.  Thanks for playing along all.

relevant C code below (via: http://svn.freebsd.org/base/stable/8/usr.bin/xinstall/xinstall.c ... web viewable here: http://svnweb.freebsd.org/base/stable/8/usr.bin/xinstall/xinstall.c)

/*
 * install_dir --
 *      build directory hierarchy
 */
void
install_dir(char *path)
{
        char *p; 
        struct stat sb; 
        int ch; 

        for (p = path;; ++p)
                if (!*p || (p != path && *p  == '/')) {
                        ch = *p; 
                        *p = '\0';
                        if (stat(path, &sb)) {
                                if (errno != ENOENT || mkdir(path, 0755) < 0) {
                                        err(EX_OSERR, "mkdir %s", path);
                                        /* NOTREACHED */
                                } else if (verbose)
                                        (void)printf("install: mkdir %s\n",
                                                     path);
                        } else if (!S_ISDIR(sb.st_mode))
                                errx(EX_OSERR, "%s exists but is not a directory", path);
                        if (!(*p = ch))
                                break;
                }   

        if ((gid != (gid_t)-1 || uid != (uid_t)-1) && chown(path, uid, gid))
                warn("chown %u:%u %s", uid, gid, path);
        if (chmod(path, mode))
                warn("chmod %o %s", mode, path);
}

On Dec 12, 2011, at 9:55 AM, Matthew Story wrote:

> one correction below ...
> 
> On Dec 12, 2011, at 9:41 AM, Matthew Story wrote:
> 
>> [snip...]
>> A couple of work-arounds I'm using for now below (if anyone is interested) ...
>> 
>> # assume success until failure method ...
>> shout() { echo "$0: $*" >&2; }
>> barf() { shout "$*"; exit 111; }
>> cleanup() {
>>   rc="$?"
>>   [ ! -z "$working" -a -e "$working" ] && rm -r "$working"
>>   exit "$rc"
>> }
>> dir_install() {
>>   eval working=\"\$$#\"
>>   trap cleanup INT TERM EXIT
>     install "$@" 2>&1 | {
>>       read err 
>>       [ -z "$err" ] || barf "cannot install dir: $working"
>>   } || exit $?
>>   trap "" INT TERM EXIT
>> }
>> dir_install -o foo -d "some.dir"
>> 
>> # test for failure up front, with a file
>> test_file="test_install-d.$$.`hostname`.`date +%s`"
>> install -o "user" -g "group" "$test_file"|| rc=$?
>> [ -e "$test_file" ] && rm "$test_file"
>> [ 0 -ne "$rc" ] && exit "$rc"
>> install -o "user" -g "group" -d "$dir" || exit $?
>> 
>> 
>> On Dec 9, 2011, at 8:08 PM, Matthew Story wrote:
>> 
>>> Know this isn't a bug list or anything like that, but figured I'd throw this out there ... the only thing I found on the interweb was regarding RPMs on CentOS ... so this is apparently not even a BSDism, but it makes writing my installer scripts vastly less reliable.  Lots more below, but the gist is this:
>>> 
>>> install -o root -d foo/
>>> 
>>> Will create the directory with my user and effective group id, and my umask ... and yield an error message ... but still exit 0 (success).  At the very least I would expect the directory to linger with these permissions and exit non-zero (failure) ... this is also not consistant with the behavior of files.
>>> 
>>> Anyway ... anyone know a historical reason for this, or have a good hack around this ... getting patches into section 1 to trickle down into my world takes a while, assuming the behavior is even a bug ...
>>> 
>>> ------------------------lots of copy-paste below----------------------------------
>>> 
>>> [matt ~]$ mkdir install-showcase
>>> [matt ~]$ cd install-showcase/
>>> [matt ~/install-showcase]$ who am i
>>> matt             14       Dec  9 19:52 
>>> [matt ~/install-showcase]$ install -d ./test.dir; echo $? # install a directory as me, success!
>>> 0
>>> [matt ~/install-showcase]$ ls -d ./test.dir; echo $? # see, success!
>>> ./test.dir
>>> 0
>>> [matt ~/install-showcase]$ install -o root -d ./test.dir.1; echo $? # install a directory as root, success?!                                                                   
>>> install: chown 0:4294967295 ./test.dir.1: Operation not permitted
>>> 0
>>> [matt ~/install-showcase]$ ls -d ./test.dir.1 # see, success!
>>> ./test.dir.1
>>> [matt ~/install-showcase]$ ls -d ./test.dir.1; echo $? # see, success!                                                                                                         
>>> ./test.dir.1
>>> 0
>>> [matt ~/install-showcase]$ install -o doesnotexist -d ./test.dir.2; echo $? # install directory as a user that doesn't exist, failure!
>>> install: doesnotexist: Invalid argument
>>> 67
>>> [matt ~/install-showcase]$ touch filetobeinstalled; ls filetobeinstalled; echo $? # but does it work with files?                                                               
>>> filetobeinstalled
>>> 0
>>> [matt ~/install-showcase]$ install filetobeinstalled test.file; echo $? # install a file as me, success!                                                                       
>>> 0
>>> [matt ~/install-showcase]$ ls filetobeinstalled; echo $?
>>> filetobeinstalled
>>> 0
>>> [matt ~/install-showcase]$ install -o root filetobeinstalled test.file.1; echo $? # install a file as root, success?                                                           
>>> install: test.file.1: chown/chgrp: Operation not permitted
>>> 71
>>> [matt ~/install-showcase]$ ls test.file.1; echo $? # not even a lingering file                                                                                                 
>>> ls: test.file.1: No such file or directory
>>> 1
>>> [matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode?                                                                     
>>> install: chown 0:4294967295 test.dir.2: Operation not permitted
>>> 0
>>> [matt ~/install-showcase]$ ls -d test.dir.2; echo $? # yup ... safe mode too ... :(
>>> test.dir.2
>>> 0
>>> [matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode ... with an existing target
>>> install: chown 0:4294967295 test.dir.2: Operation not permitted
>>> 0
>>> [matt ~/install-showcase]$ ls -ld test.dir.2; echo $? # yup ... safe mode too ... :(
>>> drwxr-xr-x  2 matt  matt  2 Dec  9 20:00 test.dir.2
>>> 0
>>> [matt ~/install-showcase]$ install -S -o root -d test.dir.2; echo $? # how about a directory in safe mode ... with an existing target
>>> install: chown 0:4294967295 test.dir.2: Operation not permitted
>>> 0
>>> _______________________________________________
>>> talk mailing list
>>> talk at lists.nycbug.org
>>> http://lists.nycbug.org/mailman/listinfo/talk
>>> 
>> 
>> 
> 
> 



From ike at blackskyresearch.net  Mon Dec 12 12:09:30 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Mon, 12 Dec 2011 12:09:30 -0500
Subject: [nycbug-talk] anyone know why install functions this way?
In-Reply-To: <7A1CAC66-43DA-4511-B278-31FE6FDD5B2D@tablethotels.com>
References: <34B61D8A-8013-454A-8FF4-7E7A069FC02B@tablethotels.com>
	<4556A9E5-7168-4A4D-8EAC-D53F115ED14F@tablethotels.com>
	<EA1F6346-C878-4260-B7E9-D56609ECFFCC@tablethotels.com>
	<7A1CAC66-43DA-4511-B278-31FE6FDD5B2D@tablethotels.com>
Message-ID: <201112121710.pBCHA45s032716@rs134.luxsci.com>

To interrupt the chirping crickets,

On Dec 9, 2011, at 8:08 PM, Matthew Story wrote:

> install -o root -d foo/
> 
> Will create the directory with my user and effective group id, and my umask ... and yield an error message ... but still exit 0 (success).  At the very least I would expect the directory to linger with these permissions and exit non-zero (failure) ... this is also not consistant with the behavior of files.
> 
> Anyway ... anyone know a historical reason for this, or have a good hack around this ... getting patches into section 1 to trickle down into my world takes a while, assuming the behavior is even a bug ...

yadda yadda ancient 4.2BSD specific implementation of install(1) over gnu make builtin...

On Dec 12, 2011, at 11:58 AM, Matthew Story wrote:

> For anyone interested, there does actually seem to be a valid reason for this behavior, the reason is that you can `install' an existing directory with owner/group flags set.  This means that if you are installing to an existing directory, and it fails ... unlink would remove the directory and any contents ... hardly desirable.  install -d does the following:
> 
> 1. byte-wise walk path, stopping at each '/' character
>   +-> for each sub-path
>   +----> stat sub-path
>   +--------> doesn't exist (ENOENT)
>   +-------------> mkdir with file mode 755 || exit error
>   +--------> exists but is not a directory && exit error
> 
> 2. if gid or uid is specified and not equal to effective uid/gid:
>   +-> chown leaf directory || warn
> 3. chmod leaf directory || warn
> 
> Might be interesting to make this a little more robust for "new" directories, and to exit non-0 even for existing dirs, while preserving content.  Will take this up on bugs list for FreeBSD.  Thanks for playing along all.


Without inferring you go elsewhere, (I want to know what happens with this one), may I point you at the following lists:

FreeBSD bugs:
http://lists.FreeBSD.org/mailman/listinfo/freebsd-bugs

FreeBSD ports and ports-bugs:
(packagers there could be very interested...)
http://lists.FreeBSD.org/mailman/listinfo/freebsd-ports
http://lists.FreeBSD.org/mailman/listinfo/freebsd-ports-bugs

As a last resort, the RELENG team listed here may be able to engage this, (but hit the lists first!):
http://www.freebsd.org/releng/index.html

--
This however may not be a FreeBSD issue in particular?

Rocket-
.ike




From pete at nomadlogic.org  Mon Dec 12 19:31:13 2011
From: pete at nomadlogic.org (Pete Wright)
Date: Mon, 12 Dec 2011 16:31:13 -0800
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
Message-ID: <20111213003111.GA44941@arp.nomadlogic.org>

On Mon, Dec 12, 2011 at 11:57:41AM -0500, Isaac Levy wrote:
> Hi All,
> 
> Disks: I need to measure disk performance across heterogeneous UNIX boxes, and I'm sick of bonnie++ for a bunch of reasons I won't' waste time on here.
> 
> I know from conversations that a few folks here have their own ways of testing disks-
> 
> I'd really like to know what people do to measure general disk performance?  e.g. really simple tests:
> 
>   - r/w/d large files
>   - r/w/d small files
>   - disk performance when directories contain large numbers of files
> 
> I commonly have need to test things like:
>   - different block sizes
>   - different inode allocations (UFS/ext3
>   - different filesystem partition layouts
>   - different filesystem features (think ZFS fun)
> 
> Any thoughts, experiences, urls or shell utils to share?
> 

hey ike!
there is actually a pretty decent chapter on measuring disk and
filesystem performance in "High Performance PostgreSQL 9.0".  they talk
about using a tool, bundled with bonie++, called zcav that will track
transfer rates from the begining to end of a disk subsystem.  it also
will output data into gnuplot friendly format for pcitures.  i used this
quite extensivly while tuning a linux dataware house a while back.

other tools that I'm happy with are iozone and fio:

http://www.iozone.org/
http://freecode.com/projects/fio

i find that when doing benchmarking of systems for eval purposes or
benchmarking i end up using a mixture of many different tools.  i find
that differnent tools will stress different parts of a given i/o
subsystem.  so i'd generally do something like:

- initial test using dd with variable blocksizes (dependent upon
underlying filesystem block size)
- several bonie++ tests, followed by some tests using iozone and fio
- depending on how system will be used in production i try some
  application level tests.  for a db - pgbench, webserver apache bench
  etc..

I have also done some interesting testing using this package written in
erlang that does a good job in generating load on a wide range of
appliation called Tsung:
http://tsung.erlang-projects.org/

Hope This Helps!
-pete

-- 
Pete Wright
pete at nomadlogic.org



From jhb at freebsd.org  Tue Dec 13 11:58:46 2011
From: jhb at freebsd.org (John Baldwin)
Date: Tue, 13 Dec 2011 11:58:46 -0500
Subject: [nycbug-talk] anyone know why install functions this way?
In-Reply-To: <201112121710.pBCHA45s032716@rs134.luxsci.com>
References: <34B61D8A-8013-454A-8FF4-7E7A069FC02B@tablethotels.com>
	<7A1CAC66-43DA-4511-B278-31FE6FDD5B2D@tablethotels.com>
	<201112121710.pBCHA45s032716@rs134.luxsci.com>
Message-ID: <201112131158.46705.jhb@freebsd.org>

On Monday, December 12, 2011 12:09:30 pm Isaac Levy wrote:
> To interrupt the chirping crickets,
> 
> On Dec 9, 2011, at 8:08 PM, Matthew Story wrote:
> 
> > install -o root -d foo/
> > 
> > Will create the directory with my user and effective group id, and my 
umask ... and yield an error message ... but still exit 0 (success).  At the 
very least I would expect the directory to linger with these permissions and 
exit non-zero (failure) ... this is also not consistant with the behavior of 
files.
> > 
> > Anyway ... anyone know a historical reason for this, or have a good hack 
around this ... getting patches into section 1 to trickle down into my world 
takes a while, assuming the behavior is even a bug ...
> 
> yadda yadda ancient 4.2BSD specific implementation of install(1) over gnu 
make builtin...

Actually, the install -d bits were added to FreeBSD (at least) in 1996, they
didn't come from 4.2BSD.  (Have to do some svn annotate dumpster diving.)  
Looks like chown/chmod failures were non-fatal from the start.  Hmm, in 
NetBSD's history install -d briefly error'd if the chown/chmod failed, but
that was changed to a warning when 4.4BSD lite was imported, so presumably
install -d was in 4.4BSD at least.

-- 
John Baldwin


From matt at tablethotels.com  Wed Dec 14 17:28:42 2011
From: matt at tablethotels.com (Matthew Story)
Date: Wed, 14 Dec 2011 17:28:42 -0500
Subject: [nycbug-talk] nifty(1)
Message-ID: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>

hoping to start a long-living thread filled with the community's (1)-fu, and discussion of the (1)-fu-ness.  if this actually takes off, Henry <henry at tablethotels.com> has volunteered to compile this into a nifty port, filled only with nifty man pages full of fun hacks.

so what nifty shit do you do with "General Commands"?  What are your favorite sh functions/patterns?  How do you leverage the latent power of bc?  When do you bust out awk?  And what are the most useful sed, cut invocations you've found?

I'll kick it off with a sh/find/xargs pattern I'm quite fond of, which i call the `bax mv':

# move the contents of dir ($1) to target dir ($2)
bax_mv() { 
  find "$1" -depth 1 -maxdepth 1 -print0 |
    xargs -0 sh -c '
      target="$1"; shift
      mv "$@" "$target"
    ' baxter_mv "$2"
}
# glob expansion is unreliable for scripting (too many args)
# and mv's interface is ill-suited to use with xargs ... enter bax_mv
baxter_mv dir1 dir2

who's up next?

From matt at tablethotels.com  Wed Dec 14 17:35:20 2011
From: matt at tablethotels.com (Matthew Story)
Date: Wed, 14 Dec 2011 17:35:20 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
Message-ID: <DEFF0C12-0F98-42C8-A11E-DE9F25A11305@tablethotels.com>

... oh boy ... way for me to start things with a bug ... so like me, I always do something stupid like misplacing a decimal point ... fix inline.

On Dec 14, 2011, at 5:28 PM, Matthew Story wrote:

> hoping to start a long-living thread filled with the community's (1)-fu, and discussion of the (1)-fu-ness.  if this actually takes off, Henry <henry at tablethotels.com> has volunteered to compile this into a nifty port, filled only with nifty man pages full of fun hacks.
> 
> so what nifty shit do you do with "General Commands"?  What are your favorite sh functions/patterns?  How do you leverage the latent power of bc?  When do you bust out awk?  And what are the most useful sed, cut invocations you've found?
> 
> I'll kick it off with a sh/find/xargs pattern I'm quite fond of, which i call the `bax mv':
> 
> # move the contents of dir ($1) to target dir ($2)
> bax_mv() { 
>  find "$1" -depth 1 -maxdepth 1 -print0 |
>    xargs -0 sh -c '
>      target="$1"; shift
>      mv "$@" "$target"
    ' bax_mv "$2"
> }
> # glob expansion is unreliable for scripting (too many args)
> # and mv's interface is ill-suited to use with xargs ... enter bax_mv
bax_mv dir1 dir2
> 
> who's up next?
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
> 



From ike at blackskyresearch.net  Wed Dec 14 18:13:11 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Wed, 14 Dec 2011 18:13:11 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
Message-ID: <201112142314.pBENE3X8002735@rs134.luxsci.com>

On Dec 14, 2011, at 5:28 PM, Matthew Story wrote:
> who's up next?
This nifty(1) thread is a great idea!

"The 3-finger Claw Technique"  The sweetest 3 functions, ever.

  shout() { echo "$0: $*" >&2; }
  barf() { shout "$*"; exit 111; }
  safe() { "$@" || barf "cannot $*"; }

I use these daily, they're extremely cross-platform friendly in any bourne-derived shell I've  used in the last 3 years.


safexample.sh :
--
#!/bin/sh

shout() { echo "$0: $*" >&2; }
barf() { shout "$*"; exit 111; }
safe() { "$@" || barf "cannot $*"; }

# consider the following lines
safe cd /some/dir
safe tar xzvfp /my/big/tarball.tbz

exit 0
--


In the above example, using 'safe' suddenly gives the user the following special powers:

1) if '/some/dir' does not exist, the script will safely exit before that un-tarring does any damage.

2) the actual 'directory does not exist' will return to stderr for the script, (as opposed to just telling us the useless fact that the enclosing script failed).
Now bourne shell starts behaving like a modern language!  (ala Python/Ruby tracebacks, Perl errors, etc?)

Additonally, if you 'exit 0' at the end, you can run non-safe operations, and always be guaranteed that if the shell exits, it was 'complete'.
An example:

# consider the following lines
safe cd /some/dir
tar xzvfp /my/big/tarball.tbz

Now, safe was removed from the un-tar command, right?  So, imaging tarballs created by people using a Mac (with HFS+ and some filesystem-specific sticky bit somewhere in the filesystem which was tar'd up).
Now, you un-tar it on some *BSD or other *NIX box, and tar complains as it goes- and exits non-zero.  One way to handle this, is to consider the tar 'reliable', and not use safe for it.

Now, if we continue to be conscious of this simple safe/notsafe distinction with these scripts, they can be called by other safe scripts- and behave just like any respectable UNIX program!

Now, consider this crontab entry:
1	3	*	*	*	someuser flock -k /tmp/safexample.lock /path/to/safexample.sh

If it fails, (for bad perms, no tarball to unpack, etc?)  cron will actually have a reasonable message to email/log on failure!


Thanks to the noble efforts of the Clan of the White Lotus, these 3 functions originate during the late thirteenth century, around Guangzhou.  It is rumored these 3 functions took 15 years to boil down- I beileve that after mercilessly abusing them for the last 3 years myself?  (I'm not kididng about that part!)

Attached, is another example with more bits.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: nonsense.sh
Type: application/octet-stream
Size: 338 bytes
Desc: not available
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111214/75e8010b/attachment.obj>

From henry95 at gmail.com  Wed Dec 14 18:21:26 2011
From: henry95 at gmail.com (Henry M)
Date: Wed, 14 Dec 2011 18:21:26 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <201112142314.pBENE3X8002735@rs134.luxsci.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
Message-ID: <CAK1r8CV06CW71YAv7brFV6nh_jOJjo8PZo6nOxvueCxGDsBzFg@mail.gmail.com>

Wouldn't it be nice to just 'man nifty' and a have a large collection of
tricks?

One nifty trick with curl and awk:

Curl a website and print it's status code only if it's the code you are
looking for.

Example 1: (This will print 200.)
curl -sL -w "\n%{http_code}\n" "nycbug.org/" |
awk 'BEGIN { getline line } { print line; line=$0 } END { if (line ~/200/)
{print line;}}'


Example 2: (This will not print 404)
curl -sL -w "\n%{http_code}\n" "nycbug.org/" |
awk 'BEGIN { getline line } { print line; line=$0 } END { if (line ~/404/)
{print line;}}'

Useless? I say not!

~Henry

On Wed, Dec 14, 2011 at 6:13 PM, Isaac Levy <ike at blackskyresearch.net>wrote:

> On Dec 14, 2011, at 5:28 PM, Matthew Story wrote:
> > who's up next?
> This nifty(1) thread is a great idea!
>
> "The 3-finger Claw Technique"  The sweetest 3 functions, ever.
>
>  shout() { echo "$0: $*" >&2; }
>  barf() { shout "$*"; exit 111; }
>  safe() { "$@" || barf "cannot $*"; }
>
> I use these daily, they're extremely cross-platform friendly in any
> bourne-derived shell I've  used in the last 3 years.
>
>
> safexample.sh :
> --
> #!/bin/sh
>
> shout() { echo "$0: $*" >&2; }
> barf() { shout "$*"; exit 111; }
> safe() { "$@" || barf "cannot $*"; }
>
> # consider the following lines
> safe cd /some/dir
> safe tar xzvfp /my/big/tarball.tbz
>
> exit 0
> --
>
>
> In the above example, using 'safe' suddenly gives the user the following
> special powers:
>
> 1) if '/some/dir' does not exist, the script will safely exit before that
> un-tarring does any damage.
>
> 2) the actual 'directory does not exist' will return to stderr for the
> script, (as opposed to just telling us the useless fact that the enclosing
> script failed).
> Now bourne shell starts behaving like a modern language!  (ala Python/Ruby
> tracebacks, Perl errors, etc?)
>
> Additonally, if you 'exit 0' at the end, you can run non-safe operations,
> and always be guaranteed that if the shell exits, it was 'complete'.
> An example:
>
> # consider the following lines
> safe cd /some/dir
> tar xzvfp /my/big/tarball.tbz
>
> Now, safe was removed from the un-tar command, right?  So, imaging
> tarballs created by people using a Mac (with HFS+ and some
> filesystem-specific sticky bit somewhere in the filesystem which was tar'd
> up).
> Now, you un-tar it on some *BSD or other *NIX box, and tar complains as it
> goes- and exits non-zero.  One way to handle this, is to consider the tar
> 'reliable', and not use safe for it.
>
> Now, if we continue to be conscious of this simple safe/notsafe
> distinction with these scripts, they can be called by other safe scripts-
> and behave just like any respectable UNIX program!
>
> Now, consider this crontab entry:
> 1       3       *       *       *       someuser flock -k
> /tmp/safexample.lock /path/to/safexample.sh
>
> If it fails, (for bad perms, no tarball to unpack, etc?)  cron will
> actually have a reasonable message to email/log on failure!
>
>
> Thanks to the noble efforts of the Clan of the White Lotus, these 3
> functions originate during the late thirteenth century, around Guangzhou.
>  It is rumored these 3 functions took 15 years to boil down- I beileve that
> after mercilessly abusing them for the last 3 years myself?  (I'm not
> kididng about that part!)
>
> Attached, is another example with more bits.
>
>
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111214/00fb3e26/attachment.htm>

From matt at tablethotels.com  Wed Dec 14 20:19:07 2011
From: matt at tablethotels.com (Matthew Story)
Date: Wed, 14 Dec 2011 20:19:07 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <CAK1r8CV06CW71YAv7brFV6nh_jOJjo8PZo6nOxvueCxGDsBzFg@mail.gmail.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<CAK1r8CV06CW71YAv7brFV6nh_jOJjo8PZo6nOxvueCxGDsBzFg@mail.gmail.com>
Message-ID: <4F85E9C3-ADAC-496D-AF36-8F1785DD4335@tablethotels.com>

i believe that this is useful ... but i think this is yet more useful:

curl_ok() {
    curl -sL -w "\n%{http_code}\n" "$1" |
    awk 'BEGIN { 
        getline line 
    } { 
        print line; line=$0 
    } END { 
        if (line ~ /200/) { 
            print "curl: fatal: bad status code:", line | "cat - >&2"
            exit 111
        }
        exit 0
    }'
}
# now it exits non-0 for bad status
curl_ok 'nycbug.org' || exit $?

in either case, useful hack

On Dec 14, 2011, at 6:21 PM, Henry M wrote:

> Wouldn't it be nice to just 'man nifty' and a have a large collection of tricks? 
> 
> One nifty trick with curl and awk:
> 
> Curl a website and print it's status code only if it's the code you are looking for.
> 
> Example 1: (This will print 200.) 
> curl -sL -w "\n%{http_code}\n" "nycbug.org/" |
> awk 'BEGIN { getline line } { print line; line=$0 } END { if (line ~/200/) {print line;}}'
> 
> 
> Example 2: (This will not print 404)
> curl -sL -w "\n%{http_code}\n" "nycbug.org/" |
> awk 'BEGIN { getline line } { print line; line=$0 } END { if (line ~/404/) {print line;}}'
> 
> Useless? I say not! 
> 
> ~Henry
> 
> On Wed, Dec 14, 2011 at 6:13 PM, Isaac Levy <ike at blackskyresearch.net> wrote:
> On Dec 14, 2011, at 5:28 PM, Matthew Story wrote:
> > who's up next?
> This nifty(1) thread is a great idea!
> 
> "The 3-finger Claw Technique"  The sweetest 3 functions, ever.
> 
>  shout() { echo "$0: $*" >&2; }
>  barf() { shout "$*"; exit 111; }
>  safe() { "$@" || barf "cannot $*"; }
> 
> I use these daily, they're extremely cross-platform friendly in any bourne-derived shell I've  used in the last 3 years.
> 
> 
> safexample.sh :
> --
> #!/bin/sh
> 
> shout() { echo "$0: $*" >&2; }
> barf() { shout "$*"; exit 111; }
> safe() { "$@" || barf "cannot $*"; }
> 
> # consider the following lines
> safe cd /some/dir
> safe tar xzvfp /my/big/tarball.tbz
> 
> exit 0
> --
> 
> 
> In the above example, using 'safe' suddenly gives the user the following special powers:
> 
> 1) if '/some/dir' does not exist, the script will safely exit before that un-tarring does any damage.
> 
> 2) the actual 'directory does not exist' will return to stderr for the script, (as opposed to just telling us the useless fact that the enclosing script failed).
> Now bourne shell starts behaving like a modern language!  (ala Python/Ruby tracebacks, Perl errors, etc?)
> 
> Additonally, if you 'exit 0' at the end, you can run non-safe operations, and always be guaranteed that if the shell exits, it was 'complete'.
> An example:
> 
> # consider the following lines
> safe cd /some/dir
> tar xzvfp /my/big/tarball.tbz
> 
> Now, safe was removed from the un-tar command, right?  So, imaging tarballs created by people using a Mac (with HFS+ and some filesystem-specific sticky bit somewhere in the filesystem which was tar'd up).
> Now, you un-tar it on some *BSD or other *NIX box, and tar complains as it goes- and exits non-zero.  One way to handle this, is to consider the tar 'reliable', and not use safe for it.
> 
> Now, if we continue to be conscious of this simple safe/notsafe distinction with these scripts, they can be called by other safe scripts- and behave just like any respectable UNIX program!
> 
> Now, consider this crontab entry:
> 1       3       *       *       *       someuser flock -k /tmp/safexample.lock /path/to/safexample.sh
> 
> If it fails, (for bad perms, no tarball to unpack, etc?)  cron will actually have a reasonable message to email/log on failure!
> 
> 
> Thanks to the noble efforts of the Clan of the White Lotus, these 3 functions originate during the late thirteenth century, around Guangzhou.  It is rumored these 3 functions took 15 years to boil down- I beileve that after mercilessly abusing them for the last 3 years myself?  (I'm not kididng about that part!)
> 
> Attached, is another example with more bits.
> 
> 
> 
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
> 
> 
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111214/b24f6729/attachment.htm>

From ike at blackskyresearch.net  Wed Dec 14 20:20:23 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Wed, 14 Dec 2011 20:20:23 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <201112142314.pBENE3X8002735@rs134.luxsci.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
Message-ID: <201112150121.pBF1L3Qu003071@rs134.luxsci.com>

On Dec 14, 2011, at 6:30 PM, Someone wrote:

> attribution of the original author if the "3-Finger Claw Technique"?

On Dec 14, 2011, at 6:13 PM, Isaac Levy wrote:

>  shout() { echo "$0: $*" >&2; }
>  barf() { shout "$*"; exit 111; }
>  safe() { "$@" || barf "cannot $*"; }

My bad- William Baxter <www.superscript.com> showed me the 3-Finger Claw Technique, and stated that the functions took 15 years to boil down.

Rocket-
.ike




From henry95 at gmail.com  Thu Dec 15 00:54:12 2011
From: henry95 at gmail.com (Henry M)
Date: Thu, 15 Dec 2011 00:54:12 -0500
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <20111213003111.GA44941@arp.nomadlogic.org>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
Message-ID: <CAK1r8CUHOVpZzSGj=D_OrqM28GvKy+Kc7Bmxg-+AbeMyaYsbvQ@mail.gmail.com>

Whenever I've bench-marked disks, I've always just used dd and /dev/zero

 Example: I want to see how fast I can write a 1GB file

$ dd if=/dev/zero of=1GB bs=1024 count=1048576
1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB) copied, 8.72947 s, 123 MB/s

As long as the system load is consistent, you should get consistent
results. You can have fun by running multiple versions at once, to simulate
heavier "real-world" load.

You can change the byte size, or count accordingly. Just be careful what
values you give dd, you can easily fill up your disk, or break something
nasty with a typo (Yes I've done both )

-Henry

On Mon, Dec 12, 2011 at 7:31 PM, Pete Wright <pete at nomadlogic.org> wrote:

> On Mon, Dec 12, 2011 at 11:57:41AM -0500, Isaac Levy wrote:
> > Hi All,
> >
> > Disks: I need to measure disk performance across heterogeneous UNIX
> boxes, and I'm sick of bonnie++ for a bunch of reasons I won't' waste time
> on here.
> >
> > I know from conversations that a few folks here have their own ways of
> testing disks-
> >
> > I'd really like to know what people do to measure general disk
> performance?  e.g. really simple tests:
> >
> >   - r/w/d large files
> >   - r/w/d small files
> >   - disk performance when directories contain large numbers of files
> >
> > I commonly have need to test things like:
> >   - different block sizes
> >   - different inode allocations (UFS/ext3
> >   - different filesystem partition layouts
> >   - different filesystem features (think ZFS fun)
> >
> > Any thoughts, experiences, urls or shell utils to share?
> >
>
> hey ike!
> there is actually a pretty decent chapter on measuring disk and
> filesystem performance in "High Performance PostgreSQL 9.0".  they talk
> about using a tool, bundled with bonie++, called zcav that will track
> transfer rates from the begining to end of a disk subsystem.  it also
> will output data into gnuplot friendly format for pcitures.  i used this
> quite extensivly while tuning a linux dataware house a while back.
>
> other tools that I'm happy with are iozone and fio:
>
> http://www.iozone.org/
> http://freecode.com/projects/fio
>
> i find that when doing benchmarking of systems for eval purposes or
> benchmarking i end up using a mixture of many different tools.  i find
> that differnent tools will stress different parts of a given i/o
> subsystem.  so i'd generally do something like:
>
> - initial test using dd with variable blocksizes (dependent upon
> underlying filesystem block size)
> - several bonie++ tests, followed by some tests using iozone and fio
> - depending on how system will be used in production i try some
>  application level tests.  for a db - pgbench, webserver apache bench
>  etc..
>
> I have also done some interesting testing using this package written in
> erlang that does a good job in generating load on a wide range of
> appliation called Tsung:
> http://tsung.erlang-projects.org/
>
> Hope This Helps!
> -pete
>
> --
> Pete Wright
> pete at nomadlogic.org
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111215/c420bf6b/attachment.htm>

From matt at tablethotels.com  Thu Dec 15 00:54:24 2011
From: matt at tablethotels.com (Matthew Story)
Date: Thu, 15 Dec 2011 00:54:24 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <201112142314.pBENE3X8002735@rs134.luxsci.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
Message-ID: <7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>

i believe the most portable version is slightly uglier:

On Dec 14, 2011, at 6:13 PM, Isaac Levy wrote:

>  shout() { echo "$0: $*" >&2; }
>  barf() { shout "$*"; exit 111; }
safe() { ${1+"$@"} || barf "cannot $*"; }
          ^^^^^^^^ this bit is more reliable for sh on Solaris (real Bourne shell)

you can install the shells/heirloom-sh package from ports (it installs as `jsh') for `compatibility' testing, I believe Bourne shell is typically considered the most
restrictive of the POSIX shells (although i believe it is not technically compliant with IEEE Std 1003.2 (``POSIX.2'') ... neither is FreeBSD shell, although it is closer).

> 
> I use these daily, they're extremely cross-platform friendly in any bourne-derived shell I've  used in the last 3 years.
> 



From lists at eitanadler.com  Thu Dec 15 01:00:08 2011
From: lists at eitanadler.com (Eitan Adler)
Date: Thu, 15 Dec 2011 01:00:08 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
Message-ID: <CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>

This is a fun one:

% function try_this_in_zsh {
 local lines columns colour a b p q i pnew
 ((columns=COLUMNS-1, lines=LINES-1, colour=0))
 for ((b=-1.5; b<=1.5; b+=3.0/lines)) do
 for ((a=-2.0; a<=1; a+=3.0/columns)) do
 for ((p=0.0, q=0.0, i=0; p*p+q*q < 4 && i < 32; i++)) do
 ((pnew=p*p-q*q+a, q=2*p*q+b, p=pnew))
 done
 ((colour=(i/4)%8))
 echo -n "\\e[4${colour}m "
 done
 echo
 done
 }
-- 
Eitan Adler


From ike at blackskyresearch.net  Thu Dec 15 09:13:55 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Thu, 15 Dec 2011 09:13:55 -0500
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <20111213003111.GA44941@arp.nomadlogic.org>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
Message-ID: <201112151414.pBFEE4jg016317@rs134.luxsci.com>

On Dec 12, 2011, at 7:31 PM, Pete Wright wrote:

> On Mon, Dec 12, 2011 at 11:57:41AM -0500, Isaac Levy wrote:
>> Hi All,
>> 
>> Disks: I need to measure disk performance across heterogeneous UNIX boxes, and I'm sick of bonnie++ for a bunch of reasons I won't' waste time on here.
>> 
>> I know from conversations that a few folks here have their own ways of testing disks-
>> 
>> I'd really like to know what people do to measure general disk performance?  e.g. really simple tests:
>> 
>>  - r/w/d large files
>>  - r/w/d small files
>>  - disk performance when directories contain large numbers of files
>> 
>> I commonly have need to test things like:
>>  - different block sizes
>>  - different inode allocations (UFS/ext3
>>  - different filesystem partition layouts
>>  - different filesystem features (think ZFS fun)
>> 
>> Any thoughts, experiences, urls or shell utils to share?
>> 
> 
> hey ike!
> there is actually a pretty decent chapter on measuring disk and
> filesystem performance in "High Performance PostgreSQL 9.0".  

Is it this book:
"PostgreSQL 9.0 High Performance"
Author: Gregory Smith
Publisher: Packt Publishing

> they talk
> about using a tool, bundled with bonie++, called zcav that will track
> transfer rates from the begining to end of a disk subsystem.  it also
> will output data into gnuplot friendly format for pcitures.  i used this
> quite extensivly while tuning a linux dataware house a while back.
> 
> other tools that I'm happy with are iozone and fio:
> 
> http://www.iozone.org/
> http://freecode.com/projects/fio
> 
> i find that when doing benchmarking of systems for eval purposes or
> benchmarking i end up using a mixture of many different tools.  i find
> that differnent tools will stress different parts of a given i/o
> subsystem.  so i'd generally do something like:
> 
> - initial test using dd with variable blocksizes (dependent upon
> underlying filesystem block size)
> - several bonie++ tests, followed by some tests using iozone and fio
> - depending on how system will be used in production i try some
>  application level tests.  for a db - pgbench, webserver apache bench
>  etc..

Understood, really comprehensive, and valuable!

Have you posted your methodology and/or resulting tests anywhere?

--
Conversely, what I'm looking to really do here, is more of a cave-man sanity test: test a few small common cases in a relative vaccum, ideally using UNIX builtins spitting a few integers back to stdout.  (mix a bit of xargs with rm, mkdir, dd, etc?)

I'm actually looking to avoid going down this path alltogether, as I'd rather focus this level of detail on measuring disk/system performance as the team hammers our apps.

> 
> I have also done some interesting testing using
..
> http://tsung.erlang-projects.org/

*That* looks rad- way off direct disk testing, but that's a very compelling toolkit...

Thanks Pete!

Rocket-
.ike




From matt at tablethotels.com  Thu Dec 15 09:17:12 2011
From: matt at tablethotels.com (Matthew Story)
Date: Thu, 15 Dec 2011 09:17:12 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
	<CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>
Message-ID: <6AE1EC8D-7076-4C2A-949F-3B938482358A@tablethotels.com>

... you just made my morning sir ... that is awesome ... 

On Dec 15, 2011, at 1:00 AM, Eitan Adler wrote:

> This is a fun one:
> 
> % function try_this_in_zsh {
> local lines columns colour a b p q i pnew
> ((columns=COLUMNS-1, lines=LINES-1, colour=0))
> for ((b=-1.5; b<=1.5; b+=3.0/lines)) do
> for ((a=-2.0; a<=1; a+=3.0/columns)) do
> for ((p=0.0, q=0.0, i=0; p*p+q*q < 4 && i < 32; i++)) do
> ((pnew=p*p-q*q+a, q=2*p*q+b, p=pnew))
> done
> ((colour=(i/4)%8))
> echo -n "\\e[4${colour}m "
> done
> echo
> done
> }
> -- 
> Eitan Adler
> 



From ike at blackskyresearch.net  Thu Dec 15 09:18:38 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Thu, 15 Dec 2011 09:18:38 -0500
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <CAK1r8CUHOVpZzSGj=D_OrqM28GvKy+Kc7Bmxg-+AbeMyaYsbvQ@mail.gmail.com>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
	<CAK1r8CUHOVpZzSGj=D_OrqM28GvKy+Kc7Bmxg-+AbeMyaYsbvQ@mail.gmail.com>
Message-ID: <201112151419.pBFEJ3a2025228@rs134.luxsci.com>

On Dec 15, 2011, at 12:54 AM, Henry M wrote:

> Whenever I've bench-marked disks, I've always just used dd and /dev/zero 
> 
>  Example: I want to see how fast I can write a 1GB file
> 
> $ dd if=/dev/zero of=1GB bs=1024 count=1048576
> 1048576+0 records in
> 1048576+0 records out
> 1073741824 bytes (1.1 GB) copied, 8.72947 s, 123 MB/s
> 
> As long as the system load is consistent, you should get consistent results. You can have fun by running multiple versions at once, to simulate heavier "real-world" load. 

Oh- &, a little xargs, and some date(1) and time(1) fun.

> 
> You can change the byte size, or count accordingly. Just be careful what values you give dd, you can easily fill up your disk, or break something nasty with a typo (Yes I've done both )
> 
> -Henry

Yeaaahhhh- this is exactly what I was thinking, but I didn't think to just dd from /dev/zero and build some small tests.

Sweet.

Rocket-
.ike


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111215/40619d23/attachment.htm>

From ike at blackskyresearch.net  Thu Dec 15 09:21:47 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Thu, 15 Dec 2011 09:21:47 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <6AE1EC8D-7076-4C2A-949F-3B938482358A@tablethotels.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
	<CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>
	<6AE1EC8D-7076-4C2A-949F-3B938482358A@tablethotels.com>
Message-ID: <201112151422.pBFEM3qJ031591@rs134.luxsci.com>

> On Dec 15, 2011, at 1:00 AM, Eitan Adler wrote:
> 
>> This is a fun one:
>> 
>> % function try_this_in_zsh {
>> local lines columns colour a b p q i pnew
>> ((columns=COLUMNS-1, lines=LINES-1, colour=0))
>> for ((b=-1.5; b<=1.5; b+=3.0/lines)) do
>> for ((a=-2.0; a<=1; a+=3.0/columns)) do
>> for ((p=0.0, q=0.0, i=0; p*p+q*q < 4 && i < 32; i++)) do
>> ((pnew=p*p-q*q+a, q=2*p*q+b, p=pnew))
>> done
>> ((colour=(i/4)%8))
>> echo -n "\\e[4${colour}m "
>> done
>> echo
>> done
>> }
>> -- 
>> Eitan Adler
>> 
> 

On Dec 15, 2011, at 9:17 AM, Matthew Story wrote:

> ... you just made my morning sir ... that is awesome ? 

I just spit coffee all over my desk- that's RAD.

Rocket-
.ike




From bonsaime at gmail.com  Thu Dec 15 10:12:33 2011
From: bonsaime at gmail.com (Jesse Callaway)
Date: Thu, 15 Dec 2011 10:12:33 -0500
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <201112151419.pBFEJ3a2025228@rs134.luxsci.com>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
	<CAK1r8CUHOVpZzSGj=D_OrqM28GvKy+Kc7Bmxg-+AbeMyaYsbvQ@mail.gmail.com>
	<201112151419.pBFEJ3a2025228@rs134.luxsci.com>
Message-ID: <CAJU5ZBPPzf5P8rRpHiYw7Nu90P-bh4f-T8T9Pv8AW0Br6CQscw@mail.gmail.com>

the advantage to some of the heavier tools is that they attempt to break
caching mechanisms so that you can see worst case performance.
dd looks cool since it certainly skips any possibility of filesystem
speedups.
I'd just say to make sure 1G (or whatever you test with) is large enough to
break controller and on-disk cache.
On Dec 15, 2011 9:21 AM, "Isaac Levy" <ike at blackskyresearch.net> wrote:

> On Dec 15, 2011, at 12:54 AM, Henry M wrote:
>
> Whenever I've bench-marked disks, I've always just used dd and /dev/zero
>
>  Example: I want to see how fast I can write a 1GB file
>
> $ dd if=/dev/zero of=1GB bs=1024 count=1048576
> 1048576+0 records in
> 1048576+0 records out
> 1073741824 bytes (1.1 GB) copied, 8.72947 s, 123 MB/s
>
> As long as the system load is consistent, you should get consistent
> results. You can have fun by running multiple versions at once, to simulate
> heavier "real-world" load.
>
>
> Oh- &, a little xargs, and some date(1) and time(1) fun.
>
>
> You can change the byte size, or count accordingly. Just be careful what
> values you give dd, you can easily fill up your disk, or break something
> nasty with a typo (Yes I've done both )
>
> -Henry
>
>
> Yeaaahhhh- this is exactly what I was thinking, but I didn't think to just
> dd from /dev/zero and build some small tests.
>
> Sweet.
>
> Rocket-
> .ike
>
>
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111215/b06333c1/attachment.htm>

From george at ceetonetechnology.com  Thu Dec 15 10:22:52 2011
From: george at ceetonetechnology.com (George Rosamond)
Date: Thu, 15 Dec 2011 10:22:52 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <201112151422.pBFEM3qJ031591@rs134.luxsci.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
	<CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>
	<6AE1EC8D-7076-4C2A-949F-3B938482358A@tablethotels.com>
	<201112151422.pBFEM3qJ031591@rs134.luxsci.com>
Message-ID: <4EEA10CC.1050400@ceetonetechnology.com>

On 12/15/11 09:21, Isaac Levy wrote:
>> On Dec 15, 2011, at 1:00 AM, Eitan Adler wrote:
>>
>>> This is a fun one:
>>>
>>> % function try_this_in_zsh {
>>> local lines columns colour a b p q i pnew
>>> ((columns=COLUMNS-1, lines=LINES-1, colour=0))
>>> for ((b=-1.5; b<=1.5; b+=3.0/lines)) do
>>> for ((a=-2.0; a<=1; a+=3.0/columns)) do
>>> for ((p=0.0, q=0.0, i=0; p*p+q*q<  4&&  i<  32; i++)) do
>>> ((pnew=p*p-q*q+a, q=2*p*q+b, p=pnew))
>>> done
>>> ((colour=(i/4)%8))
>>> echo -n "\\e[4${colour}m "
>>> done
>>> echo
>>> done
>>> }
>>> --
>>> Eitan Adler
>>>
>>
>
> On Dec 15, 2011, at 9:17 AM, Matthew Story wrote:
>
>> ... you just made my morning sir ... that is awesome ?
>
> I just spit coffee all over my desk- that's RAD.
>
> Rocket-
> .ike

I'm loosely following this thread. . . a bit swamped right now.

Not to divert it, but. . .

Is this a "Shell from Hell" meeting?

g



From edlinuxguru at gmail.com  Thu Dec 15 10:50:52 2011
From: edlinuxguru at gmail.com (Edward Capriolo)
Date: Thu, 15 Dec 2011 10:50:52 -0500
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <CAJU5ZBPPzf5P8rRpHiYw7Nu90P-bh4f-T8T9Pv8AW0Br6CQscw@mail.gmail.com>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
	<CAK1r8CUHOVpZzSGj=D_OrqM28GvKy+Kc7Bmxg-+AbeMyaYsbvQ@mail.gmail.com>
	<201112151419.pBFEJ3a2025228@rs134.luxsci.com>
	<CAJU5ZBPPzf5P8rRpHiYw7Nu90P-bh4f-T8T9Pv8AW0Br6CQscw@mail.gmail.com>
Message-ID: <CAENxBwwm1VeJtqd8-ctX0s6eHjzdjGqtge9jOajexhMbUDU0iw@mail.gmail.com>

Unless you are using a very specific application DD does not prove much. DD
is doing linear writes which are very fast even with low RPM sata. If you
are benchmarking a disk you are likely doing it for a database. Database
workload are usually more seeking then streaming. That was one of the
reasons why I mentioned YCSB even though it is not a traditional disk tool.
You can create a solid amount of random data and read it back with
distributions like zipfan, latest, or random. This gets your disks seeking
and shows how they perform under some load. Another way to go is use the
TPH-C relational database type benchmarks, because they represent a real
world type workload. But iozone makes the nicest graphs for sure :)

Edward

On Thu, Dec 15, 2011 at 10:12 AM, Jesse Callaway <bonsaime at gmail.com> wrote:

> the advantage to some of the heavier tools is that they attempt to break
> caching mechanisms so that you can see worst case performance.
> dd looks cool since it certainly skips any possibility of filesystem
> speedups.
> I'd just say to make sure 1G (or whatever you test with) is large enough
> to break controller and on-disk cache.
> On Dec 15, 2011 9:21 AM, "Isaac Levy" <ike at blackskyresearch.net> wrote:
>
>> On Dec 15, 2011, at 12:54 AM, Henry M wrote:
>>
>> Whenever I've bench-marked disks, I've always just used dd and /dev/zero
>>
>>  Example: I want to see how fast I can write a 1GB file
>>
>> $ dd if=/dev/zero of=1GB bs=1024 count=1048576
>> 1048576+0 records in
>> 1048576+0 records out
>> 1073741824 bytes (1.1 GB) copied, 8.72947 s, 123 MB/s
>>
>> As long as the system load is consistent, you should get consistent
>> results. You can have fun by running multiple versions at once, to simulate
>> heavier "real-world" load.
>>
>>
>> Oh- &, a little xargs, and some date(1) and time(1) fun.
>>
>>
>> You can change the byte size, or count accordingly. Just be careful what
>> values you give dd, you can easily fill up your disk, or break something
>> nasty with a typo (Yes I've done both )
>>
>> -Henry
>>
>>
>> Yeaaahhhh- this is exactly what I was thinking, but I didn't think to
>> just dd from /dev/zero and build some small tests.
>>
>> Sweet.
>>
>> Rocket-
>> .ike
>>
>>
>>
>> _______________________________________________
>> talk mailing list
>> talk at lists.nycbug.org
>> http://lists.nycbug.org/mailman/listinfo/talk
>>
>>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111215/301f7f4a/attachment.htm>

From matt at tablethotels.com  Thu Dec 15 11:12:43 2011
From: matt at tablethotels.com (Matthew Story)
Date: Thu, 15 Dec 2011 11:12:43 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <4EEA10CC.1050400@ceetonetechnology.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
	<CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>
	<6AE1EC8D-7076-4C2A-949F-3B938482358A@tablethotels.com>
	<201112151422.pBFEM3qJ031591@rs134.luxsci.com>
	<4EEA10CC.1050400@ceetonetechnology.com>
Message-ID: <CBE4C054-1F53-489C-B48D-8696068A0330@tablethotels.com>

On Dec 15, 2011, at 10:22 AM, George Rosamond wrote:

> On 12/15/11 09:21, Isaac Levy wrote:
>>> On Dec 15, 2011, at 1:00 AM, Eitan Adler wrote:
>>> 
>>>> This is a fun one:
>> On Dec 15, 2011, at 9:17 AM, Matthew Story wrote:
>> 
>>> ... you just made my morning sir ... that is awesome ?
>> 
>> I just spit coffee all over my desk- that's RAD.
>> 
>> Rocket-
>> .ike
> 
> I'm loosely following this thread. . . a bit swamped right now.
> 
> Not to divert it, but. . .
> 
> Is this a "Shell from Hell" meeting?

henry and I were hoping to compile a compendium of useful every-day patterns, functions, non-obvious but super-useful usages of "General Commands" ... nycbug talk seemed an ideal place to have this discussion ... and following the terminal mandelbrot fractal in zsh, I'm convinced we made the right decision.

if nycbug threw a Shell From Hell meeting, i would be in attendance ... i would definitely bring a note-pad.

> 
> g
> 
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
> 
> 



From ike at blackskyresearch.net  Thu Dec 15 11:24:09 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Thu, 15 Dec 2011 11:24:09 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <CBE4C054-1F53-489C-B48D-8696068A0330@tablethotels.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
	<CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>
	<6AE1EC8D-7076-4C2A-949F-3B938482358A@tablethotels.com>
	<201112151422.pBFEM3qJ031591@rs134.luxsci.com>
	<4EEA10CC.1050400@ceetonetechnology.com>
	<CBE4C054-1F53-489C-B48D-8696068A0330@tablethotels.com>
Message-ID: <201112151625.pBFGP6vo008557@rs134.luxsci.com>

On Dec 15, 2011, at 11:12 AM, Matthew Story wrote:

>> Not to divert it, but. . .
>> 
>> Is this a "Shell from Hell" meeting?
> 
> henry and I were hoping to compile a compendium of useful every-day patterns, functions, non-obvious but super-useful usages of "General Commands" ... nycbug talk seemed an ideal place to have this discussion ... and following the terminal mandelbrot fractal in zsh, I'm convinced we made the right decision.
> 
> if nycbug threw a Shell From Hell meeting, i would be in attendance ... i would definitely bring a note-pad.

I don't know if this thread could generate a meeting, but it sure does eradicate the need for a snowball/bikeshed of a NYC*BUG 'misc goodies repo':

http://lists.nycbug.org/pipermail/talk/2010-March/013033.html
>> does anyone on list object to using Git?
>>>>>>> 'while [ 1 ] ; do objections | objections ; done'

This nifty(1) thread is a social answer, not technical- IMHO perfect?

Rocket-
.ike



From matt at tablethotels.com  Thu Dec 15 11:48:08 2011
From: matt at tablethotels.com (Matthew Story)
Date: Thu, 15 Dec 2011 11:48:08 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <201112151625.pBFGP6vo008557@rs134.luxsci.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
	<CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>
	<6AE1EC8D-7076-4C2A-949F-3B938482358A@tablethotels.com>
	<201112151422.pBFEM3qJ031591@rs134.luxsci.com>
	<4EEA10CC.1050400@ceetonetechnology.com>
	<CBE4C054-1F53-489C-B48D-8696068A0330@tablethotels.com>
	<201112151625.pBFGP6vo008557@rs134.luxsci.com>
Message-ID: <1DCC112B-7219-4B45-B2D5-C5D2D9CB2F67@tablethotels.com>

On Dec 15, 2011, at 11:24 AM, Isaac Levy wrote:

> On Dec 15, 2011, at 11:12 AM, Matthew Story wrote:
> 
>>> Not to divert it, but. . .
>>> 
>>> Is this a "Shell from Hell" meeting?
>> 
>> henry and I were hoping to compile a compendium of useful every-day patterns, functions, non-obvious but super-useful usages of "General Commands" ... nycbug talk seemed an ideal place to have this discussion ... and following the terminal mandelbrot fractal in zsh, I'm convinced we made the right decision.
>> 
>> if nycbug threw a Shell From Hell meeting, i would be in attendance ... i would definitely bring a note-pad.
> 
> I don't know if this thread could generate a meeting, but it sure does eradicate the need for a snowball/bikeshed of a NYC*BUG 'misc goodies repo':
> 
> http://lists.nycbug.org/pipermail/talk/2010-March/013033.html
>>> does anyone on list object to using Git?
>>>>>>>> 'while [ 1 ] ; do objections | objections ; done'

# a script for managing consensus based decision making
# ... make sure everyone's voice is heard ... at web scale
exec >/dev/null
# ... make sure everyone's other voice is heard ... 
exec 2>&1

trolls
parse_objection() { echo "no" "no" "yes" "$1" "$2"; }
is_troll() {
    trolls="$trolls 
    $troll" 
    echo "$trolls" | while read troll; do
        [ "$troll" = "$1" ] && echo "troll"
    done
}
while [ 1 ]; do
while read author objection; do
   parse_objection "$author" "$objection" | {
       read reasonable relevant bikeshed author comment
       if [ ! "$reasonable" = "yes" -a "$relevant" = "yes" ]; then
           if [ "$bikeshed" = "yes" ]; then
               echo "$author (`is_troll "$author"`) has made a reasonable and relevant comment, but it is a bikeshed: $comment"
           else
               echo "something impossible happened ... neo is likely happening."
           fi
       else
           echo "$author (`is_troll "$author"`) has made a typically comment: $comment"
       fi
done
# it's important for every shell script to have an arbitrary sleep for unexplained reasons, better with an unnecessary comment
sleep 2 # sleep for 2
done

> 
> This nifty(1) thread is a social answer, not technical- IMHO perfect?
> 
> Rocket-
> .ike
> 
> 



From mspitzer at gmail.com  Thu Dec 15 12:29:23 2011
From: mspitzer at gmail.com (Marc Spitzer)
Date: Thu, 15 Dec 2011 12:29:23 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
	<CAF6rxg=6zj+6eHHyYfH8yy3OKX_DAjMq8C2850uqB7asgvUg-A@mail.gmail.com>
Message-ID: <CA+qr9e5wqos3XGaftDqJpNafSHY_-QHjjo-y47kh4=WO4++qsw@mail.gmail.com>

that is cool

marc

On Thu, Dec 15, 2011 at 1:00 AM, Eitan Adler <lists at eitanadler.com> wrote:
> This is a fun one:
>
> % function try_this_in_zsh {
> ?local lines columns colour a b p q i pnew
> ?((columns=COLUMNS-1, lines=LINES-1, colour=0))
> ?for ((b=-1.5; b<=1.5; b+=3.0/lines)) do
> ?for ((a=-2.0; a<=1; a+=3.0/columns)) do
> ?for ((p=0.0, q=0.0, i=0; p*p+q*q < 4 && i < 32; i++)) do
> ?((pnew=p*p-q*q+a, q=2*p*q+b, p=pnew))
> ?done
> ?((colour=(i/4)%8))
> ?echo -n "\\e[4${colour}m "
> ?done
> ?echo
> ?done
> ?}
> --
> Eitan Adler
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk



-- 
Freedom is nothing but a chance to be better.
--Albert Camus

?The problem with socialism is that eventually you run out
of other people's money.
--Margaret Thatcher

Do the arithmetic or be doomed to talk nonsense.
--John McCarthy


From spork at bway.net  Thu Dec 15 14:55:13 2011
From: spork at bway.net (Charles Sprickman)
Date: Thu, 15 Dec 2011 14:55:13 -0500
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <201112151414.pBFEE4jg016317@rs134.luxsci.com>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
	<201112151414.pBFEE4jg016317@rs134.luxsci.com>
Message-ID: <A290D621-22E9-4949-AEDA-DBDE8FB2721B@bway.net>


On Dec 15, 2011, at 9:13 AM, Isaac Levy wrote:

> On Dec 12, 2011, at 7:31 PM, Pete Wright wrote:
> 
>> On Mon, Dec 12, 2011 at 11:57:41AM -0500, Isaac Levy wrote:
>>> Hi All,
>>> 
>>> Disks: I need to measure disk performance across heterogeneous UNIX boxes, and I'm sick of bonnie++ for a bunch of reasons I won't' waste time on here.
>>> 
>>> I know from conversations that a few folks here have their own ways of testing disks-
>>> 
>>> I'd really like to know what people do to measure general disk performance?  e.g. really simple tests:
>>> 
>>> - r/w/d large files
>>> - r/w/d small files
>>> - disk performance when directories contain large numbers of files
>>> 
>>> I commonly have need to test things like:
>>> - different block sizes
>>> - different inode allocations (UFS/ext3
>>> - different filesystem partition layouts
>>> - different filesystem features (think ZFS fun)
>>> 
>>> Any thoughts, experiences, urls or shell utils to share?
>>> 
>> 
>> hey ike!
>> there is actually a pretty decent chapter on measuring disk and
>> filesystem performance in "High Performance PostgreSQL 9.0".  
> 
> Is it this book:
> "PostgreSQL 9.0 High Performance"
> Author: Gregory Smith
> Publisher: Packt Publishing

It is.  I just got that, haven't read the whole thing yet, but skimmed it and it is chock-full of good stuff.  I'm just starting to come up with a benchmark methodology for getting hard numbers comparing a spinning disk postgres box vs. and all-SSD postgres box.  That book (and the author, and lots of insanely great people on the postgresql-performance list) has steered me away from some boring and time-wasting benchmarks.

There are also a bunch of tools for postgres that help you test with your own data/workload:

http://wiki.postgresql.org/wiki/Statement_Playback

But yeah, even pgbench is going to give your disks a good workout.

I can't emphasize enough just how many hours you can lose if you start browsing the performance mailing list archives.  Fascinating stuff in there.

Charles

> 
>> they talk
>> about using a tool, bundled with bonie++, called zcav that will track
>> transfer rates from the begining to end of a disk subsystem.  it also
>> will output data into gnuplot friendly format for pcitures.  i used this
>> quite extensivly while tuning a linux dataware house a while back.
>> 
>> other tools that I'm happy with are iozone and fio:
>> 
>> http://www.iozone.org/
>> http://freecode.com/projects/fio
>> 
>> i find that when doing benchmarking of systems for eval purposes or
>> benchmarking i end up using a mixture of many different tools.  i find
>> that differnent tools will stress different parts of a given i/o
>> subsystem.  so i'd generally do something like:
>> 
>> - initial test using dd with variable blocksizes (dependent upon
>> underlying filesystem block size)
>> - several bonie++ tests, followed by some tests using iozone and fio
>> - depending on how system will be used in production i try some
>> application level tests.  for a db - pgbench, webserver apache bench
>> etc..
> 
> Understood, really comprehensive, and valuable!
> 
> Have you posted your methodology and/or resulting tests anywhere?
> 
> --
> Conversely, what I'm looking to really do here, is more of a cave-man sanity test: test a few small common cases in a relative vaccum, ideally using UNIX builtins spitting a few integers back to stdout.  (mix a bit of xargs with rm, mkdir, dd, etc?)
> 
> I'm actually looking to avoid going down this path alltogether, as I'd rather focus this level of detail on measuring disk/system performance as the team hammers our apps.
> 
>> 
>> I have also done some interesting testing using
> ..
>> http://tsung.erlang-projects.org/
> 
> *That* looks rad- way off direct disk testing, but that's a very compelling toolkit...
> 
> Thanks Pete!
> 
> Rocket-
> .ike
> 
> 
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk

--
Charles Sprickman
NetEng/SysAdmin
Bway.net - New York's Best Internet www.bway.net
spork at bway.net - 212.982.9800



From web-nycbug at superscript.com  Thu Dec 15 21:24:56 2011
From: web-nycbug at superscript.com (William Baxter)
Date: Thu, 15 Dec 2011 21:24:56 -0500
Subject: [nycbug-talk] nifty(1)
In-Reply-To: <7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
References: <750FC28E-C033-436B-A17A-052E24C09CF3@tablethotels.com>
	<201112142314.pBENE3X8002735@rs134.luxsci.com>
	<7E9045EC-84D0-49AD-9620-49EF73C04F5D@tablethotels.com>
Message-ID: <1324001249-sup-9155@hermes>

Excerpts from Matthew Story's message of Thu Dec 15 00:54:24 -0500 2011:
> i believe the most portable version is slightly uglier:
> 
> On Dec 14, 2011, at 6:13 PM, Isaac Levy wrote:
> 
> >  shout() { echo "$0: $*" >&2; }
> >  barf() { shout "$*"; exit 111; }
> safe() { ${1+"$@"} || barf "cannot $*"; }
>           ^^^^^^^^ this bit is more reliable for sh on Solaris (real Bourne shell)
> 

Delightful as the topic of Solaris compatibility may be I don't think it's worth
going that far.  The ${1+"$@"} construct is indispensable for compatibility in
cases when an empty list is legitimate while an unintended empty argument is
not:

  for ${1+"$@"}

versus

  for "$@"

But an empty string and an empty list are both erroneous for safe().  While the
execution details may vary from shell to shell, I consider it a case of pilot
error to pass in either an empty argument or no argument to a function that
requires a program.  A function cannot ultimately protect the caller from his
own error.

Cheers, W.


From pete at nomadlogic.org  Fri Dec 16 21:04:25 2011
From: pete at nomadlogic.org (Pete Wright)
Date: Fri, 16 Dec 2011 18:04:25 -0800
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <201112151414.pBFEE4jg016317@rs134.luxsci.com>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
	<201112151414.pBFEE4jg016317@rs134.luxsci.com>
Message-ID: <20111217020423.GD44941@arp.nomadlogic.org>

On Thu, Dec 15, 2011 at 09:13:55AM -0500, Isaac Levy wrote:
> > hey ike!
> > there is actually a pretty decent chapter on measuring disk and
> > filesystem performance in "High Performance PostgreSQL 9.0".  
> 
> Is it this book:
> "PostgreSQL 9.0 High Performance"
> Author: Gregory Smith
> Publisher: Packt Publishing

yea that's the one - can't recommend it enough for both sys admins as
well as for DBA's and programmers.  it is on everyone's desk here that
is involved in our datawarehouse/ETL projects.

> 
> Understood, really comprehensive, and valuable!
> 
> Have you posted your methodology and/or resulting tests anywhere?
>
well it's on talk@ now :p  i have been meaning to do a more in depth
write-up but b/w work, and trying not to think about work when i'm not
at work it's been hard to find the time :)

although i'm going to be testing solaris vs. freebsd zfs early next
year, maybe i should do a proper write-up...
 
> --
> Conversely, what I'm looking to really do here, is more of a cave-man sanity test: test a few small common cases in a relative vaccum, ideally using UNIX builtins spitting a few integers back to stdout.  (mix a bit of xargs with rm, mkdir, dd, etc?)
> 
> I'm actually looking to avoid going down this path alltogether, as I'd rather focus this level of detail on measuring disk/system performance as the team hammers our apps.
> 
> > 
> > I have also done some interesting testing using
> ..
> > http://tsung.erlang-projects.org/
> 
> *That* looks rad- way off direct disk testing, but that's a very compelling toolkit...
>
yea what i really liked about this tool was that i was able to do more
end-to-end testing of my disk subsystem.  we had a pool of
lighttpd+wscgi nodes mounting a NFS filesystem.  i was able to see how
the filer, OS and web scripts worked under pretty close to real-world
condition, which was pretty good.  so maybe tsung could help you with
your tests.



-pete
 
-- 
Pete Wright
pete at nomadlogic.org



From pete at nomadlogic.org  Fri Dec 16 21:14:11 2011
From: pete at nomadlogic.org (Pete Wright)
Date: Fri, 16 Dec 2011 18:14:11 -0800
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <CAK1r8CUHOVpZzSGj=D_OrqM28GvKy+Kc7Bmxg-+AbeMyaYsbvQ@mail.gmail.com>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
	<CAK1r8CUHOVpZzSGj=D_OrqM28GvKy+Kc7Bmxg-+AbeMyaYsbvQ@mail.gmail.com>
Message-ID: <20111217021409.GE44941@arp.nomadlogic.org>

On Thu, Dec 15, 2011 at 12:54:12AM -0500, Henry M wrote:
> Whenever I've bench-marked disks, I've always just used dd and /dev/zero
> 
>  Example: I want to see how fast I can write a 1GB file
> 
> $ dd if=/dev/zero of=1GB bs=1024 count=1048576
> 1048576+0 records in
> 1048576+0 records out
> 1073741824 bytes (1.1 GB) copied, 8.72947 s, 123 MB/s
> 
> As long as the system load is consistent, you should get consistent
> results. You can have fun by running multiple versions at once, to simulate
> heavier "real-world" load.
> 
> You can change the byte size, or count accordingly. Just be careful what
> values you give dd, you can easily fill up your disk, or break something
> nasty with a typo (Yes I've done both )
> 

not to rehash what everyone else mentioned regarding "dd", another but
interesting tool worth checking out in "lmbench".  lmdd supports
direct-io as well as altering fsync() behaviour, which can be helpful 
for testing hardware:

This is lmbench-3.0-a9, a (sometimes controversial) system performance
measurement tool.  lmbench is a suite of simple, portable, ANSI/C
microbenchmarks for UNIX/POSIX. In general, it measures two key
features:
latency and bandwidth. lmbench is intended to give system developers
insight
into basic costs of key operations. You can go to /usr/local/lib/lmbench
and
do one of the following:

make results (to run the benchmarks)
make rerun   (to rerun the benchmarks)
make see     (to see how you did)

-pete

-- 
Pete Wright
pete at nomadlogic.org



From ike at blackskyresearch.net  Mon Dec 19 09:17:38 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Mon, 19 Dec 2011 09:17:38 -0500
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <20111217020423.GD44941@arp.nomadlogic.org>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
	<201112151414.pBFEE4jg016317@rs134.luxsci.com>
	<20111217020423.GD44941@arp.nomadlogic.org>
Message-ID: <201112191419.pBJEJ4gH019238@rs134.luxsci.com>

On Dec 16, 2011, at 9:04 PM, Pete Wright wrote:

> On Thu, Dec 15, 2011 at 09:13:55AM -0500, Isaac Levy wrote:
> 
>> Have you posted your methodology and/or resulting tests anywhere?
>> 
> well it's on talk@ now :p  i have been meaning to do a more in depth
> write-up but b/w work, and trying not to think about work when i'm not
> at work it's been hard to find the time :)

Ha!  That's healthy!

> 
> although i'm going to be testing solaris vs. freebsd zfs early next
> year, maybe i should do a proper write-up?

That would be about as interesting on talk@, as a Yankees vs Mets in the world series... No big thing really?

(awesome please post this somewhere!!!)

Rocket-
.ike




From matt at tablethotels.com  Mon Dec 19 09:56:03 2011
From: matt at tablethotels.com (Matthew Story)
Date: Mon, 19 Dec 2011 09:56:03 -0500
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <201112191419.pBJEJ4gH019238@rs134.luxsci.com>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
	<201112151414.pBFEE4jg016317@rs134.luxsci.com>
	<20111217020423.GD44941@arp.nomadlogic.org>
	<201112191419.pBJEJ4gH019238@rs134.luxsci.com>
Message-ID: <2891D2A3-31B9-4FD2-8373-EE5DFEE4A19A@tablethotels.com>


> [...snip]
> That would be about as interesting on talk@, as a Yankees vs Mets in the world series... No big thing really?

isn't Redhat v. Debian more akin to Yankees v. Mets ... 




From bonsaime at gmail.com  Mon Dec 19 10:06:26 2011
From: bonsaime at gmail.com (Jesse Callaway)
Date: Mon, 19 Dec 2011 10:06:26 -0500
Subject: [nycbug-talk] Measuring Disk Performance
In-Reply-To: <2891D2A3-31B9-4FD2-8373-EE5DFEE4A19A@tablethotels.com>
References: <201112121658.pBCGw5A1032042@rs134.luxsci.com>
	<20111213003111.GA44941@arp.nomadlogic.org>
	<201112151414.pBFEE4jg016317@rs134.luxsci.com>
	<20111217020423.GD44941@arp.nomadlogic.org>
	<201112191419.pBJEJ4gH019238@rs134.luxsci.com>
	<2891D2A3-31B9-4FD2-8373-EE5DFEE4A19A@tablethotels.com>
Message-ID: <CAJU5ZBM7uAtut3_=qKW9SjD=WW+PiCQOvLavtkMsverHe+0Ezg@mail.gmail.com>

Nobody answer that! ; )
On Dec 19, 2011 9:57 AM, "Matthew Story" <matt at tablethotels.com> wrote:

>
> > [...snip]
> > That would be about as interesting on talk@, as a Yankees vs Mets in
> the world series... No big thing really?
>
> isn't Redhat v. Debian more akin to Yankees v. Mets ...
>
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111219/1f424c3d/attachment.htm>

From thomasjwinston at yahoo.com  Mon Dec 19 10:26:54 2011
From: thomasjwinston at yahoo.com (Thomas Winston)
Date: Mon, 19 Dec 2011 10:26:54 -0500
Subject: [nycbug-talk] Batteries FS: 2x 2U APC SmartUPS 3000 (208V)
Message-ID: <000a01ccbe62$a41a6f10$ec4f4d30$@com>

Sorry for the spam - I hope this isn't against the rules.

 

I'm only sending this because I have a car in the city this week and I can
deliver - so I'm a motivated seller. 

 

They work 100%, rack was upgraded to a larger battery system and I can't
take them home or the wife will kill me. These are 208V, generation just
prior to the newest LCD models (no LCD). I'm happy to deliver, need them
gone. 

 

Mods/Admins - if this is against the rules please delete & I'm really sorry.
Let me know and I won't do it again. 

 

Thx!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111219/fc599570/attachment.htm>

From matt at atopia.net  Mon Dec 19 20:27:42 2011
From: matt at atopia.net (Matt Juszczak)
Date: Mon, 19 Dec 2011 20:27:42 -0500 (EST)
Subject: [nycbug-talk] Load Balanced jabber server in FreeBSD ports?
Message-ID: <alpine.BSF.2.00.1112192026090.65661@venus.atopia.net>

Hi folks,

I have a client who is in need of a jabber server that can handle a lot of 
users and scale horizontally. I looked into OpenFire but it looks like 
their enterprise version is the only thing that supports clustering.

Can anyone recommend a simple jabber server that supports load balacing 
and/or horizontal scaling, perhaps with a shared back end database and 
simple load balancer, preferably available in the FreeBSD ports 
collection?

Clustering (ie: being fault tolerant) isn't as important as being able to 
handle more users than one server can handle.

-Matt


From bcully at gmail.com  Mon Dec 19 20:45:39 2011
From: bcully at gmail.com (Brian Cully)
Date: Mon, 19 Dec 2011 20:45:39 -0500
Subject: [nycbug-talk] Load Balanced jabber server in FreeBSD ports?
In-Reply-To: <alpine.BSF.2.00.1112192026090.65661@venus.atopia.net>
References: <alpine.BSF.2.00.1112192026090.65661@venus.atopia.net>
Message-ID: <E61A2E3F-07C1-42DC-AB13-D9023D922D00@gmail.com>

On Dec 19, 2011, at 8:27 PM, Matt Juszczak wrote:

> Can anyone recommend a simple jabber server that supports load balacing and/or horizontal scaling, perhaps with a shared back end database and simple load balancer, preferably available in the FreeBSD ports collection?
> 
> Clustering (ie: being fault tolerant) isn't as important as being able to handle more users than one server can handle.

	IIRC, ejabberd is in ports. Out-of-the-box it can handle more load than any other XMPP server out there and is extremely reliable, mostly due to running on top of Erlang. It does clustering as well, once again using Erlang's native clustering support.

	There are /no/ servers that I know of that do real horizontal scaling, although you can take most of them and fake it using domain-based sharding or other such techniques, and that's hardly a simple solution. Clustering will work for most people, but mainly as a fail-safe, not as much for scaling, since they all rely on ACID-compliant databases, which will always prove a bottleneck to scaling at some point.

	ejabberd will be your best bet. OpenFire is basically dead in the water and rarely gets any updates or support anymore. The C based ones are all at least as bad. The only alternative is Prosody, which is fast and works well, but it's a single-threaded design that attempts to hack around being single-threaded by using coroutines. If you don't need to modify it, though, it's worth looking at (but don't expect it to use more than one CPU, it won't).

-bjc

From riegersteve at gmail.com  Mon Dec 19 20:59:28 2011
From: riegersteve at gmail.com (Steve)
Date: Mon, 19 Dec 2011 17:59:28 -0800
Subject: [nycbug-talk] tomcat question
Message-ID: <4EEFEC00.3070508@gmail.com>

Its been a while, so am asking for your opinion. Will a heavily used 
tomcat run well on fbsd ?

i know that many years back there were some java issues, but you folks 
are the experts...




From pete at nomadlogic.org  Tue Dec 20 11:13:46 2011
From: pete at nomadlogic.org (Pete Wright)
Date: Tue, 20 Dec 2011 08:13:46 -0800
Subject: [nycbug-talk] Load Balanced jabber server in FreeBSD ports?
In-Reply-To: <E61A2E3F-07C1-42DC-AB13-D9023D922D00@gmail.com>
References: <alpine.BSF.2.00.1112192026090.65661@venus.atopia.net>
	<E61A2E3F-07C1-42DC-AB13-D9023D922D00@gmail.com>
Message-ID: <4EF0B43A.5070704@nomadlogic.org>

On 12/19/11 5:45 PM, Brian Cully wrote:
> On Dec 19, 2011, at 8:27 PM, Matt Juszczak wrote:
>
>> Can anyone recommend a simple jabber server that supports load balacing and/or horizontal scaling, perhaps with a shared back end database and simple load balancer, preferably available in the FreeBSD ports collection?
>>
>> Clustering (ie: being fault tolerant) isn't as important as being able to handle more users than one server can handle.
> 	IIRC, ejabberd is in ports. Out-of-the-box it can handle more load than any other XMPP server out there and is extremely reliable, mostly due to running on top of Erlang. It does clustering as well, once again using Erlang's native clustering support.

+1 on ejabberd as well.  i actually find it to be the easiest jabber 
implementation going administration wise, and after working in an large 
erlang shop for a while i came to appreciate the power of Mnesia 
(erlang's distributed in-memory database) when it comes to scaling 
applications.

-pete

-- 
Pete Wright
pete at nomadlogic.org
www.nomadlogic.org



From pete at nomadlogic.org  Wed Dec 21 20:09:44 2011
From: pete at nomadlogic.org (Pete Wright)
Date: Wed, 21 Dec 2011 17:09:44 -0800
Subject: [nycbug-talk] tomcat question
In-Reply-To: <4EEFEC00.3070508@gmail.com>
References: <4EEFEC00.3070508@gmail.com>
Message-ID: <20111222010942.GC85087@arp.nomadlogic.org>

On Mon, Dec 19, 2011 at 05:59:28PM -0800, Steve wrote:
> Its been a while, so am asking for your opinion. Will a heavily used 
> tomcat run well on fbsd ?
> 
> i know that many years back there were some java issues, but you folks 
> are the experts...
> 
hi steve - was hoping someone else had production experience running
tomcat and/or java on talk@ but i guess not :)

i'm hoping to spend a couple days poking at it next week during our
slack peroid at work and i'll let ya know how it looks.

-p

-- 
Pete Wright
pete at nomadlogic.org



From mark.saad at ymail.com  Wed Dec 21 15:45:09 2011
From: mark.saad at ymail.com (Mark Saad)
Date: Wed, 21 Dec 2011 15:45:09 -0500
Subject: [nycbug-talk] tomcat question
In-Reply-To: <20111222010942.GC85087@arp.nomadlogic.org>
References: <4EEFEC00.3070508@gmail.com>
	<20111222010942.GC85087@arp.nomadlogic.org>
Message-ID: <CAMXt9Nag7ZkO7PWvCMUddtC8V_P0unEBXukY4xwOCuGgYFD8vA@mail.gmail.com>

On Wed, Dec 21, 2011 at 8:09 PM, Pete Wright <pete at nomadlogic.org> wrote:
> On Mon, Dec 19, 2011 at 05:59:28PM -0800, Steve wrote:
>> Its been a while, so am asking for your opinion. Will a heavily used
>> tomcat run well on fbsd ?

Generally it will , I have used Tomcat 5 and 6 using Diablo JDK 5 and
6 and OpenJDK 6 on both FreeBSD 6.3 and 7.2 with out many issues that
were BSD related.


>>
>> i know that many years back there were some java issues, but you folks
>> are the experts...

So the issues I had with java code running in tomcat on freebsd were
related to developers doing bad things and assuming that their code
was only going to run on linux. This also meant that their code ran
poorly or not at all on windows and other supported platforms.

>>
> hi steve - was hoping someone else had production experience running
> tomcat and/or java on talk@ but i guess not :)
>
> i'm hoping to spend a couple days poking at it next week during our
> slack peroid at work and i'll let ya know how it looks.
>
> -p
>
> --
> Pete Wright
> pete at nomadlogic.org
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk



-- 

Mark Saad | mark.saad at ymail.com


From mark.saad at ymail.com  Wed Dec 21 15:52:20 2011
From: mark.saad at ymail.com (Mark Saad)
Date: Wed, 21 Dec 2011 15:52:20 -0500
Subject: [nycbug-talk] SAMBA Question
Message-ID: <CAMXt9NZsxApZLTbrW1Gy1WAJ3oJP0kdCAy5nXdK7thEfo9s_1A@mail.gmail.com>

All
 I am working on upgrading a old SAMBA 3.0.16 server to 3.6.1 . I am
using Openldap as my password and user back end.
The issue I have is after the upgrade users are prompted with a error
stating their password has expired.


Take this example from pbedit -v msaad


Unix username:        msaad
NT username:          msaad
Account Flags:        [UX        ]
User SID:             S-1-5-21-64374432-364290046-XXXXXXXX-2970
Primary Group SID:    S-1-5-21-3988802677-3356876598-XXXXXXXX--513
Full Name:            Mark Saad
Home Directory:       \\nycifs3\msaad
HomeDir Drive:
Logon Script:
Profile Path:         \\nycifs3\msaad\profile
Domain:               NYCIFS3
Account desc:         badluck
Workstations:
Munged dial:
Logon time:           0
Logoff time:          never
Kickoff time:         never
Password last set:    Thu, 01 Jan 1970 00:00:10 GMT
Password can change:  Thu, 01 Jan 1970 00:00:10 GMT
Password must change: never
Last bad password   : 0
Bad password count  : 0
Logon hours         : FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF


For some odd reason under samba 3.6.1 my password last set / change
are set to Jan 1 1970 and updating my ldap password has no effect ?

Does anyone have any ideas ?

-- 

Mark Saad | mark.saad at ymail.com


From jhell at DataIX.net  Thu Dec 22 01:28:50 2011
From: jhell at DataIX.net (Jason Hellenthal)
Date: Thu, 22 Dec 2011 01:28:50 -0500
Subject: [nycbug-talk] SAMBA Question
In-Reply-To: <CAMXt9NZsxApZLTbrW1Gy1WAJ3oJP0kdCAy5nXdK7thEfo9s_1A@mail.gmail.com>
References: <CAMXt9NZsxApZLTbrW1Gy1WAJ3oJP0kdCAy5nXdK7thEfo9s_1A@mail.gmail.com>
Message-ID: <20111222062850.GA57651@DataIX.net>



On Wed, Dec 21, 2011 at 03:52:20PM -0500, Mark Saad wrote:
> All
>  I am working on upgrading a old SAMBA 3.0.16 server to 3.6.1 . I am
> using Openldap as my password and user back end.
> The issue I have is after the upgrade users are prompted with a error
> stating their password has expired.
> 
> 
> Take this example from pbedit -v msaad
> 
> 
> Unix username:        msaad
> NT username:          msaad
> Account Flags:        [UX        ]
> User SID:             S-1-5-21-64374432-364290046-XXXXXXXX-2970
> Primary Group SID:    S-1-5-21-3988802677-3356876598-XXXXXXXX--513
> Full Name:            Mark Saad
> Home Directory:       \\nycifs3\msaad
> HomeDir Drive:
> Logon Script:
> Profile Path:         \\nycifs3\msaad\profile
> Domain:               NYCIFS3
> Account desc:         badluck
> Workstations:
> Munged dial:
> Logon time:           0
> Logoff time:          never
> Kickoff time:         never
> Password last set:    Thu, 01 Jan 1970 00:00:10 GMT
> Password can change:  Thu, 01 Jan 1970 00:00:10 GMT
> Password must change: never
> Last bad password   : 0
> Bad password count  : 0
> Logon hours         : FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
> 
> 
> For some odd reason under samba 3.6.1 my password last set / change
> are set to Jan 1 1970 and updating my ldap password has no effect ?
> 
> Does anyone have any ideas ?
> 

Is it possible you have a registry hive in place enforcing these settings ?

net registry <?>

Also check out:

net sam policy <...>

-- 
;s =;


From george at ceetonetechnology.com  Fri Dec 23 20:57:10 2011
From: george at ceetonetechnology.com (George Rosamond)
Date: Fri, 23 Dec 2011 20:57:10 -0500
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security Team
In-Reply-To: <4EF4A0A8.3000707@freebsd.org>
References: <4EF4A0A8.3000707@freebsd.org>
Message-ID: <4EF53176.2080902@ceetonetechnology.com>

For those who didn't see this. . . wow.

g

-------- Original Message --------
Subject: Merry Christmas from the FreeBSD Security Team
Date: Fri, 23 Dec 2011 07:39:20 -0800
From: FreeBSD Security Officer <cpeSNIPbsd.org>
Reply-To: security-ofSNIPbsd.org
Organization: FreeBSD Project
To: freebsd-announce at freebsd.org, freebsd-security-notifications at freebsd.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

No, the Grinch didn't steal the FreeBSD security officer GPG key, and 
your eyes
aren't deceiving you: We really did just send out 5 security advisories.

The timing, to put it bluntly, sucks.  We normally aim to release 
advisories on
Wednesdays in order to maximize the number of system administrators who 
will be
at work already; and we try very hard to avoid issuing advisories any 
time close
to holidays for the same reason.  The start of the Christmas weekend -- 
in some
parts of the world it's already Saturday -- is absolutely not when we 
want to be
releasing security advisories.

Unfortunately my hand was forced: One of the issues 
(FreeBSD-SA-11:08.telnetd)
is a remote root vulnerability which is being actively exploited in the 
wild;
bugs really don't come any worse than this.  On the positive side, most 
people
have moved past telnet and on to SSH by now; but this is still not an 
issue we
could postpone until a more convenient time.

While I'm writing, a note to freebsd-update users: 
FreeBSD-SA-11:07.chroot has a
rather messy fix involving adding a new interface to libc; this has the 
awkward
side effect of causing the sizes of some "symbols" (aka. functions) in 
libc to
change, resulting in cascading changes into many binaries.  The long list of
updated files is irritating, but isn't a sign that anything in 
freebsd-update
went wrong.

- -- 
Colin Percival
Security Officer, FreeBSD | freebsd.org | The power to serve
Founder / author, Tarsnap | tarsnap.com | Online backups for the truly 
paranoid
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (FreeBSD)

iEYEARECAAYFAk70oKgACgkQFdaIBMps37IsdACgh01CeO+zVGe3o9dn2cLvhh70
ISoAoJCeLUAbJ+0ibyfbVM4fYxpiEfo0
=vt5I
-----END PGP SIGNATURE-----
_______________________________________________
freebsd-security-notifications at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security-notifications
To unsubscribe, send any mail to 
"freebsd-security-notifications-unsubscribe at freebsd.org"


From pete at nomadlogic.org  Fri Dec 23 22:07:36 2011
From: pete at nomadlogic.org (Pete Wright)
Date: Fri, 23 Dec 2011 19:07:36 -0800
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
	Team
In-Reply-To: <4EF53176.2080902@ceetonetechnology.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
Message-ID: <op.v6y4ayjdovpiju@rincon.phyber.com>

On Fri, 23 Dec 2011 17:57:10 -0800, George Rosamond  
<george at ceetonetechnology.com> wrote:

> For those who didn't see this. . . wow.
>
thanks for the heads up gman!

-p

--
Pete Wright
pete at nomadlogic.org
www.nomadlogic.org


From edlinuxguru at gmail.com  Sat Dec 24 10:45:49 2011
From: edlinuxguru at gmail.com (Edward Capriolo)
Date: Sat, 24 Dec 2011 10:45:49 -0500
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
	Team
In-Reply-To: <op.v6y4ayjdovpiju@rincon.phyber.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
	<op.v6y4ayjdovpiju@rincon.phyber.com>
Message-ID: <CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>

Telnet? Crap I am really worried that my internet facing echo service might
be compromised. Not.

On Friday, December 23, 2011, Pete Wright <pete at nomadlogic.org> wrote:
> On Fri, 23 Dec 2011 17:57:10 -0800, George Rosamond <
george at ceetonetechnology.com> wrote:
>
>> For those who didn't see this. . . wow.
>>
> thanks for the heads up gman!
>
> -p
>
> --
> Pete Wright
> pete at nomadlogic.org
> www.nomadlogic.org
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111224/d25d98da/attachment.htm>

From george at ceetonetechnology.com  Sat Dec 24 11:09:55 2011
From: george at ceetonetechnology.com (George Rosamond)
Date: Sat, 24 Dec 2011 11:09:55 -0500
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
 Team
In-Reply-To: <CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
	<op.v6y4ayjdovpiju@rincon.phyber.com>
	<CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
Message-ID: <4EF5F953.7070101@ceetonetechnology.com>

On 12/24/11 10:45, Edward Capriolo wrote:
> Telnet? Crap I am really worried that my internet facing echo service might
> be compromised. Not.
>
> On Friday, December 23, 2011, Pete Wright<pete at nomadlogic.org>  wrote:
>> On Fri, 23 Dec 2011 17:57:10 -0800, George Rosamond<
> george at ceetonetechnology.com>  wrote:
>>
>>> For those who didn't see this. . . wow.
>>>
>> thanks for the heads up gman!
>>

Awww Ed. . . you told me all your boxes still were listening on tcp/23!

The significance of the vulnerabilities is more about quantity.  Thought 
Colin Percival's email was compromised.

And freebsd-update seems to bork on some of the libc stuff.

g


From matt at atopia.net  Sat Dec 24 10:47:29 2011
From: matt at atopia.net (Matt Juszczak)
Date: Sat, 24 Dec 2011 15:47:29 +0000
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD
	Security	Team
In-Reply-To: <CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
	<op.v6y4ayjdovpiju@rincon.phyber.com>
	<CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
Message-ID: <61981217-1324741651-cardhu_decombobulator_blackberry.rim.net-1100310521-@b27.c7.bise6.blackberry>

There are others :(

-----Original Message-----
From: Edward Capriolo <edlinuxguru at gmail.com>
Sender: talk-bounces at lists.nycbug.org
Date: Sat, 24 Dec 2011 10:45:49 
To: Pete Wright<pete at nomadlogic.org>
Cc: talk<talk at lists.nycbug.org>
Subject: Re: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
	Team

_______________________________________________
talk mailing list
talk at lists.nycbug.org
http://lists.nycbug.org/mailman/listinfo/talk




From pete at nomadlogic.org  Sat Dec 24 13:33:33 2011
From: pete at nomadlogic.org (Pete Wright)
Date: Sat, 24 Dec 2011 10:33:33 -0800
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
	Team
In-Reply-To: <4EF5F953.7070101@ceetonetechnology.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
	<op.v6y4ayjdovpiju@rincon.phyber.com>
	<CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
	<4EF5F953.7070101@ceetonetechnology.com>
Message-ID: <op.v60a57d4ovpiju@rincon.phyber.com>

On Sat, 24 Dec 2011 08:09:55 -0800, George Rosamond  
<george at ceetonetechnology.com> wrote:


> The significance of the vulnerabilities is more about quantity.  Thought  
> Colin Percival's email was compromised.
>
> And freebsd-update seems to bork on some of the libc stuff.
>

yea noticed that as well, although it looks harmless (on my system at  
least).

-pg


-- 
Pete Wright
pete at nomadlogic.org
www.nomadlogic.org


From bonsaime at gmail.com  Sat Dec 24 15:04:18 2011
From: bonsaime at gmail.com (Jesse Callaway)
Date: Sat, 24 Dec 2011 15:04:18 -0500
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
	Team
In-Reply-To: <4EF5F953.7070101@ceetonetechnology.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
	<op.v6y4ayjdovpiju@rincon.phyber.com>
	<CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
	<4EF5F953.7070101@ceetonetechnology.com>
Message-ID: <CAJU5ZBNtZsUtStdSQWroU4S5TdsEwBsZ4XPo-FVBKtaxCqZs_g@mail.gmail.com>

telnet might be useful for running things like machinery such as a
generator or maybe an industrial centrifuge...
of course such things only run in a network which is properly firewalled.
an internal private network where node based security need not be worried
about in the first place.
On Dec 24, 2011 11:10 AM, "George Rosamond" <george at ceetonetechnology.com>
wrote:

> On 12/24/11 10:45, Edward Capriolo wrote:
>
>> Telnet? Crap I am really worried that my internet facing echo service
>> might
>> be compromised. Not.
>>
>> On Friday, December 23, 2011, Pete Wright<pete at nomadlogic.org>  wrote:
>>
>>> On Fri, 23 Dec 2011 17:57:10 -0800, George Rosamond<
>>>
>> george at ceetonetechnology.com>  wrote:
>>
>>>
>>>  For those who didn't see this. . . wow.
>>>>
>>>>  thanks for the heads up gman!
>>>
>>>
> Awww Ed. . . you told me all your boxes still were listening on tcp/23!
>
> The significance of the vulnerabilities is more about quantity.  Thought
> Colin Percival's email was compromised.
>
> And freebsd-update seems to bork on some of the libc stuff.
>
> g
> ______________________________**_________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/**mailman/listinfo/talk<http://lists.nycbug.org/mailman/listinfo/talk>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111224/0f72b6ee/attachment.htm>

From billtotman at billtotman.com  Sat Dec 24 15:47:32 2011
From: billtotman at billtotman.com (Bill Totman)
Date: Sat, 24 Dec 2011 15:47:32 -0500
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
	Team
In-Reply-To: <CAJU5ZBNtZsUtStdSQWroU4S5TdsEwBsZ4XPo-FVBKtaxCqZs_g@mail.gmail.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
	<op.v6y4ayjdovpiju@rincon.phyber.com>
	<CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
	<4EF5F953.7070101@ceetonetechnology.com>
	<CAJU5ZBNtZsUtStdSQWroU4S5TdsEwBsZ4XPo-FVBKtaxCqZs_g@mail.gmail.com>
Message-ID: <2125739B-36F4-458B-AF08-D5F8A4E79D9C@billtotman.com>

On Dec 24, 2011, at 15:04, Jesse Callaway <bonsaime at gmail.com> wrote:

> telnet might be useful for running things like machinery such as a generator or maybe an industrial centrifuge... 
> of course such things only run in a network which is properly firewalled. an internal private network where node based security need not be worried about in the first place.
> 

Why depend on the firewalling being properly configured when layers of security, particularly if SSH is freely available, can be taken advantage of?


-bt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111224/4f66a07b/attachment.htm>

From george at ceetonetechnology.com  Sat Dec 24 15:54:43 2011
From: george at ceetonetechnology.com (George Rosamond)
Date: Sat, 24 Dec 2011 15:54:43 -0500
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
 Team
In-Reply-To: <CAJU5ZBNtZsUtStdSQWroU4S5TdsEwBsZ4XPo-FVBKtaxCqZs_g@mail.gmail.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
	<op.v6y4ayjdovpiju@rincon.phyber.com>
	<CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
	<4EF5F953.7070101@ceetonetechnology.com>
	<CAJU5ZBNtZsUtStdSQWroU4S5TdsEwBsZ4XPo-FVBKtaxCqZs_g@mail.gmail.com>
Message-ID: <4EF63C13.2050102@ceetonetechnology.com>

On 12/24/11 15:04, Jesse Callaway wrote:
> telnet might be useful for running things like machinery such as a
> generator or maybe an industrial centrifuge...
> of course such things only run in a network which is properly firewalled.
> an internal private network where node based security need not be worried
> about in the first place.

Yeah, very valid point.  Mock telnet all you want, but it's still there.

Same with ftp.  Rumor has it lots of trading data moves between the 
various players by ftp.  Still.

Of course, there were other vulnerabilities revealed, like the pam and 
libc-related stuff.

I had no problem updating boxes from source with make with FreeBSD 7.x 
and 8.x, but freebsd-update seemed to bork, as I mentioned earlier, with 
libc stuff on both 7.x and 8.x.

/usr/src/lib/libc/gen/libc_dlopen.c

This 'cheat' solves it. . . but icky.

http://icesquare.com/wordpress/freebsd-updateinstalling-updates-install-usrsrcliblibcgenlibc_dlopen-c-no-such-file-or-directory/

g


From george at ceetonetechnology.com  Sat Dec 24 16:00:39 2011
From: george at ceetonetechnology.com (George Rosamond)
Date: Sat, 24 Dec 2011 16:00:39 -0500
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
 Team
In-Reply-To: <2125739B-36F4-458B-AF08-D5F8A4E79D9C@billtotman.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
	<op.v6y4ayjdovpiju@rincon.phyber.com>
	<CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
	<4EF5F953.7070101@ceetonetechnology.com>
	<CAJU5ZBNtZsUtStdSQWroU4S5TdsEwBsZ4XPo-FVBKtaxCqZs_g@mail.gmail.com>
	<2125739B-36F4-458B-AF08-D5F8A4E79D9C@billtotman.com>
Message-ID: <4EF63D77.5070603@ceetonetechnology.com>

On 12/24/11 15:47, Bill Totman wrote:
> On Dec 24, 2011, at 15:04, Jesse Callaway<bonsaime at gmail.com>
> wrote:
>
>> telnet might be useful for running things like machinery such as a
>> generator or maybe an industrial centrifuge... of course such
>> things only run in a network which is properly firewalled. an
>> internal private network where node based security need not be
>> worried about in the first place.
>>
>
> Why depend on the firewalling being properly configured when layers
> of security, particularly if SSH is freely available, can be taken
> advantage of?
>

Typical security SE ;)

Not everyone can just upgrade their Cisco IOS with the release schedule 
or whatever.

Think embedded systems and industrial environments.  The majority of 
computer infrastructures are not cutsy friendly user interfaces with 
admins tailling logs.  It's more like clunky, dated systems that can't 
be overhauled without screwing up elevators or shutting down a 4000 
person factory for a week.

g




From bonsaime at gmail.com  Sat Dec 24 23:10:26 2011
From: bonsaime at gmail.com (Jesse Callaway)
Date: Sat, 24 Dec 2011 23:10:26 -0500
Subject: [nycbug-talk] Fwd: Merry Christmas from the FreeBSD Security
	Team
In-Reply-To: <2125739B-36F4-458B-AF08-D5F8A4E79D9C@billtotman.com>
References: <4EF4A0A8.3000707@freebsd.org>
	<4EF53176.2080902@ceetonetechnology.com>
	<op.v6y4ayjdovpiju@rincon.phyber.com>
	<CAENxBww6OfcS9JcYhOVwwEjXzN9RBRuqa40cAuFMf-+GoTdkug@mail.gmail.com>
	<4EF5F953.7070101@ceetonetechnology.com>
	<CAJU5ZBNtZsUtStdSQWroU4S5TdsEwBsZ4XPo-FVBKtaxCqZs_g@mail.gmail.com>
	<2125739B-36F4-458B-AF08-D5F8A4E79D9C@billtotman.com>
Message-ID: <CAJU5ZBOW9ADbFsyeyJJg63EGDPXQ0actnO_L-mJKoKW4dgVM4w@mail.gmail.com>

On Sat, Dec 24, 2011 at 3:47 PM, Bill Totman <billtotman at billtotman.com> wrote:
> On Dec 24, 2011, at 15:04, Jesse Callaway <bonsaime at gmail.com> wrote:
>
> telnet might be useful for running things like machinery such as a generator
> or maybe an industrial centrifuge...
> of course such things only run in a network which is properly firewalled. an
> internal private network where node based security need not be worried about
> in the first place.
>
>
> Why depend on the firewalling being properly configured when layers of
> security, particularly if SSH is freely available, can be taken advantage
> of?
>
>
> -bt
>
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>

Oh I was joking. It was a jab at the saps in Iran who got saddled with stuxnet.


-- 
-jesse


From bob at redivi.com  Sun Dec 25 13:03:07 2011
From: bob at redivi.com (Bob Ippolito)
Date: Sun, 25 Dec 2011 13:03:07 -0500
Subject: [nycbug-talk] Shmoocon 2012?
Message-ID: <CACwMPm8bCYZS+qWCo8D-sGt_EYOwMBoqksxMesVYMMU4jvVmfg@mail.gmail.com>

Anyone going to this Shmoocon? Thinking about flying out for this one, been
a while since I've gone to something on the east coast and I'm trying to
convince myself that it's about time :)

-bob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111225/2059b5a9/attachment.htm>

From wxs at FreeBSD.org  Sun Dec 25 14:54:54 2011
From: wxs at FreeBSD.org (Wesley Shields)
Date: Sun, 25 Dec 2011 14:54:54 -0500
Subject: [nycbug-talk] Shmoocon 2012?
In-Reply-To: <CACwMPm8bCYZS+qWCo8D-sGt_EYOwMBoqksxMesVYMMU4jvVmfg@mail.gmail.com>
References: <CACwMPm8bCYZS+qWCo8D-sGt_EYOwMBoqksxMesVYMMU4jvVmfg@mail.gmail.com>
Message-ID: <20111225195454.GB44709@atarininja.org>

On Sun, Dec 25, 2011 at 01:03:07PM -0500, Bob Ippolito wrote:
> Anyone going to this Shmoocon? Thinking about flying out for this one, been
> a while since I've gone to something on the east coast and I'm trying to
> convince myself that it's about time :)

If you don't already have a Shmoocon ticket I wouldn't be booking your
flight. Tickets sold out insanely fast this year, like every year.

-- WXS


From bob at redivi.com  Sun Dec 25 16:35:35 2011
From: bob at redivi.com (Bob Ippolito)
Date: Sun, 25 Dec 2011 16:35:35 -0500
Subject: [nycbug-talk] Shmoocon 2012?
In-Reply-To: <20111225195454.GB44709@atarininja.org>
References: <CACwMPm8bCYZS+qWCo8D-sGt_EYOwMBoqksxMesVYMMU4jvVmfg@mail.gmail.com>
	<20111225195454.GB44709@atarininja.org>
Message-ID: <CACwMPm-efHk-0AzGrfNKzYqiAqsk9CH1bxJJ6zLXASYt-20-qg@mail.gmail.com>

On Sunday, December 25, 2011, Wesley Shields <wxs at freebsd.org> wrote:
> On Sun, Dec 25, 2011 at 01:03:07PM -0500, Bob Ippolito wrote:
>> Anyone going to this Shmoocon? Thinking about flying out for this one,
been
>> a while since I've gone to something on the east coast and I'm trying to
>> convince myself that it's about time :)
>
> If you don't already have a Shmoocon ticket I wouldn't be booking your
> flight. Tickets sold out insanely fast this year, like every year.

Yeah, I always forget before they sell out but they're selling another
batch on Jan 1, so I'd try and buy one then.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111225/950dd029/attachment.htm>

From henry95 at gmail.com  Sun Dec 25 21:38:57 2011
From: henry95 at gmail.com (Henry M)
Date: Sun, 25 Dec 2011 21:38:57 -0500
Subject: [nycbug-talk] Shmoocon 2012?
In-Reply-To: <CACwMPm-efHk-0AzGrfNKzYqiAqsk9CH1bxJJ6zLXASYt-20-qg@mail.gmail.com>
References: <CACwMPm8bCYZS+qWCo8D-sGt_EYOwMBoqksxMesVYMMU4jvVmfg@mail.gmail.com>
	<20111225195454.GB44709@atarininja.org>
	<CACwMPm-efHk-0AzGrfNKzYqiAqsk9CH1bxJJ6zLXASYt-20-qg@mail.gmail.com>
Message-ID: <CAK1r8CUtPVBYOuJkG=3ZTcBmn7ZpaA+kHMv3Zg7YUi+UjFQUXQ@mail.gmail.com>

The last batch sold out in less than 10 seconds. If you get an extra one,
let me know : ]



On Sun, Dec 25, 2011 at 4:35 PM, Bob Ippolito <bob at redivi.com> wrote:

>
>
> On Sunday, December 25, 2011, Wesley Shields <wxs at freebsd.org> wrote:
> > On Sun, Dec 25, 2011 at 01:03:07PM -0500, Bob Ippolito wrote:
> >> Anyone going to this Shmoocon? Thinking about flying out for this one,
> been
> >> a while since I've gone to something on the east coast and I'm trying to
> >> convince myself that it's about time :)
> >
> > If you don't already have a Shmoocon ticket I wouldn't be booking your
> > flight. Tickets sold out insanely fast this year, like every year.
>
> Yeah, I always forget before they sell out but they're selling another
> batch on Jan 1, so I'd try and buy one then.
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111225/0dbfa5bf/attachment.htm>

From SPORK at BWAY.NET  Mon Dec 26 02:18:01 2011
From: SPORK at BWAY.NET (Charles Sprickman)
Date: Mon, 26 Dec 2011 02:18:01 -0500
Subject: [nycbug-talk] console "screencast" tools?
Message-ID: <257F1F29-7772-4038-ADD1-6D82C8099A90@BWAY.NET>

I really like to save my late-night oddball questions for this list.

When I'm documenting things in a wiki (in my case, mediawiki), I
often find myself doing a tedious series of copy and paste
operations from my terminal to "pre" blocks in the wiki.  It's time
consuming, error-prone, and not really great documentation when you
need to show a longer series of "do this, look for output like this"
operations.

I know I could run some fancy screen capture thing, upload some huge
video to one of the video sharing sites (forgetting the privacy
concerns for the moment), and then embed that in the wiki.  But that
seems incredibly wasteful.  We're just talking text here, not a bunch
of point-n-drool config panels?

Is there any clever way of doing this that I've missed?  I have seen
some neat examples of a bunch of text getting "played back" in a
broswer using javascript to control a page element with a ton of
text, but not anything that maintains a certain pacing.  I'd settle
for video as a final format if I could find some alternate way of
capturing the session that is more like "script" (but with timing
cues) than a real screen capture.  The idea is that whatever tool
builds the movie is doing the actual text rendering so I'd at least
end up with a clean, easy to compress video.  Hope that makes some
kind of sense.

Any of you folks that make it to lots of conferences ever see a
presenter using some novel way of showing a sequence of commands
like this?

Thanks,

Charles

From jhell at DataIX.net  Mon Dec 26 09:26:30 2011
From: jhell at DataIX.net (Jason Hellenthal)
Date: Mon, 26 Dec 2011 09:26:30 -0500
Subject: [nycbug-talk] console "screencast" tools?
In-Reply-To: <257F1F29-7772-4038-ADD1-6D82C8099A90@BWAY.NET>
References: <257F1F29-7772-4038-ADD1-6D82C8099A90@BWAY.NET>
Message-ID: <20111226142630.GA17435@DataIX.net>


See script(1) or vidcontrol(1) -p or -P

On Mon, Dec 26, 2011 at 02:18:01AM -0500, Charles Sprickman wrote:
> I really like to save my late-night oddball questions for this list.
> 
> When I'm documenting things in a wiki (in my case, mediawiki), I
> often find myself doing a tedious series of copy and paste
> operations from my terminal to "pre" blocks in the wiki.  It's time
> consuming, error-prone, and not really great documentation when you
> need to show a longer series of "do this, look for output like this"
> operations.
> 
> I know I could run some fancy screen capture thing, upload some huge
> video to one of the video sharing sites (forgetting the privacy
> concerns for the moment), and then embed that in the wiki.  But that
> seems incredibly wasteful.  We're just talking text here, not a bunch
> of point-n-drool config panels?
> 
> Is there any clever way of doing this that I've missed?  I have seen
> some neat examples of a bunch of text getting "played back" in a
> broswer using javascript to control a page element with a ton of
> text, but not anything that maintains a certain pacing.  I'd settle
> for video as a final format if I could find some alternate way of
> capturing the session that is more like "script" (but with timing
> cues) than a real screen capture.  The idea is that whatever tool
> builds the movie is doing the actual text rendering so I'd at least
> end up with a clean, easy to compress video.  Hope that makes some
> kind of sense.
> 
> Any of you folks that make it to lots of conferences ever see a
> presenter using some novel way of showing a sequence of commands
> like this?
> 
> Thanks,
> 
> Charles
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk

-- 
;s =;


From ike at blackskyresearch.net  Mon Dec 26 13:30:19 2011
From: ike at blackskyresearch.net (Isaac Levy)
Date: Mon, 26 Dec 2011 13:30:19 -0500
Subject: [nycbug-talk] Shmoocon 2012?
In-Reply-To: <CACwMPm8bCYZS+qWCo8D-sGt_EYOwMBoqksxMesVYMMU4jvVmfg@mail.gmail.com>
References: <CACwMPm8bCYZS+qWCo8D-sGt_EYOwMBoqksxMesVYMMU4jvVmfg@mail.gmail.com>
Message-ID: <201112261831.pBQIV3tL006139@rs134.luxsci.com>

On Dec 25, 2011, at 1:03 PM, Bob Ippolito wrote:

> Anyone going to this Shmoocon? Thinking about flying out for this one, been a while since I've gone to something on the east coast and I'm trying to convince myself that it's about time :)
> 
> -bob 

I'm going- scored a ticket through quite a long story...

If anyone isn't going, but can help snag tickets for the people who want to, please try to get registration codes when they're doled out on the 1st!

Rocket-
.ike




From matt at tablethotels.com  Mon Dec 26 17:07:38 2011
From: matt at tablethotels.com (Matthew Story)
Date: Mon, 26 Dec 2011 17:07:38 -0500
Subject: [nycbug-talk] Shmoocon 2012?
In-Reply-To: <201112261831.pBQIV3tL006139@rs134.luxsci.com>
References: <CACwMPm8bCYZS+qWCo8D-sGt_EYOwMBoqksxMesVYMMU4jvVmfg@mail.gmail.com>
	<201112261831.pBQIV3tL006139@rs134.luxsci.com>
Message-ID: <1C3FA5A6-04D5-412F-A875-1361B983A854@tablethotels.com>


On Dec 26, 2011, at 1:30 PM, Isaac Levy wrote:

> If anyone isn't going, but can help snag tickets for the people who want to, please try to get registration codes when they're doled out on the 1st!

I agree with this anon ...

> 
> Rocket-
> .ike
> 
> 
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
> 



From wxs at FreeBSD.org  Mon Dec 26 18:06:10 2011
From: wxs at FreeBSD.org (Wesley Shields)
Date: Mon, 26 Dec 2011 18:06:10 -0500
Subject: [nycbug-talk] Shmoocon 2012?
In-Reply-To: <201112261831.pBQIV3tL006139@rs134.luxsci.com>
References: <CACwMPm8bCYZS+qWCo8D-sGt_EYOwMBoqksxMesVYMMU4jvVmfg@mail.gmail.com>
	<201112261831.pBQIV3tL006139@rs134.luxsci.com>
Message-ID: <20111226230610.GC53551@atarininja.org>

On Mon, Dec 26, 2011 at 01:30:19PM -0500, Isaac Levy wrote:
> On Dec 25, 2011, at 1:03 PM, Bob Ippolito wrote:
> 
> > Anyone going to this Shmoocon? Thinking about flying out for this one, been a while since I've gone to something on the east coast and I'm trying to convince myself that it's about time :)
> > 
> > -bob 
> 
> I'm going- scored a ticket through quite a long story...
> 
> If anyone isn't going, but can help snag tickets for the people who
> want to, please try to get registration codes when they're doled out
> on the 1st!

I'll be there too. If you see one of the security guys with short hair
and a bunch of earrings stop me and say hi. ;)

-- WXS


From george at ceetonetechnology.com  Thu Dec 29 09:51:37 2011
From: george at ceetonetechnology.com (George Rosamond)
Date: Thu, 29 Dec 2011 09:51:37 -0500
Subject: [nycbug-talk] Two insights for year end
Message-ID: <4EFC7E79.2000700@ceetonetechnology.com>

You can quickly change your vim text to ROT-13 for 2nd century BCE 
encryption with:

ggVGg?

https://secure.wikimedia.org/wikipedia/en/wiki/Rot13

Reminds me of that fake work document online you could quickly switch to 
in the mid-1990's.  Some found it a useful tool when someone was looking 
over your shoulder.

*	*	*

And most importantly:

Biometrics with butts from Schneier:

https://www.schneier.com/blog/archives/2011/12/butt_identifica.html

I wonder if those with boney butts are flagged easier.  Like having some 
scar on your fingerprint from flicking those childhood red cap-gun 
ribbons with your fingernails. . .

g


From mark.saad at ymail.com  Fri Dec 30 15:15:49 2011
From: mark.saad at ymail.com (Mark Saad)
Date: Fri, 30 Dec 2011 15:15:49 -0500
Subject: [nycbug-talk] Strange Apache Coredump behavior
In-Reply-To: <alpine.LRH.2.00.1112291011370.10486@ecs00.cozzaenterprises.com>
References: <CAMXt9Nao0C0uJuBar84nEqEykybPat-8yZWdE0bFPo4idNdH9A@mail.gmail.com>
	<alpine.LRH.2.00.1112291011370.10486@ecs00.cozzaenterprises.com>
Message-ID: <CAMXt9NbcuPpQzX5ZX60RS9_3-vdAoxBk6YiwBfmFRr_Wubj9Tg@mail.gmail.com>

On Thu, Dec 29, 2011 at 10:14 AM, Brian Seklecki (Mobile)
<root at probikesllc.com> wrote:
>>
>>
>> I have no cores and I am stumped.
>>
>
> Check the apache RC script; it may be setting ulimit(1) on coredump.
>
> Also try to run a "Hello World" segfault tester as the www/httpd user and
> see if you get your dump.
>
> ?int main(void)
> ?{
> ? ? char *s = "hello world";
> ? ? *s = 'H';
> ?}

The end result of that issue was I forgot to set kern.sugid_coredump=1
in sysctl.conf .

 Also in FreeBSD 7.3 gcore would not produce any usable coredumps from
apache. I could not figure that out.


-- 

Mark Saad | mark.saad at ymail.com


From bonsaime at gmail.com  Fri Dec 30 15:43:13 2011
From: bonsaime at gmail.com (Jesse Callaway)
Date: Fri, 30 Dec 2011 15:43:13 -0500
Subject: [nycbug-talk] Strange Apache Coredump behavior
In-Reply-To: <CAMXt9NbcuPpQzX5ZX60RS9_3-vdAoxBk6YiwBfmFRr_Wubj9Tg@mail.gmail.com>
References: <CAMXt9Nao0C0uJuBar84nEqEykybPat-8yZWdE0bFPo4idNdH9A@mail.gmail.com>
	<alpine.LRH.2.00.1112291011370.10486@ecs00.cozzaenterprises.com>
	<CAMXt9NbcuPpQzX5ZX60RS9_3-vdAoxBk6YiwBfmFRr_Wubj9Tg@mail.gmail.com>
Message-ID: <CAJU5ZBN-uaBpptCFWqVAEYKyrXhWdO0tRzpZRFTT7=DFVyd=gw@mail.gmail.com>

Apache https has its own corefile options. Since it starts as root it may
override or ignore your system settings. I don't know the syntax offhand,
and but the docs at apache.org should give some love.
On Dec 30, 2011 3:17 PM, "Mark Saad" <mark.saad at ymail.com> wrote:

> On Thu, Dec 29, 2011 at 10:14 AM, Brian Seklecki (Mobile)
> <root at probikesllc.com> wrote:
> >>
> >>
> >> I have no cores and I am stumped.
> >>
> >
> > Check the apache RC script; it may be setting ulimit(1) on coredump.
> >
> > Also try to run a "Hello World" segfault tester as the www/httpd user and
> > see if you get your dump.
> >
> >  int main(void)
> >  {
> >     char *s = "hello world";
> >     *s = 'H';
> >  }
>
> The end result of that issue was I forgot to set kern.sugid_coredump=1
> in sysctl.conf .
>
>  Also in FreeBSD 7.3 gcore would not produce any usable coredumps from
> apache. I could not figure that out.
>
>
> --
>
> Mark Saad | mark.saad at ymail.com
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111230/b292b986/attachment.htm>

From bonsaime at gmail.com  Fri Dec 30 15:43:42 2011
From: bonsaime at gmail.com (Jesse Callaway)
Date: Fri, 30 Dec 2011 15:43:42 -0500
Subject: [nycbug-talk] Strange Apache Coredump behavior
In-Reply-To: <CAJU5ZBN-uaBpptCFWqVAEYKyrXhWdO0tRzpZRFTT7=DFVyd=gw@mail.gmail.com>
References: <CAMXt9Nao0C0uJuBar84nEqEykybPat-8yZWdE0bFPo4idNdH9A@mail.gmail.com>
	<alpine.LRH.2.00.1112291011370.10486@ecs00.cozzaenterprises.com>
	<CAMXt9NbcuPpQzX5ZX60RS9_3-vdAoxBk6YiwBfmFRr_Wubj9Tg@mail.gmail.com>
	<CAJU5ZBN-uaBpptCFWqVAEYKyrXhWdO0tRzpZRFTT7=DFVyd=gw@mail.gmail.com>
Message-ID: <CAJU5ZBMUH+y-9mKBKBpTd_b=7Gww0pqaxRfQaeS+r8s6AxOzHA@mail.gmail.com>

s/https/ with a d
On Dec 30, 2011 3:43 PM, "Jesse Callaway" <bonsaime at gmail.com> wrote:

> Apache https has its own corefile options. Since it starts as root it may
> override or ignore your system settings. I don't know the syntax offhand,
> and but the docs at apache.org should give some love.
> On Dec 30, 2011 3:17 PM, "Mark Saad" <mark.saad at ymail.com> wrote:
>
>> On Thu, Dec 29, 2011 at 10:14 AM, Brian Seklecki (Mobile)
>> <root at probikesllc.com> wrote:
>> >>
>> >>
>> >> I have no cores and I am stumped.
>> >>
>> >
>> > Check the apache RC script; it may be setting ulimit(1) on coredump.
>> >
>> > Also try to run a "Hello World" segfault tester as the www/httpd user
>> and
>> > see if you get your dump.
>> >
>> >  int main(void)
>> >  {
>> >     char *s = "hello world";
>> >     *s = 'H';
>> >  }
>>
>> The end result of that issue was I forgot to set kern.sugid_coredump=1
>> in sysctl.conf .
>>
>>  Also in FreeBSD 7.3 gcore would not produce any usable coredumps from
>> apache. I could not figure that out.
>>
>>
>> --
>>
>> Mark Saad | mark.saad at ymail.com
>> _______________________________________________
>> talk mailing list
>> talk at lists.nycbug.org
>> http://lists.nycbug.org/mailman/listinfo/talk
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111230/15abcc90/attachment.htm>

From mark.saad at ymail.com  Fri Dec 30 16:46:39 2011
From: mark.saad at ymail.com (Mark Saad)
Date: Fri, 30 Dec 2011 16:46:39 -0500
Subject: [nycbug-talk] Strange Apache Coredump behavior
In-Reply-To: <CAJU5ZBMUH+y-9mKBKBpTd_b=7Gww0pqaxRfQaeS+r8s6AxOzHA@mail.gmail.com>
References: <CAMXt9Nao0C0uJuBar84nEqEykybPat-8yZWdE0bFPo4idNdH9A@mail.gmail.com>
	<alpine.LRH.2.00.1112291011370.10486@ecs00.cozzaenterprises.com>
	<CAMXt9NbcuPpQzX5ZX60RS9_3-vdAoxBk6YiwBfmFRr_Wubj9Tg@mail.gmail.com>
	<CAJU5ZBN-uaBpptCFWqVAEYKyrXhWdO0tRzpZRFTT7=DFVyd=gw@mail.gmail.com>
	<CAJU5ZBMUH+y-9mKBKBpTd_b=7Gww0pqaxRfQaeS+r8s6AxOzHA@mail.gmail.com>
Message-ID: <CAMXt9NZPxsj7GmYmt6ZL0EHCNH=Uz5x53ohgEaF8OLWn2eSc0Q@mail.gmail.com>

On Fri, Dec 30, 2011 at 3:43 PM, Jesse Callaway <bonsaime at gmail.com> wrote:
> s/https/ with a d
>
> On Dec 30, 2011 3:43 PM, "Jesse Callaway" <bonsaime at gmail.com> wrote:
>>
>> Apache https has its own corefile options. Since it starts as root it may
>> override or ignore your system settings. I don't know the syntax offhand,
>> and but the docs at apache.org should give some love.
>>
About that, the CoreDumpDirectory option did not seem to make any
difference. I ended up removing it since it did not do anything for
us.

 From what I could tell it only helps when you want to override the
kern.corefile sysctl . In my case I had defined it as
kern.corefile=/var/coredumps/%U-%N-%P.core . The biggest pit fall with
that sysctl was that I originally quoted the value which did not work
ie kern.corefile="/var/coredumps/%U-%N-%P.core" This instructed the
kernel to try and make cores named "/var/coredumps/... and it would
fail since there was not directory named "



>> On Dec 30, 2011 3:17 PM, "Mark Saad" <mark.saad at ymail.com> wrote:
>>>
>>> On Thu, Dec 29, 2011 at 10:14 AM, Brian Seklecki (Mobile)
>>> <root at probikesllc.com> wrote:
>>> >>
>>> >>
>>> >> I have no cores and I am stumped.
>>> >>
>>> >
>>> > Check the apache RC script; it may be setting ulimit(1) on coredump.
>>> >
>>> > Also try to run a "Hello World" segfault tester as the www/httpd user
>>> > and
>>> > see if you get your dump.
>>> >
>>> > ?int main(void)
>>> > ?{
>>> > ? ? char *s = "hello world";
>>> > ? ? *s = 'H';
>>> > ?}
>>>
>>> The end result of that issue was I forgot to set kern.sugid_coredump=1
>>> in sysctl.conf .
>>>
>>> ?Also in FreeBSD 7.3 gcore would not produce any usable coredumps from
>>> apache. I could not figure that out.
>>>
>>>
>>> --
>>>
>>> Mark Saad | mark.saad at ymail.com
>>> _______________________________________________
>>> talk mailing list
>>> talk at lists.nycbug.org
>>> http://lists.nycbug.org/mailman/listinfo/talk



-- 

Mark Saad | mark.saad at ymail.com


From bonsaime at gmail.com  Fri Dec 30 19:19:22 2011
From: bonsaime at gmail.com (Jesse Callaway)
Date: Fri, 30 Dec 2011 19:19:22 -0500
Subject: [nycbug-talk] Strange Apache Coredump behavior
In-Reply-To: <CAMXt9NZPxsj7GmYmt6ZL0EHCNH=Uz5x53ohgEaF8OLWn2eSc0Q@mail.gmail.com>
References: <CAMXt9Nao0C0uJuBar84nEqEykybPat-8yZWdE0bFPo4idNdH9A@mail.gmail.com>
	<alpine.LRH.2.00.1112291011370.10486@ecs00.cozzaenterprises.com>
	<CAMXt9NbcuPpQzX5ZX60RS9_3-vdAoxBk6YiwBfmFRr_Wubj9Tg@mail.gmail.com>
	<CAJU5ZBN-uaBpptCFWqVAEYKyrXhWdO0tRzpZRFTT7=DFVyd=gw@mail.gmail.com>
	<CAJU5ZBMUH+y-9mKBKBpTd_b=7Gww0pqaxRfQaeS+r8s6AxOzHA@mail.gmail.com>
	<CAMXt9NZPxsj7GmYmt6ZL0EHCNH=Uz5x53ohgEaF8OLWn2eSc0Q@mail.gmail.com>
Message-ID: <CAJU5ZBPztQmgxeuEcbTJ3whBz4Rt_MAw9eH5Tyj2tv1PAxQhgg@mail.gmail.com>

oh okay.... duh I was responding to your follow-up saying that you fixed it
On Dec 30, 2011 4:48 PM, "Mark Saad" <mark.saad at ymail.com> wrote:

> On Fri, Dec 30, 2011 at 3:43 PM, Jesse Callaway <bonsaime at gmail.com>
> wrote:
> > s/https/ with a d
> >
> > On Dec 30, 2011 3:43 PM, "Jesse Callaway" <bonsaime at gmail.com> wrote:
> >>
> >> Apache https has its own corefile options. Since it starts as root it
> may
> >> override or ignore your system settings. I don't know the syntax
> offhand,
> >> and but the docs at apache.org should give some love.
> >>
> About that, the CoreDumpDirectory option did not seem to make any
> difference. I ended up removing it since it did not do anything for
> us.
>
>  From what I could tell it only helps when you want to override the
> kern.corefile sysctl . In my case I had defined it as
> kern.corefile=/var/coredumps/%U-%N-%P.core . The biggest pit fall with
> that sysctl was that I originally quoted the value which did not work
> ie kern.corefile="/var/coredumps/%U-%N-%P.core" This instructed the
> kernel to try and make cores named "/var/coredumps/... and it would
> fail since there was not directory named "
>
>
>
> >> On Dec 30, 2011 3:17 PM, "Mark Saad" <mark.saad at ymail.com> wrote:
> >>>
> >>> On Thu, Dec 29, 2011 at 10:14 AM, Brian Seklecki (Mobile)
> >>> <root at probikesllc.com> wrote:
> >>> >>
> >>> >>
> >>> >> I have no cores and I am stumped.
> >>> >>
> >>> >
> >>> > Check the apache RC script; it may be setting ulimit(1) on coredump.
> >>> >
> >>> > Also try to run a "Hello World" segfault tester as the www/httpd user
> >>> > and
> >>> > see if you get your dump.
> >>> >
> >>> >  int main(void)
> >>> >  {
> >>> >     char *s = "hello world";
> >>> >     *s = 'H';
> >>> >  }
> >>>
> >>> The end result of that issue was I forgot to set kern.sugid_coredump=1
> >>> in sysctl.conf .
> >>>
> >>>  Also in FreeBSD 7.3 gcore would not produce any usable coredumps from
> >>> apache. I could not figure that out.
> >>>
> >>>
> >>> --
> >>>
> >>> Mark Saad | mark.saad at ymail.com
> >>> _______________________________________________
> >>> talk mailing list
> >>> talk at lists.nycbug.org
> >>> http://lists.nycbug.org/mailman/listinfo/talk
>
>
>
> --
>
> Mark Saad | mark.saad at ymail.com
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20111230/6e77d632/attachment.htm>