From riegersteve at gmail.com Fri Sep 1 14:09:46 2006 From: riegersteve at gmail.com (Steve Rieger) Date: Fri, 01 Sep 2006 11:09:46 -0700 Subject: [nycbug-talk] zabbix agentd question Message-ID: <44F8776A.6030809@gmail.com> i know we have at least one person here that uses zabbix can i use awk and $ in agentd.conf, or do you know a way around the error below. UserParameter=descriptors.available,awk '{print $3-$1+$2}' /proc/sys/fs/file-nr UserParameter=descriptors.error,awk '($1-$2)>(.9*$3) {exit 1}' /proc/sys/fs/file-nr [root at stg2 log]# zabbix_agentd -t descriptors.available awk: cmd. line:1: {print -+} awk: cmd. line:1: ^ syntax error descriptors.available [m|ZBX_NOTSUPPORTED] [root at stg2 log]# zabbix_agentd -t descriptors.error awk: cmd. line:1: (-)>(.9*) {exit 1} awk: cmd. line:1: ^ syntax error awk: cmd. line:1: (-)>(.9*) {exit 1} awk: cmd. line:1: ^ syntax error awk: cmd. line:1: (-)>(.9*) {exit 1} awk: cmd. line:1: ^ syntax error awk: cmd. line:1: (-)>(.9*) {exit 1} awk: cmd. line:1: ^ syntax error descriptors.error [m|ZBX_NOTSUPPORTED] -- -- eats the blues for breakfast, does unix for rent, plays harp for food, will play the flute for kicks rides for the freedom scrapes for thechallenge From skreuzer at f2o.org Fri Sep 1 15:02:39 2006 From: skreuzer at f2o.org (Steven Kreuzer) Date: Fri, 01 Sep 2006 15:02:39 -0400 Subject: [nycbug-talk] Charles M. Hannum on NetBSD In-Reply-To: <20060831090240.4aadb83a@wit.genoverly.com> References: <20060831090240.4aadb83a@wit.genoverly.com> Message-ID: <44F883CF.8050207@f2o.org> michael wrote: > wow.. this has hit several lists, but if you missed it somehow.. > > "The NetBSD Project has stagnated to the point of irrelevance. > It has gotten to the point that being associated with the > project is often more of a liability than an asset. I will > attempt to explain how this happened, what the current state of > affairs is, and what needs to be done to attempt to fix the > situation." > > http://mail-index.netbsd.org/netbsd-users/2006/08/30/0016.html > Looks like the NetBSD foundation is kicking out a number of developers: http://mail-index.netbsd.org/netbsd-announce/2006/09/01/0000.html Cut and Paste of the interesting part: Over the past year, as the last step in the process of reorganization of the Foundation that began in 2002, we have made a concerted effort to contact those remaining developers without current agreements and ensure their continued participation in NetBSD. Despite hundreds of hours spent on this process by our volunteers we have not obtained agreements from a few people. Our Board and Executive Committee on Membership therefore directed that developer access for those without agreements be disabled effective Friday, September 1, 2006; all those affected by this change were notified one week in advance. We therefore bid a fond farewell to the following NetBSD developers, who have made many significant contributions to NetBSD in the past 13 years, for which we are very grateful. We expect that as time and circumstances permit, we may see many of their faces again, along with the 346 others who help make NetBSD what it is today: Lennart Augustsson Matt Debergalis Brian C. Grayson Charles M. Hannum Matthias Pfaller Dante Profeta Darren Reed Kazuki Sakamoto From george at sddi.net Fri Sep 1 15:14:52 2006 From: george at sddi.net (George R.) Date: Fri, 01 Sep 2006 15:14:52 -0400 Subject: [nycbug-talk] Charles M. Hannum on NetBSD In-Reply-To: <44F883CF.8050207@f2o.org> References: <20060831090240.4aadb83a@wit.genoverly.com> <44F883CF.8050207@f2o.org> Message-ID: <44F886AC.1010308@sddi.net> Steven Kreuzer wrote: > michael wrote: >> wow.. this has hit several lists, but if you missed it somehow.. >> >> "The NetBSD Project has stagnated to the point of irrelevance. >> It has gotten to the point that being associated with the >> project is often more of a liability than an asset. I will >> attempt to explain how this happened, what the current state of >> affairs is, and what needs to be done to attempt to fix the >> situation." >> >> http://mail-index.netbsd.org/netbsd-users/2006/08/30/0016.html >> > > Looks like the NetBSD foundation is kicking out a number of developers: > > http://mail-index.netbsd.org/netbsd-announce/2006/09/01/0000.html > > Cut and Paste of the interesting part: > > Over the past year, as the last step in the process of reorganization > of the Foundation that began in 2002, we have made a concerted effort > to contact those remaining developers without current agreements and > ensure their continued participation in NetBSD. Despite hundreds of > hours spent on this process by our volunteers we have not obtained > agreements from a few people. Our Board and Executive Committee on > Membership therefore directed that developer access for those without > agreements be disabled effective Friday, September 1, 2006; all those > affected by this change were notified one week in advance. > > We therefore bid a fond farewell to the following NetBSD developers, > who have made many significant contributions to NetBSD in the past 13 > years, for which we are very grateful. We expect that as time and > circumstances permit, we may see many of their faces again, along with > the 346 others who help make NetBSD what it is today: > > Lennart Augustsson > Matt Debergalis > Brian C. Grayson > Charles M. Hannum > Matthias Pfaller > Dante Profeta > Darren Reed > Kazuki Sakamoto With all the civil wars that have taken place in the open source world, I'm a little shocked when submerged civil wars like this become press release material. This is somewhere between some people losing commit privileges and the OpenBSD split. . . and it's being portrayed as the former, but seems to be more of the later. I really don't know why the NBSD Foundation would be publicizing this. . . g From mspitzer at gmail.com Fri Sep 1 15:32:11 2006 From: mspitzer at gmail.com (Marc Spitzer) Date: Fri, 1 Sep 2006 15:32:11 -0400 Subject: [nycbug-talk] Charles M. Hannum on NetBSD In-Reply-To: <44F886AC.1010308@sddi.net> References: <20060831090240.4aadb83a@wit.genoverly.com> <44F883CF.8050207@f2o.org> <44F886AC.1010308@sddi.net> Message-ID: <8c50a3c30609011232u147a002cs872b37d8868d3d80@mail.gmail.com> On 9/1/06, George R. wrote: > > With all the civil wars that have taken place in the open source world, > I'm a little shocked when submerged civil wars like this become press > release material. > > This is somewhere between some people losing commit privileges and the > OpenBSD split. . . and it's being portrayed as the former, but seems to > be more of the later. > > I really don't know why the NBSD Foundation would be publicizing this. . . > > g Could be a case of get your spin out first, iff they think that there will be a mess created by the ex developers it makes sense to try to minimize it before it gets going marc -- "We trained very hard, but it seemed that every time we were beginning to form into teams we would be reorganized. I was to learn later in life that we tend to meet any new situation by reorganizing, and a wonderful method it can be for creating the illusion of progress, while producing confusion, inefficiency and demoralization." -Gaius Petronius, 1st Century AD From lists at genoverly.net Fri Sep 1 15:44:37 2006 From: lists at genoverly.net (michael) Date: Fri, 1 Sep 2006 15:44:37 -0400 Subject: [nycbug-talk] Charles M. Hannum on NetBSD In-Reply-To: <8c50a3c30609011232u147a002cs872b37d8868d3d80@mail.gmail.com> References: <20060831090240.4aadb83a@wit.genoverly.com> <44F883CF.8050207@f2o.org> <44F886AC.1010308@sddi.net> <8c50a3c30609011232u147a002cs872b37d8868d3d80@mail.gmail.com> Message-ID: <20060901154437.72f0bab6@wit.genoverly.com> On Fri, 1 Sep 2006 15:32:11 -0400 "Marc Spitzer" wrote: > On 9/1/06, George R. wrote: > > > > With all the civil wars that have taken place in the open source > > world, I'm a little shocked when submerged civil wars like this > > become press release material. > > > > This is somewhere between some people losing commit privileges and > > the OpenBSD split. . . and it's being portrayed as the former, but > > seems to be more of the later. > > > > I really don't know why the NBSD Foundation would be publicizing > > this. . . > > > > g > > Could be a case of get your spin out first, iff they think that there > will be a mess created by the ex developers it makes sense to try to > minimize it before it gets going > > marc Yes, but.. Hannum was on the list of dismissed. Looks like he "got his spin on" before the official announcement. >From a user's perspective, NetBSD puts out a solid product. Why would they publicize this? Marc gave one reason. Another is.. because hiding secrets in an open source project could be counter-productive. It looks like they are just doing some house cleaning and it creating a little noise. Hey, we know many of the guys here in NYC.. maybe they could shed a little light. -- Michael From jschauma at netmeister.org Fri Sep 1 20:34:06 2006 From: jschauma at netmeister.org (Jan Schaumann) Date: Fri, 1 Sep 2006 20:34:06 -0400 Subject: [nycbug-talk] Charles M. Hannum on NetBSD In-Reply-To: <44F886AC.1010308@sddi.net> References: <20060831090240.4aadb83a@wit.genoverly.com> <44F883CF.8050207@f2o.org> <44F886AC.1010308@sddi.net> Message-ID: <20060902003406.GF17256@netmeister.org> "George R." wrote: > This is somewhere between some people losing commit privileges and the > OpenBSD split. . . and it's being portrayed as the former, but seems to > be more of the later. > > I really don't know why the NBSD Foundation would be publicizing this. . . The NetBSD Foundation published this because the NetBSD Foundation believes that open communication with their users on all matters is important. NetBSD is an _Open_ Source project, after all. As stated in the announcement, a lot of effort has been put into getting the developers to sign the developer's agreement. Some have chosen not to sign the agreement. Since their contributions are valued, they are not silently "kicked out", but instead they are publicly thanked for their work. Regrettably have to relinquish their privileges, but it would be trivial for them to gain them back (by signing said agreement). Note: I do not speak on behalf of the NetBSD Foundation. This is my personal take on things. -Jan -- Free Speech Online - Stop Internet Censorship --- Electronic Frontier Foundation -- http://www.eff.org --- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 186 bytes Desc: not available URL: From george at galis.org Fri Sep 1 22:54:34 2006 From: george at galis.org (George Georgalis) Date: Fri, 1 Sep 2006 22:54:34 -0400 Subject: [nycbug-talk] Charles M. Hannum on NetBSD In-Reply-To: <20060902003406.GF17256@netmeister.org> References: <20060831090240.4aadb83a@wit.genoverly.com> <44F883CF.8050207@f2o.org> <44F886AC.1010308@sddi.net> <20060902003406.GF17256@netmeister.org> Message-ID: <20060902025434.GI10960@run.galis.org> On Fri, Sep 01, 2006 at 08:34:06PM -0400, Jan Schaumann wrote: >"George R." wrote: > >> This is somewhere between some people losing commit privileges and the >> OpenBSD split. . . and it's being portrayed as the former, but seems to >> be more of the later. >> >> I really don't know why the NBSD Foundation would be publicizing this. . . > >The NetBSD Foundation published this because the NetBSD Foundation >believes that open communication with their users on all matters is >important. NetBSD is an _Open_ Source project, after all. right. I read the first announcement, it was pretty matter of fact. As a legal foundation, certain rules are inflexible. While I didn't read the particular documents, it is my sense that the developers that lost their bit where simply unavailable or not willing to sign a commitment to follow bylaws. Here's another paragraph from the announcement. In 2002 the developers of NetBSD, who are the members and owners of the Foundation, voted to reorganize the corporation through an open process of instituting new Bylaws, electing a new Board, and making good on all legal obligations, such as back taxes and fees. In the past 4 years, NetBSD has grown and flourished under the supervision of four Boards of directors elected by the membership, adding 83 developers, and releasing 6 new versions of NetBSD and 12 quarterly branches of pkgsrc, its third-party packaging system, with (currently) 6226 packages. NetBSD rocks btw. // George -- George Georgalis, systems architect, administrator < From trish at bsdunix.net Sat Sep 2 23:06:47 2006 From: trish at bsdunix.net (Trish Lynch) Date: Sun, 3 Sep 2006 03:06:47 +0000 Subject: [nycbug-talk] Fw: Recovering partition table and ntfs filesystem (intact) (on and off-topic) Message-ID: <675034872-1157252814-cardhu_blackberry.rim.net-1433514753-@bxe029-cell01.bisx.prod.on.blackberry> Sent to wrong address.... ------Original Message------ To: talk at nycbug.org Sent: Sep 2, 2006 10:45 PM Subject: Recovering partition table and ntfs filesystem (intact) (on and off-topic) I accidentally deleted the partition table of the wrong drive that contains 15,000 dollars worth of encoded cds that have been lost in moves or opacked in boxes and such and would take more then that type of investment to re-encode. I lost the backup of it last week in a machine crash, and I was attempting to rebuild it when I deleted the wrong disk partition table (I just had a back surgery, teach me to dfo stuff on drugs) Is there any way to recover the partition table and the intact ntfs partition on there? Or do I have to send it out to be recovered? (Which is more than I can afford right now, I cannot believe I lost all my music *grrrrr*)) -Trish -- Trish Lynch From alex at pilosoft.com Sat Sep 2 23:34:32 2006 From: alex at pilosoft.com (alex at pilosoft.com) Date: Sat, 2 Sep 2006 23:34:32 -0400 (EDT) Subject: [nycbug-talk] Fw: Recovering partition table and ntfs filesystem (intact) (on and off-topic) In-Reply-To: <675034872-1157252814-cardhu_blackberry.rim.net-1433514753-@bxe029-cell01.bisx.prod.on.blackberry> Message-ID: On Sun, 3 Sep 2006, Trish Lynch wrote: > To: talk at nycbug.org Sent: Sep 2, 2006 10:45 PM Subject: Recovering > partition table and ntfs filesystem (intact) (on and off-topic) > > I accidentally deleted the partition table of the wrong drive that > contains 15,000 dollars worth of encoded cds that have been lost in > moves or opacked in boxes and such and would take more then that type of > investment to re-encode. > I lost the backup of it last week in a machine crash, and I was > attempting to rebuild it when I deleted the wrong disk partition table > (I just had a back surgery, teach me to dfo stuff on drugs) > > Is there any way to recover the partition table and the intact ntfs > partition on there? Or do I have to send it out to be recovered? (Which > is more than I can afford right now, I cannot believe I lost all my > music *grrrrr*)) Yes try this first: http://www.cgsecurity.org/wiki/TestDisk then http://www.vanheusden.com/findfile/ There's a whole bunch of commercial partition table recovery software: just google for 'partition table recovery' From nycbug-list at 2xlp.com Sun Sep 3 01:17:45 2006 From: nycbug-list at 2xlp.com (Jonathan Vanasco) Date: Sun, 3 Sep 2006 01:17:45 -0400 Subject: [nycbug-talk] Fw: Recovering partition table and ntfs filesystem (intact) (on and off-topic) In-Reply-To: References: Message-ID: On Sep 2, 2006, at 11:34 PM, alex at pilosoft.com wrote: > There's a whole bunch of commercial partition table recovery software: > just google for 'partition table recovery' don't sweat it. in addition to the 'partition table recovery', most commercial vendors do some sort of 'disk scan' recovery. software scans the disk bit-by-bit and trys to reassemble the filesystem ( onto an alternate drive ) on OSX, the full versions of tech tool and norton utilities will do that. on pc's, i've had luck with 'data recovery pro' or 'easy recovery pro'. can't remember the right name. if any of the open source tools work, awesome -- let me know . in any event, my 2 bits of advice are as follows: plan on time: the recovery process can take up to 2 days: as much as 1 day to find the files, and as much as 1 day to copy. sometimes you're not so lucky on the first recovery read, and need to try again. its taken me a week to recover some volumes. buy a new drive, and set whatever app to write to that and treat the data drive as read only. early windows apps would often screw that up and write onto the data you tried to recover. having a new spare drive around is always good anyways. you can have a dupe of your files should this happen again. From dlavigne6 at sympatico.ca Sun Sep 3 09:15:33 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Sun, 3 Sep 2006 09:15:33 -0400 (EDT) Subject: [nycbug-talk] Fw: Recovering partition table and ntfs filesystem (intact) (on and off-topic) In-Reply-To: <675034872-1157252814-cardhu_blackberry.rim.net-1433514753-@bxe029-cell01.bisx.prod.on.blackberry> References: <675034872-1157252814-cardhu_blackberry.rim.net-1433514753-@bxe029-cell01.bisx.prod.on.blackberry> Message-ID: <20060903091418.T624@dru.domain.org> On Sun, 3 Sep 2006, Trish Lynch wrote: > Sent to wrong address.... > ------Original Message------ > To: talk at nycbug.org > Sent: Sep 2, 2006 10:45 PM > Subject: Recovering partition table and ntfs filesystem (intact) (on and off-topic) > > I accidentally deleted the partition table of the wrong drive that contains 15,000 dollars worth of encoded cds that have been lost in moves or opacked in boxes and such and would take more then that type of investment to re-encode. > I lost the backup of it last week in a machine crash, and I was attempting to rebuild it when I deleted the wrong disk partition table (I just had a back surgery, teach me to dfo stuff on drugs) > > Is there any way to recover the partition table and the intact ntfs partition on there? Or do I have to send it out to be recovered? (Which is more than I can afford right now, I cannot believe I lost all my music *grrrrr*)) If this was UFS (I'm assuming it is NTFS....) this is a lifesaver: http://www.freebsd.org/cgi/url.cgi?ports/sysutils/scan_ffs/pkg-descr Not sure if this one will be useful in your case: http://www.freshports.org/sysutils/magicrescue/ Dru From o_sleep at belovedarctos.com Sun Sep 3 11:02:39 2006 From: o_sleep at belovedarctos.com (Bjorn Nelson) Date: Sun, 3 Sep 2006 11:02:39 -0400 Subject: [nycbug-talk] dragonflybsd: process sharing/virtual kernels Message-ID: <3E2E7AD4-2261-47D4-86A7-E6C152693E26@belovedarctos.com> All, I was wondering what people think about being able to cluster at the OS level. Matthew Dillon is proposing virtual kernels with caching as an easier alternative to their goal of process sharing between machines: http://www.shiningsilence.com/dbsdlog/index.php/2006/09/02/1853.html The end result will be that you can have two or more machines operate as effectively one operating system. Virtual kernels seem pretty similar to the VM stuff that has been heating up lately with vmware and xen. The main difference is that to provide the clustering ability, you need to add a component for data synchronization. Could this be built on top of the xen work being added to freebsd? Possibly using gated and carp together to take care of delegating the network/filesystem resources. Is this basically vmotion from vmware? Anyone have any experiences in this area? This looks like this is going to be a show stopping feature in the future when choosing an operating system for general serving purposes. I can just imagine many of my fears of hardware redundancy evaporating when we get to the point of having failover at the OS level. The benefit of this is basically what Google has realized with their cluster of cheap computers. You don't have to worry about redundancy at the host level nearly as much because a host is no longer a single point of failure, and you don't have to worry about accurately predicting the hardware required for your application as you can just add another host to the pool if it's not fast enough. Now, it's easy to see this and say it but as with all issues it's rarely black and white. You may still want to mirror your OS drives, to lessen the effect of the higher rate of failure of disks, and you may still want to do some homework for purchasing hardware as at a certain point you may have realized that you should have started with a faster base system as a building block (decreasing returns due to increase in overhead per performance of adding another machine, then again can this negated by "weighting" the machines so that faster machines serve more?). What other implications are here? Will sans be obsoleted? -Bjorn From trish at bsdunix.net Sun Sep 3 12:52:01 2006 From: trish at bsdunix.net (Trish Lynch) Date: Sun, 3 Sep 2006 16:52:01 +0000 Subject: [nycbug-talk] Fw: Fw: Recovering partition table and ntfs filesystem(intact) (on and off-topic) Message-ID: <618237190-1157302329-cardhu_blackberry.rim.net-725097639-@bxe056-cell01.bisx.prod.on.blackberry> After totally panicking, and crying, and realizing it wasn'tt just commercial mp3's but the only copy I have of my own album among others (masters, etc), I found a nifty tool for 40 bucks called 'partition table doctor' which rebuilt my partition table first time around, and even found ones 3 and 4 generations old! Scary. What sucks is that I had a backup of all the until the drive I was rebuilding crashed, and then I ended up deleting the partition table off the USB drive instead *sigh*. I blame it on the double narcotics I was on because of the epidural, and from now on before re-install, I'll unplug the USB drive Too bad I didn't find an open source tool I trusted immediately, but the data was too important :( -Trish ------Original Message------ From: Dru To: Trish Lynch Cc: talk at lists.nycbug.org Sent: Sep 3, 2006 9:15 AM Subject: Re: [nycbug-talk] Fw: Recovering partition table and ntfs filesystem(intact) (on and off-topic) On Sun, 3 Sep 2006, Trish Lynch wrote: > Sent to wrong address.... > ------Original Message------ > To: talk at nycbug.org > Sent: Sep 2, 2006 10:45 PM > Subject: Recovering partition table and ntfs filesystem (intact) (on and off-topic) > > I accidentally deleted the partition table of the wrong drive that contains 15,000 dollars worth of encoded cds that have been lost in moves or opacked in boxes and such and would take more then that type of investment to re-encode. > I lost the backup of it last week in a machine crash, and I was attempting to rebuild it when I deleted the wrong disk partition table (I just had a back surgery, teach me to dfo stuff on drugs) > > Is there any way to recover the partition table and the intact ntfs partition on there? Or do I have to send it out to be recovered? (Which is more than I can afford right now, I cannot believe I lost all my music *grrrrr*)) If this was UFS (I'm assuming it is NTFS....) this is a lifesaver: http://www.freebsd.org/cgi/url.cgi?ports/sysutils/scan_ffs/pkg-descr Not sure if this one will be useful in your case: http://www.freshports.org/sysutils/magicrescue/ Dru -- Trish Lynch From joshmccormack at travelersdiary.com Sun Sep 3 23:06:01 2006 From: joshmccormack at travelersdiary.com (Josh McCormack) Date: Sun, 3 Sep 2006 23:06:01 -0400 Subject: [nycbug-talk] dragonflybsd: process sharing/virtual kernels In-Reply-To: <3E2E7AD4-2261-47D4-86A7-E6C152693E26@belovedarctos.com> References: <3E2E7AD4-2261-47D4-86A7-E6C152693E26@belovedarctos.com> Message-ID: On 9/3/06, Bjorn Nelson wrote: > All, > > I was wondering what people think about being able to cluster at the > OS level. Matthew Dillon is proposing virtual kernels with caching > as an easier alternative to their goal of process sharing between > machines: > http://www.shiningsilence.com/dbsdlog/index.php/2006/09/02/1853.html > > The end result will be that you can have two or more machines operate > as effectively one operating system. I don't feel like I understand this as fully as I'd like. Part of this is from lack of the knowledge needed to understand, another might be from lack of much info on this OS on the Internet. Will it be possible at some point with DragonFly, if this works out, to do this hot? I keep thinking how awesome it would be to be able to have a laptop/palmtop and be able to plug it in to a well powered desktop and have the laptop be able to use those resources, or even possibly have this happen over a network. Josh From lists at stringsutils.com Sun Sep 3 23:13:30 2006 From: lists at stringsutils.com (Francisco Reyes) Date: Sun, 03 Sep 2006 23:13:30 -0400 Subject: [nycbug-talk] postgres incremental backups? References: <19008.160.33.20.11.1156458401.squirrel@webmail.nomadlogic.org> <6a36e7290608241555s1eb01c2bh86bf7f2e84a0d13f@mail.gmail.com> Message-ID: Charles Sprickman writes: >> [1] http://slony.info/ > > Just out of curiousity, is Slony the de-facto replication solution for > Postgres? If you consider Free only.. I believe Slony is. There is also pgcluster, but it's a different beast.. basically it is a que/caching system which can also do dual write to 2 DBs (don't know if more). Commercial there is the commandprompt replication, which is what I am going to try in the near future. Seems simpler to use, install and maintain, but it is not free. It has a licence fee and a maintenance fee. Depending on the budget for a project it may be worth looking into. For us, it is the ease (or at least I hope it will be) or use and maintenance, coupled with having someone to call what made us go with command prompt. I started my research about 3 months ago for replication. When we decided to go with commandprompt they told us they had a new version coming up in August and that the recommended we waited for it. I believe this new version is out.. so I wrote to them to check if we can proceed. Anyone interested on hearing our experience send me a note and I will make a point to write back.. or there is enough interest will write back to the list. I figure it will be a few weeks before we get it and use it. One big, potential, issue I did see with command prompt replication though was that it needed to store an entire copy of what you are replicating somewhere. I don't believe it has to be in the same as the machines replicating, but still if you have a very large setup this extra copy can be a problem. From lists at stringsutils.com Sun Sep 3 22:52:38 2006 From: lists at stringsutils.com (Francisco Reyes) Date: Sun, 03 Sep 2006 22:52:38 -0400 Subject: [nycbug-talk] postgres incremental backups? References: <19008.160.33.20.11.1156458401.squirrel@webmail.nomadlogic.org> <6a36e7290608241555s1eb01c2bh86bf7f2e84a0d13f@mail.gmail.com> <5962.160.33.20.11.1156461599.squirrel@webmail.nomadlogic.org> Message-ID: Peter Wright writes: > i did not have first hand experience with it (and the dev. left who was > running into the problem) but from what I understood was that we were > having bad transaction log's fill our cluster volume pretty quickly. he > submitted a bug (BUG #2104: pg_xlog/ trace files not reclaimed by server). > to tell you the truth - i am not sure if the bug is a side effect of his > code/app or if it is an issue with postgres.... pg_xlog.. isn't that WALL? if so .. they won't be reclaimed. I believe you just set how many you will have. What did you get from the postgresql lists? They are extremely usefull and friendly in those lists. From lists at stringsutils.com Sun Sep 3 22:59:58 2006 From: lists at stringsutils.com (Francisco Reyes) Date: Sun, 03 Sep 2006 22:59:58 -0400 Subject: [nycbug-talk] postgres incremental backups? References: <19008.160.33.20.11.1156458401.squirrel@webmail.nomadlogic.org> <6a36e7290608241555s1eb01c2bh86bf7f2e84a0d13f@mail.gmail.com> <5962.160.33.20.11.1156461599.squirrel@webmail.nomadlogic.org> <6a36e7290608241642m3ae2cfc9k8a87aa73c1810d4f@mail.gmail.com> Message-ID: Bob Ippolito writes: > The commercial bizgres is expensive but sounds like it would be good > for really big databases because it scales over a cluster I can't wait until the company I work for grows big enough so we can afford Bizgres MPP (The cluster version). It really sounds like it will handle tons of data efficiently. Right now for our, soon to be 500GB+ DB, we got a 16 drive 3U machine with 8GB of RAM and 2 raid controllers. Anything indexed is great, but once you hit a table scan it's another story.. fortunately rarely a table scan gets triggered. I am also looking into postgresql cluster feature. That will be a real performance booster. From lists at stringsutils.com Sun Sep 3 22:53:53 2006 From: lists at stringsutils.com (Francisco Reyes) Date: Sun, 03 Sep 2006 22:53:53 -0400 Subject: [nycbug-talk] postgres incremental backups? References: Message-ID: alex at pilosoft.com writes: > 7.4 at this point might be as well called prehistoric. Upgrade. If anything.. I had issues with 7.X that were solved by 8.X In particular, if I recall, there some serious issues with index and table bloat... and this was when I was dealing with much, much smaller DBs. From lists at stringsutils.com Sun Sep 3 23:04:19 2006 From: lists at stringsutils.com (Francisco Reyes) Date: Sun, 03 Sep 2006 23:04:19 -0400 Subject: [nycbug-talk] postgres incremental backups? References: <19008.160.33.20.11.1156458401.squirrel@webmail.nomadlogic.org> <6a36e7290608241555s1eb01c2bh86bf7f2e84a0d13f@mail.gmail.com> <5962.160.33.20.11.1156461599.squirrel@webmail.nomadlogic.org> <6a36e7290608241642m3ae2cfc9k8a87aa73c1810d4f@mail.gmail.com> <13577.160.33.20.11.1156463766.squirrel@webmail.nomadlogic.org> Message-ID: Peter Wright writes: > getting his code to play nice with 8.x. i'm sure he was vacuuming/etc as > we talked about that Did he ever check the vacuum worked. :-) It is possible to vacuum and to have it fail. If you go over lots of transactions and you have default seetings.. there is a resourse (FSM pages) which needs to be increased. > so for lack of further evidence i'm guessing his > code was doing something nasty (at the time of the bug report it was > causing a deadlock for example). What language was the program? I have been using postgresq for something like 5+ years.. and never had any problems until I tried a program called dspam (writen in c). That program caused me more database problems (including corrupting) in a few weeks than I had with Postgresql in all the time I have been using it. When I wrote to the dspam list they basically replied it was postgresql's fault.. Possible, but I was not about to keep trashing my DB to figure out why this one program caused so many problems. From lists at stringsutils.com Sun Sep 3 22:50:13 2006 From: lists at stringsutils.com (Francisco Reyes) Date: Sun, 03 Sep 2006 22:50:13 -0400 Subject: [nycbug-talk] postgres incremental backups? References: <19008.160.33.20.11.1156458401.squirrel@webmail.nomadlogic.org> <6a36e7290608241555s1eb01c2bh86bf7f2e84a0d13f@mail.gmail.com> Message-ID: Bob Ippolito writes: > The only thing I can think of is Slony-I [1]. It should cover all of > your needs and give you a safe online way to replicate your cluster to > some future 8.1.x or 8.2 when the time comes. Were I work we are going to go with the replication from Comamndprompt. Slony seems too cumbersome to setup. Command prompt repliation, however, is not free. >From what I have researched so far it does seem like it will be significantly easier to setup and maintain.. but I will know for sure after we get it. Our setup will likely be on the 500GB+ range so replication is a must. From pete at nomadlogic.org Tue Sep 5 12:11:50 2006 From: pete at nomadlogic.org (Pete Wright) Date: Tue, 5 Sep 2006 12:11:50 -0400 Subject: [nycbug-talk] dragonflybsd: process sharing/virtual kernels In-Reply-To: <3E2E7AD4-2261-47D4-86A7-E6C152693E26@belovedarctos.com> References: <3E2E7AD4-2261-47D4-86A7-E6C152693E26@belovedarctos.com> Message-ID: <20060905161150.GC29066@sunset.nomadlogic.org> On Sun, Sep 03, 2006 at 11:02:39AM -0400, Bjorn Nelson wrote: > All, > > I was wondering what people think about being able to cluster at the > OS level. Matthew Dillon is proposing virtual kernels with caching > as an easier alternative to their goal of process sharing between > machines: > http://www.shiningsilence.com/dbsdlog/index.php/2006/09/02/1853.html > > The end result will be that you can have two or more machines operate > as effectively one operating system. Virtual kernels seem pretty > similar to the VM stuff that has been heating up lately with vmware > and xen. The main difference is that to provide the clustering > ability, you need to add a component for data synchronization. Could > this be built on top of the xen work being added to freebsd? > Possibly using gated and carp together to take care of delegating the > network/filesystem resources. > > Is this basically vmotion from vmware? Anyone have any experiences > in this area? > this all seems pretty close the UML (user mode linux) which may or may not be a good thing. vmotion seems pretty interesting on the surface, but there are still alot of concerns I personally have with it (cough master station only run's on XP cough). i'm actually doing some testing with vmotion now...so hopefully i'll have better input in a month or so... > This looks like this is going to be a show stopping feature in the > future when choosing an operating system for general serving > purposes. I can just imagine many of my fears of hardware redundancy > evaporating when we get to the point of having failover at the OS level. > maybe, but you still will need redundant hardware regarless...think IBM (or sun for that matter) mainframes...just because you can move your os instance around does not mean you will just adandon the hardware right, and i'd rather hotswap bad parts when possible than messing around migrating production applications on the fly... > The benefit of this is basically what Google has realized with their > cluster of cheap computers. You don't have to worry about redundancy > at the host level nearly as much because a host is no longer a single > point of failure, and you don't have to worry about accurately > predicting the hardware required for your application as you can just > add another host to the pool if it's not fast enough. Now, it's easy > to see this and say it but as with all issues it's rarely black and > white. You may still want to mirror your OS drives, to lessen the > effect of the higher rate of failure of disks, and you may still want > to do some homework for purchasing hardware as at a certain point you > may have realized that you should have started with a faster base > system as a building block (decreasing returns due to increase in > overhead per performance of adding another machine, then again can > this negated by "weighting" the machines so that faster machines > serve more?). > exactly, we actually don't even bother mirroring disks...it's cheaper (and quicker) to just swap out a whole unit in the case of a hardware failure...or just rebuild the OS (via xcat www.xcat.org). in cases like this a VM would just hinder the performance of the cluster/renderfarm/etc. although i'd say HPC clusters or whatever fill a different niche than where VM's are trying to address. > What other implications are here? Will sans be obsoleted? > my take on the whole VM craze lately is this: it's great if you have many app's that need custom or isolated execution environments - *only* if said app's do not require alot of system resources. it seems to me that VM environment works great in a place like say a bank that has fairly predictable usage patterns...well the application that is going to total today's reciepts will need alot of system resources at 4:30pm...and the one that does all wire transfers will need alot of system resources at 2:00am. so there is no reason why these two applications can't share hardware. but, if you have an unpredictable usage pattern (or are constantly using <%90 of your system resources) then I frankly don't think you will gain much from a VM. will, san's be obsoleted? heh...funny you mention that. we are actually doing RnD on SAN's for some VM stuff we are doing. so yea, my best guess is that SAN's will become more important as the use of VM's grow's. -p -- ~~oO00Oo~~ Peter Wright pete at nomadlogic.org www.nomadlogic.org/~pete 310.869.9459 From spork at bway.net Tue Sep 5 20:15:03 2006 From: spork at bway.net (Charles Sprickman) Date: Tue, 5 Sep 2006 20:15:03 -0400 (EDT) Subject: [nycbug-talk] lmsensors equiv. for *BSD? Message-ID: Hi all, I've been looking around and haven't seen anything, but I thought I'd check here... Anyone seen anything like lmsensors for *BSD? I've found healthd (spotty on most of my hardware), x/mbmon (generally works), lmmon (really spotty). While mbmon is mostly working out, what this and the other packages lack is any integration with net-snmp. I already gather stats and monitor most server info via snmp which is very handy. I noticed the lmsensors+net-snmp packages on linux make all the system health info available by default via snmp. Any thoughts? I suppose I could have snmpd fire off a script when certain mibs are tickled, but I loathe reinventing the wheel. I thought I saw some talk long ago about a FreeBSD port of lmsensors, but could find no further mention of that. I have to imagine most large scale users like the Yahoo! folks are doing something... who would run a huge farm of BSD boxes and not monitor something like a $10 CPU fan that could bring a whole box down? Thanks, Charles From nycbug at cyth.net Tue Sep 5 20:24:02 2006 From: nycbug at cyth.net (Ray Lai) Date: Tue, 5 Sep 2006 20:24:02 -0400 Subject: [nycbug-talk] lmsensors equiv. for *BSD? In-Reply-To: References: Message-ID: <20060906002425.GG31505@cybertron.cyth.net> On Tue, Sep 05, 2006 at 08:15:03PM -0400, Charles Sprickman wrote: > Anyone seen anything like lmsensors for *BSD? I've found healthd (spotty > on most of my hardware), x/mbmon (generally works), lmmon (really spotty). http://www.openbsd.org/cgi-bin/man.cgi?query=sensorsd -Ray- From mspitzer at gmail.com Wed Sep 6 22:59:47 2006 From: mspitzer at gmail.com (Marc Spitzer) Date: Wed, 6 Sep 2006 22:59:47 -0400 Subject: [nycbug-talk] open BSM has been ported to freebsd 6.1 Message-ID: <8c50a3c30609061959m4ca182e2i7008c7110e6c30a9@mail.gmail.com> I just cvsuped and it was there, whoo hoo or oh krap you decied. marc ps Ike go play -- "We trained very hard, but it seemed that every time we were beginning to form into teams we would be reorganized. I was to learn later in life that we tend to meet any new situation by reorganizing, and a wonderful method it can be for creating the illusion of progress, while producing confusion, inefficiency and demoralization." -Gaius Petronius, 1st Century AD From dlavigne6 at sympatico.ca Thu Sep 7 10:57:27 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Thu, 7 Sep 2006 10:57:27 -0400 (EDT) Subject: [nycbug-talk] open BSM has been ported to freebsd 6.1 In-Reply-To: <8c50a3c30609061959m4ca182e2i7008c7110e6c30a9@mail.gmail.com> References: <8c50a3c30609061959m4ca182e2i7008c7110e6c30a9@mail.gmail.com> Message-ID: <20060907105620.V625@dru.domain.org> On Wed, 6 Sep 2006, Marc Spitzer wrote: > I just cvsuped and it was there, whoo hoo or oh krap you decied. Cool. Looks like a major release is coming soon: http://www.trustedbsd.org/openbsm.html Til then: /usr/src/contrib/openbsm/README Dru From techneck at goldenpath.org Thu Sep 7 12:34:50 2006 From: techneck at goldenpath.org (Tim Allender) Date: Thu, 07 Sep 2006 12:34:50 -0400 Subject: [nycbug-talk] m0n0wall & pfSense meeting Message-ID: <45004A2A.9090501@goldenpath.org> We were so busy with features, forgot to mention: In Ike's presentation of pfSense, the WebConfigurator GUI was in a different format from the default. It looked more like the m0n0wall WebConfigurator with the vertical expanded options side panel, and not the pfSense WebConfigurator GUI's layout with horizontal categorical drop down lists. I don't see any stock method for changing the pfSense GUI layout. So, I presume that was a custom job, bringing in the m0n0wall GUI template. From george at galis.org Thu Sep 7 12:58:26 2006 From: george at galis.org (George Georgalis) Date: Thu, 7 Sep 2006 12:58:26 -0400 Subject: [nycbug-talk] getting txt files from tex Message-ID: <20060907165826.GA26324@run.galis.org> I've written some some tex files and makefiles to produce pdf and html output, but today I was asked for text output.... how is that done? // George -- George Georgalis, systems architect, administrator < From yusuke at cs.nyu.edu Thu Sep 7 13:09:51 2006 From: yusuke at cs.nyu.edu (Yusuke Shinyama) Date: Thu, 07 Sep 2006 13:09:51 -0400 Subject: [nycbug-talk] getting txt files from tex In-Reply-To: <20060907165826.GA26324@run.galis.org> References: <20060907165826.GA26324@run.galis.org> Message-ID: <20060907170951.8899.12560.yusuke@grape.cs.nyu.edu> "George Georgalis" wrote: > I've written some some tex files and makefiles to produce pdf and > html output, but today I was asked for text output.... how is that > done? lynx -dump file.html > file.txt Yusuke From pete at nomadlogic.org Thu Sep 7 13:15:59 2006 From: pete at nomadlogic.org (Peter Wright) Date: Thu, 7 Sep 2006 10:15:59 -0700 (PDT) Subject: [nycbug-talk] getting txt files from tex In-Reply-To: <20060907165826.GA26324@run.galis.org> References: <20060907165826.GA26324@run.galis.org> Message-ID: <16641.160.33.20.11.1157649359.squirrel@webmail.nomadlogic.org> > I've written some some tex files and makefiles to produce pdf and > html output, but today I was asked for text output.... how is that > done? > belive texi2html and texi2pdf should work, i know they are part of our stock Irix and RHELinux builds...not sure if it's part of the freebsd ports. --pete -- ~~oO00Oo~~ Peter Wright pete at nomadlogic.org www.nomadlogic.org/~pete 310.869.9459 From nycbug at chrisbuechler.com Thu Sep 7 13:39:09 2006 From: nycbug at chrisbuechler.com (Chris Buechler) Date: Thu, 07 Sep 2006 13:39:09 -0400 Subject: [nycbug-talk] m0n0wall & pfSense meeting In-Reply-To: <45004A2A.9090501@goldenpath.org> References: <45004A2A.9090501@goldenpath.org> Message-ID: <4500593D.6040506@chrisbuechler.com> Tim Allender wrote: > We were so busy with features, forgot to mention: > In Ike's presentation of pfSense, the WebConfigurator GUI was in a > different format from the default. > It looked more like the m0n0wall WebConfigurator with the vertical > expanded options side panel, > and not the pfSense WebConfigurator GUI's layout with horizontal > categorical drop down lists. > > I don't see any stock method for changing the pfSense GUI layout. System -> General Setup, Theme. There are 3 themes built in, that's one of them. cheers, -Chris From george at galis.org Thu Sep 7 14:02:35 2006 From: george at galis.org (George Georgalis) Date: Thu, 7 Sep 2006 14:02:35 -0400 Subject: [nycbug-talk] getting txt files from tex In-Reply-To: <16641.160.33.20.11.1157649359.squirrel@webmail.nomadlogic.org> References: <20060907165826.GA26324@run.galis.org> <16641.160.33.20.11.1157649359.squirrel@webmail.nomadlogic.org> Message-ID: <20060907180235.GB26324@run.galis.org> On Thu, Sep 07, 2006 at 10:15:59AM -0700, Peter Wright wrote: > >> I've written some some tex files and makefiles to produce pdf and >> html output, but today I was asked for text output.... how is that >> done? >> >belive texi2html and texi2pdf should work, i know they are part of our >stock Irix and RHELinux builds...not sure if it's part of the freebsd >ports. they don't seem to have text output options. Yusuke's suggestion: lynx -dump file.html > file.txt seems the best. but it does not generate 'finished' docs. The docbook standard for linux seems to have macros to generate text output (eg an option for HOWTOs) but that doesn't not seem a standard output for tex processors. a collection of macros: http://www.tex.ac.uk/cgi-bin/texfaq2html?label=toascii the txt sty can be used for rfc generation... but, I find it odd that there is no mature tex2txt program, eg something more than stripping out latex commands. // George -- George Georgalis, systems architect, administrator < From pete at nomadlogic.org Thu Sep 7 14:19:08 2006 From: pete at nomadlogic.org (Peter Wright) Date: Thu, 7 Sep 2006 11:19:08 -0700 (PDT) Subject: [nycbug-talk] getting txt files from tex In-Reply-To: <20060907180235.GB26324@run.galis.org> References: <20060907165826.GA26324@run.galis.org> <16641.160.33.20.11.1157649359.squirrel@webmail.nomadlogic.org> <20060907180235.GB26324@run.galis.org> Message-ID: <30410.160.33.20.11.1157653148.squirrel@webmail.nomadlogic.org> > On Thu, Sep 07, 2006 at 10:15:59AM -0700, Peter Wright wrote: >> >>> I've written some some tex files and makefiles to produce pdf and >>> html output, but today I was asked for text output.... how is that >>> done? >>> >>belive texi2html and texi2pdf should work, i know they are part of our >>stock Irix and RHELinux builds...not sure if it's part of the freebsd >>ports. > > they don't seem to have text output options. > Yusuke's suggestion: > > lynx -dump file.html > file.txt > > seems the best. but it does not generate 'finished' docs. > > The docbook standard for linux seems to have macros > to generate text output (eg an option for HOWTOs) > but that doesn't not seem a standard output for tex > processors. a collection of macros: > > http://www.tex.ac.uk/cgi-bin/texfaq2html?label=toascii > > the txt sty can be used for rfc generation... > > but, I find it odd that there is no mature tex2txt program, > eg something more than stripping out latex commands. > sure there is, i think it's called awk/sed/perl ;^) -pete -- ~~oO00Oo~~ Peter Wright pete at nomadlogic.org www.nomadlogic.org/~pete 310.869.9459 From nycbug-list at 2xlp.com Thu Sep 7 14:41:05 2006 From: nycbug-list at 2xlp.com (Jonathan Vanasco) Date: Thu, 7 Sep 2006 14:41:05 -0400 Subject: [nycbug-talk] postgres memory question Message-ID: the other day i noticed what might be a memory leak in PG on my FreeBSD 6.0 REL box ( it could be in Apache/ModPerl/DBI , but some bad command line tests point to pg. I'm hoping someone here can help me figure out what is at blame ) someone else running modperl with pg on RHEL posted a problem like this a few weeks ago, so I think there might be more to it... anyways, here's the issue: i start my box up, ONLY running apache/pg ( nothing else is on ), and I have ~900 MB free. Perfect. I make some requests, Apache and PG consume memory. Expected. I restart Apache, it releases memory and the associated pg_client . PG still holds shared memory. Expected. I start stressing Apache ( 10k-50k requests )- apache works great. not visible memory issues. MaxChildRequests keeps size manageble. I stop apache, it releases memory and the associated pg_client memory. PG still holds shared memory. Expected. I stop the postmaster daemon. PG releases the lock on shared memory ( ipcs shows it ). I expect to see 900 MB free. Instead i see 700. When I noticed this first, I was swapping. The first person who experienced something like this was swapping like mad as well. I run this again a few times, I inch closer and closer to 0. Except its hard to get the free mem down to 0 - it takes a while. I'm hoping some people here might be able to suggest the following: a- A pure PG way to max out shared memory in clients, and restart, so I can tell if this behavior is from PG or Apache/ModPerl. I *think* its from PG, as start/stop a few pg clients and then stop/ start the daemon i see a decrease in memory. I was trying to think of some sort of recursive select or exponential algorithm. I couldn't find anything on google though. b- A way to better examine free memory. My knowledge is limited to top / and a little bit of libgtop. I know i was swapping because top reported it as such. But I've been told that top doesn't always report truly 'free' memory-- once that hits 0 it looks in buffers/ cached. So maybe the pg memory is there? I dunno. If anyone has a clue, I'd be appreciative. This isn't an issue as long as I don't stop/start postgres. From george at galis.org Thu Sep 7 15:18:22 2006 From: george at galis.org (George Georgalis) Date: Thu, 7 Sep 2006 15:18:22 -0400 Subject: [nycbug-talk] getting txt files from tex In-Reply-To: <30410.160.33.20.11.1157653148.squirrel@webmail.nomadlogic.org> References: <20060907165826.GA26324@run.galis.org> <16641.160.33.20.11.1157649359.squirrel@webmail.nomadlogic.org> <20060907180235.GB26324@run.galis.org> <30410.160.33.20.11.1157653148.squirrel@webmail.nomadlogic.org> Message-ID: <20060907191822.GC26324@run.galis.org> On Thu, Sep 07, 2006 at 11:19:08AM -0700, Peter Wright wrote: > >> On Thu, Sep 07, 2006 at 10:15:59AM -0700, Peter Wright wrote: >>> >>>> I've written some some tex files and makefiles to produce pdf and >>>> html output, but today I was asked for text output.... how is that >>>> done? >>>> >>>belive texi2html and texi2pdf should work, i know they are part of our >>>stock Irix and RHELinux builds...not sure if it's part of the freebsd >>>ports. >> >> they don't seem to have text output options. >> Yusuke's suggestion: >> >> lynx -dump file.html > file.txt >> >> seems the best. but it does not generate 'finished' docs. >> >> The docbook standard for linux seems to have macros >> to generate text output (eg an option for HOWTOs) >> but that doesn't not seem a standard output for tex >> processors. a collection of macros: >> >> http://www.tex.ac.uk/cgi-bin/texfaq2html?label=toascii >> >> the txt sty can be used for rfc generation... >> >> but, I find it odd that there is no mature tex2txt program, >> eg something more than stripping out latex commands. >> > >sure there is, i think it's called awk/sed/perl ;^) That's pretty funny, Pete... like the whole world is just waiting for full justification. // George -- George Georgalis, systems architect, administrator < From mspitzer at gmail.com Thu Sep 7 17:56:32 2006 From: mspitzer at gmail.com (Marc Spitzer) Date: Thu, 7 Sep 2006 17:56:32 -0400 Subject: [nycbug-talk] getting txt files from tex In-Reply-To: <20060907180235.GB26324@run.galis.org> References: <20060907165826.GA26324@run.galis.org> <16641.160.33.20.11.1157649359.squirrel@webmail.nomadlogic.org> <20060907180235.GB26324@run.galis.org> Message-ID: <8c50a3c30609071456m59e48deflb54ad3af3cef0c31@mail.gmail.com> Can you get tex to spit out troff/nroff and then dump to text? marc -- "We trained very hard, but it seemed that every time we were beginning to form into teams we would be reorganized. I was to learn later in life that we tend to meet any new situation by reorganizing, and a wonderful method it can be for creating the illusion of progress, while producing confusion, inefficiency and demoralization." -Gaius Petronius, 1st Century AD From ike at lesmuug.org Fri Sep 8 00:37:20 2006 From: ike at lesmuug.org (Isaac Levy) Date: Fri, 8 Sep 2006 00:37:20 -0400 Subject: [nycbug-talk] m0n0wall & pfSense meeting In-Reply-To: <4500593D.6040506@chrisbuechler.com> References: <45004A2A.9090501@goldenpath.org> <4500593D.6040506@chrisbuechler.com> Message-ID: <36154B1F-8FBC-4B08-B2D0-7ED64481A5DF@lesmuug.org> On Sep 7, 2006, at 1:39 PM, Chris Buechler wrote: > Tim Allender wrote: >> We were so busy with features, forgot to mention: >> In Ike's presentation of pfSense, the WebConfigurator GUI was in a >> different format from the default. >> It looked more like the m0n0wall WebConfigurator with the vertical >> expanded options side panel, >> and not the pfSense WebConfigurator GUI's layout with horizontal >> categorical drop down lists. >> >> I don't see any stock method for changing the pfSense GUI layout. > > System -> General Setup, Theme. There are 3 themes built in, > that's one > of them. > > cheers, > -Chris Thanks Chris, and Tim, my apologies for forgetting that :) I maintain both m0n0wall and pfSense in production, and it makes me really happy to have everything sortof in the same layout- makes my brain do less work... :) I'm really happy the PFSense team put the themes in there! Rocket- .ike From okan at demirmen.com Fri Sep 8 09:23:44 2006 From: okan at demirmen.com (Okan Demirmen) Date: Fri, 8 Sep 2006 09:23:44 -0400 Subject: [nycbug-talk] dragonflybsd: process sharing/virtual kernels In-Reply-To: <3E2E7AD4-2261-47D4-86A7-E6C152693E26@belovedarctos.com> References: <3E2E7AD4-2261-47D4-86A7-E6C152693E26@belovedarctos.com> Message-ID: <20060908132344.GO29883@clam.khaoz.org> On Sun 2006.09.03 at 11:02 -0400, Bjorn Nelson wrote: > All, > > I was wondering what people think about being able to cluster at the > OS level. Matthew Dillon is proposing virtual kernels with caching > as an easier alternative to their goal of process sharing between > machines: > http://www.shiningsilence.com/dbsdlog/index.php/2006/09/02/1853.html > > The end result will be that you can have two or more machines operate > as effectively one operating system. Virtual kernels seem pretty > similar to the VM stuff that has been heating up lately with vmware > and xen. The main difference is that to provide the clustering > ability, you need to add a component for data synchronization. Could > this be built on top of the xen work being added to freebsd? > Possibly using gated and carp together to take care of delegating the > network/filesystem resources. > > Is this basically vmotion from vmware? Anyone have any experiences > in this area? Not really vmotion. vmotion allows one to move a vm instance in realtime to another node in the vm pool. It works well, though it is limited to windows. However, vmware's DRS (distributed resource scheduler), is closer to what you are refering to, which I don't have access to. > This looks like this is going to be a show stopping feature in the > future when choosing an operating system for general serving > purposes. I can just imagine many of my fears of hardware redundancy > evaporating when we get to the point of having failover at the OS level. > > The benefit of this is basically what Google has realized with their > cluster of cheap computers. You don't have to worry about redundancy > at the host level nearly as much because a host is no longer a single > point of failure, and you don't have to worry about accurately > predicting the hardware required for your application as you can just > add another host to the pool if it's not fast enough. Now, it's easy > to see this and say it but as with all issues it's rarely black and > white. You may still want to mirror your OS drives, to lessen the > effect of the higher rate of failure of disks, and you may still want > to do some homework for purchasing hardware as at a certain point you > may have realized that you should have started with a faster base > system as a building block (decreasing returns due to increase in > overhead per performance of adding another machine, then again can > this negated by "weighting" the machines so that faster machines > serve more?). > > What other implications are here? Will sans be obsoleted? sans obsoleted? i don't think so - i think they become more important. From njt at ayvali.org Fri Sep 8 09:31:42 2006 From: njt at ayvali.org (N.J. Thomas) Date: Fri, 8 Sep 2006 09:31:42 -0400 Subject: [nycbug-talk] getting txt files from tex In-Reply-To: <20060907165826.GA26324@run.galis.org> References: <20060907165826.GA26324@run.galis.org> Message-ID: <20060908133142.GA14752@ayvali.org> * George Georgalis [2006-09-07 12:58:26 -0400]: > I've written some some tex files and makefiles to produce pdf and html > output, but today I was asked for text output.... how is that done? In the past, I did this by producing PS with TeX/LaTeX and then using ps2ascii to convert the PostScript to text. I didn't do this too often however....as I remember ps2ascii losing some information, especially with tabular data. Thomas -- N.J. Thomas njt at ayvali.org Etiamsi occiderit me, in ipso sperabo From nikolai at fetissov.org Fri Sep 8 10:08:50 2006 From: nikolai at fetissov.org (nikolai) Date: Fri, 8 Sep 2006 10:08:50 -0400 (EDT) Subject: [nycbug-talk] September 2006 meeting audio. Message-ID: <13862.63.66.6.15.1157724530.squirrel@www.geekisp.com> Folks, [part of] recording of Ike's presentation is up at http://www.fetissov.org/public/nycbug/ My fault - didn't remind Ike to hit that big shiny button right before he started, so about 20 minutes in the beginning are lost forever. Cheers, -- nikolai From dlavigne6 at sympatico.ca Fri Sep 8 10:37:56 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Fri, 8 Sep 2006 10:37:56 -0400 (EDT) Subject: [nycbug-talk] pf book Message-ID: <20060908103452.M624@dru.domain.org> My copy of Jeremy Reed's pf book has arrived: http://www.reedmedia.net/books/pf-book/ I'll definitely be using it in the classroom and keeping a copy near my administrative workstation. Very clear explanations, diagrams, and working examples for taking advantage of pf's features. Gurus won't find anything new but the rest of us will get "aha" moments on the features we're foggy on. Cheers, Dru From spork at bway.net Fri Sep 8 14:28:08 2006 From: spork at bway.net (Charles Sprickman) Date: Fri, 8 Sep 2006 14:28:08 -0400 (EDT) Subject: [nycbug-talk] pf book In-Reply-To: <20060908103452.M624@dru.domain.org> References: <20060908103452.M624@dru.domain.org> Message-ID: On Fri, 8 Sep 2006, Dru wrote: > My copy of Jeremy Reed's pf book has arrived: > > http://www.reedmedia.net/books/pf-book/ > > I'll definitely be using it in the classroom and keeping a copy near my > administrative workstation. Very clear explanations, diagrams, and working > examples for taking advantage of pf's features. Gurus won't find anything > new but the rest of us will get "aha" moments on the features we're foggy > on. Did you find tags and QoS to be well-explained? I'm OK with basic stuff but I totally rely on pfSense when I start playing with QoS and the like. And I can barely follow the complex configs pfSense generates... Thanks, Charles > Cheers, > > Dru > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > From ike at lesmuug.org Sat Sep 9 14:11:27 2006 From: ike at lesmuug.org (Isaac Levy) Date: Sat, 9 Sep 2006 14:11:27 -0400 Subject: [nycbug-talk] September 2006 meeting audio. In-Reply-To: <13862.63.66.6.15.1157724530.squirrel@www.geekisp.com> References: <13862.63.66.6.15.1157724530.squirrel@www.geekisp.com> Message-ID: Hi Nikolai, On Sep 8, 2006, at 10:08 AM, nikolai wrote: > Folks, > [part of] recording of Ike's presentation > is up at http://www.fetissov.org/public/nycbug/ As always, Thanks- you rock. :) > My fault - didn't remind Ike to hit that > big shiny button right before he started, > so about 20 minutes in the beginning are > lost forever. Actually, that was *MY* fault entirely. Sorry all. Best, .ike From ike at lesmuug.org Sat Sep 9 16:15:36 2006 From: ike at lesmuug.org (Isaac Levy) Date: Sat, 9 Sep 2006 16:15:36 -0400 Subject: [nycbug-talk] ipfw, ipf, pf comparison matrix In-Reply-To: <20060908174233.GF25206@cybertron.cyth.net> References: <20060908174233.GF25206@cybertron.cyth.net> Message-ID: Hi All, On Sep 8, 2006, at 1:42 PM, Ray Lai wrote: > You promised us a link to slides comparing ipfw and pf! Can you > send it > to the list? Thanks! Ray reminded me to post the comparison of ipfw, ipf, and pf to the talk list- here it is, in ASCII. Again, with all the love and buzz over PF, it seems clear below why IPFW is still the 'stock' packet filter in FreeBSD- many esoteric low- level features, but no packet filter has it all... Rocket, .ike ######################################################################## #### BSD Firewalling Options - comparing IPFW, IPFILTER, and PF - List originally compiled for BSDCAN 2006, by Scott Ullrich and Chris Buechler Original Lecture Slides: http://pfsense.org/bsdcan/ FEATURE IPFW IPFILTER PF QUEUE DUMMYNET * * QUEUE ALTQ * * SKIPTO * * RULESETS * CONNECTION FORWARDING * * * IPTOS * IPTTL * IPPOS * IPVERSION * LAYER2 MATCHING * MAC ADDRESS FILTERING * TABLES * PROBABILITY (PROB) * COUNT * TEE * * * ?ME? SUPPORT * * IPV6 * JAIL * IPSEC * IPTOS - LOW DELAY * * * IPTOS - THROUGHPUT * * * IPTOS - RELIABILITY * * * IPTOS - MINCOST * * IPTOS - CONGESTION * * * UID * VERREVPATH * QUICK * * KEEP STATE * * * MODULATE STATE * SYNPROXY STATE * OVERLOAD SUPPORT * FINGERPRINT SCANNING * LIMIT STATES PER RULE * PF http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls- pf.html IPFilter http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls- ipf.html IPFW http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls- ipfw.html ######################################################################## #### From ike at lesmuug.org Sat Sep 9 16:58:44 2006 From: ike at lesmuug.org (Isaac Levy) Date: Sat, 9 Sep 2006 16:58:44 -0400 Subject: [nycbug-talk] ipfw, ipf, pf comparison matrix In-Reply-To: References: <20060908174233.GF25206@cybertron.cyth.net> Message-ID: Hi All, On Sep 9, 2006, at 4:15 PM, Isaac Levy wrote: > Ray reminded me to post the comparison of ipfw, ipf, and pf to the > talk list- here it is, in ASCII. So I've gotten some polite offlist replies to this already, basically, I've been told the list is fairly out of date, especially where PF is concerned. With that, I'm no packet-filter guru, I'd totally love it if folks on list would double-check the features, and re-post it! Best, .ike From george at sddi.net Sat Sep 9 17:06:48 2006 From: george at sddi.net (George R.) Date: Sat, 09 Sep 2006 17:06:48 -0400 Subject: [nycbug-talk] ipfw, ipf, pf comparison matrix In-Reply-To: References: <20060908174233.GF25206@cybertron.cyth.net> Message-ID: <45032CE8.7070701@sddi.net> Isaac Levy wrote: > Hi All, > > On Sep 9, 2006, at 4:15 PM, Isaac Levy wrote: > >> Ray reminded me to post the comparison of ipfw, ipf, and pf to the >> talk list- here it is, in ASCII. > > So I've gotten some polite offlist replies to this already, > basically, I've been told the list is fairly out of date, especially > where PF is concerned. > > With that, I'm no packet-filter guru, I'd totally love it if folks on > list would double-check the features, and re-post it! Come on Ike, don't give us that. . . *You* aren't a guru to review this? ;-' BTW, has anyone used PF on the master jail in FBSD to filter for the jails? I know ipfw is the standard way to do packet-filtering with jails. . . g From ike at lesmuug.org Sat Sep 9 17:23:08 2006 From: ike at lesmuug.org (Isaac Levy) Date: Sat, 9 Sep 2006 17:23:08 -0400 Subject: [nycbug-talk] ipfw, ipf, pf comparison matrix In-Reply-To: <45032CE8.7070701@sddi.net> References: <20060908174233.GF25206@cybertron.cyth.net> <45032CE8.7070701@sddi.net> Message-ID: Hi George, On Sep 9, 2006, at 5:06 PM, George R. wrote: >> With that, I'm no packet-filter guru, I'd totally love it if folks on >> list would double-check the features, and re-post it! > > Come on Ike, don't give us that. . . *You* aren't a guru to review > this? > > ;-' Ok- I can comment on *some* things here. > > BTW, has anyone used PF on the master jail in FBSD to filter for the > jails? I know ipfw is the standard way to do packet-filtering with > jails. . . > > g No- jailed systems have no access to ipfw, or anything else- they are explicitly restricted from doing so. One can run packet filters on the host machine, conceptually making a jailing host the perimeter firewall is common practice for jailing. Best, .ike From george at sddi.net Sat Sep 9 18:13:09 2006 From: george at sddi.net (George R.) Date: Sat, 09 Sep 2006 18:13:09 -0400 Subject: [nycbug-talk] FreeBSD minimum memory Message-ID: <45033C75.2060008@sddi.net> Interesting post I found from the Richard B's Tao of Security blog: http://lists.freebsd.org/pipermail/freebsd-doc/2006-August/011029.html From dlavigne6 at sympatico.ca Sat Sep 9 19:21:51 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Sat, 9 Sep 2006 19:21:51 -0400 (EDT) Subject: [nycbug-talk] ipfw, ipf, pf comparison matrix In-Reply-To: <20060909164530.F624@dru.domain.org> References: <20060908174233.GF25206@cybertron.cyth.net> <20060909164530.F624@dru.domain.org> Message-ID: <20060909183115.V624@dru.domain.org> Okay, so I'm into firewalls and incomplete charts bug me... Here's a start at a table that only compares ipfw and pf. Functionality has been alphabetized. Comparisons were interesting as similar functionality was described using different terminology in the documentation for the two firewalls. I haven't had a need to make firewall rules that included the IP fields with ipfw keywords (man ipfw) and would appreciate anyone confirming if pf also allows you to refer to those fields and how to do so. I'd also like feedback on further functionality that should be added to the chart and a reference proving that a missing * is indeed possible in that firewall. Have fun :-) Dru --- Feature ipfw pf ---------------------------------- ADDRESS POOLS * * ALTQ * * ANCHORS/RULESETS * * ANTISPOOF * * AUTHPF * CARP * DUMMYNET * DYNAMIC NAT * * FLUSH * FTP PROXY * GROUP * * ICMP STATE * ICMP/6 CODES * ICMP/6 TYPES * * INCOMING LOAD BALANCING * * IP OPTIONS * * IP TOS 5 ALL IPSec * * IPv6 * * JAIL * NOT YET? LABELS * LISTS * * MAC FILTERING * * MAC-TYPE * MACROS * * MAX # * MAX-SRC-CONN-RATE * MAX-SRC-CONN/LIMIT SRC * * MAX-SRC-NODES * MAX-SRC-STATES * OPTIMIZATION * OSFP * OUTGOING LOAD BALANCING * OVERLOAD * PFSYNC * PORT FORWARDING * * PROBABILITY * * PROTOCOL ID * * PROXY FORWARDING * * QUICK * SCRUB/FRAG * * SCRUB/MIN-TTL * * SCRUB/MSS * * SCRUB/NO-DF * SCRUB/RANDOM-ID * SCRUB/REASSEMBLE * * SCRUB/RFC1323 * * SOURCE-TRACK * STATE MODULATION * STATIC NAT * * SYNPROXY * TABLES (IPv4) * * TABLES (IPv6) * TAGGING * TCP FLAGS * * TCP STATE * * UDP STATE * * USER * * VERREVPATH/URPF * * VERSRCREACH/ROUTING * * From tux at penguinnetwerx.net Sun Sep 10 10:30:21 2006 From: tux at penguinnetwerx.net (Kevin Reiter) Date: Sun, 10 Sep 2006 10:30:21 -0400 Subject: [nycbug-talk] Squid + squidGuard on FreeBSD 6.1 Message-ID: <4504217D.5010103@penguinnetwerx.net> All, Does anyone out there have Squid + squidGuard working on 6.x? I've been playing around with it for a few months, but the squidGuard list seems to have dried up, and all the docs I've seen are kinda old (not sure if they're outdated or not.) If anyone can provide a squid.conf that's actually working (with or without authentication), I'd appreciate it. The farthest I've gotten sofar is getting them both running, no errors or core dumps when starting, but the box refuses to do any filtering whatsoever, and the logs all say that everything is working fine. Thanks, Kev From dlavigne6 at sympatico.ca Mon Sep 11 09:19:40 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Mon, 11 Sep 2006 09:19:40 -0400 (EDT) Subject: [nycbug-talk] ipfw, ipf, pf comparison matrix In-Reply-To: <20060910192231.Y7416@daemon.bsdunix.net> References: <20060908174233.GF25206@cybertron.cyth.net> <20060909164530.F624@dru.domain.org> <20060909183115.V624@dru.domain.org> <20060910192231.Y7416@daemon.bsdunix.net> Message-ID: <20060911091736.R624@dru.domain.org> On Sun, 10 Sep 2006, Trish Lynch wrote: > What do you mean by "Flush", as ipfw has 'ipfw flush', if it means to flush > rulesets 'in place'. Sounds like time for a footnote as "flush" means something different to each firewall. In ipfw it is used to flush rules, in pf it is used to flush a connection out of the state table. Dru From trish at bsdunix.net Mon Sep 11 10:30:03 2006 From: trish at bsdunix.net (Trish Lynch) Date: Mon, 11 Sep 2006 14:30:03 +0000 Subject: [nycbug-talk] ipfw, ipf, pf comparison matrix In-Reply-To: <20060911091736.R624@dru.domain.org> References: <20060908174233.GF25206@cybertron.cyth.net> <20060909164530.F624@dru.domain.org><20060909183115.V624@dru.domain.org> <20060910192231.Y7416@daemon.bsdunix.net> <20060911091736.R624@dru.domain.org> Message-ID: <73585329-1157985008-cardhu_blackberry.rim.net-1366796796-@bxe035-cell01.bisx.prod.on.blackberry> Yes, as far as I know, there is no way to remove a connection from ther state table 'in place' like that, though I could be wrong. You might be able to reset the rule keeping track of that connection, but its not granular enough. I'll look into it more later if I have time. -Trish -- Trish Lynch M: 646-401-1405 H: 201-378-0434 -----Original Message----- From: Dru Date: Mon, 11 Sep 2006 09:19:40 To:Trish Lynch Cc:Isaac Levy , NYC Bug List Subject: Re: [nycbug-talk] ipfw, ipf, pf comparison matrix On Sun, 10 Sep 2006, Trish Lynch wrote: > What do you mean by "Flush", as ipfw has 'ipfw flush', if it means to flush > rulesets 'in place'. Sounds like time for a footnote as "flush" means something different to each firewall. In ipfw it is used to flush rules, in pf it is used to flush a connection out of the state table. Dru From dlavigne6 at sympatico.ca Mon Sep 11 11:59:33 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Mon, 11 Sep 2006 11:59:33 -0400 (EDT) Subject: [nycbug-talk] pf book In-Reply-To: References: <20060908103452.M624@dru.domain.org> Message-ID: <20060911114535.Q624@dru.domain.org> On Fri, 8 Sep 2006, Charles Sprickman wrote: > Did you find tags and QoS to be well-explained? I'm OK with basic stuff but > I totally rely on pfSense when I start playing with QoS and the like. And I > can barely follow the complex configs pfSense generates... The tagging chapter is verbatim to http://www.openbsd.org/faq/pf/tagging.html except for the Ethernet frames section where a paragraph has been added to show how to do this on NetBSD as its brconfig differs from OpenBSD's. Similarly, the QOS chapter is much of this verbatim: http://www.openbsd.org/faq/pf/queueing.html with BSD specific implementations added as needed. The Configuring Queueing section starts with which beginning versions of each BSD support ALTQ (useful in itself) as well as the necessary kernel compile options for Free/Dragonfly and the URL to the NetBSD patches. Diagrams have also replaced the ASCII versions. HTH, Dru From george at galis.org Mon Sep 11 14:23:37 2006 From: george at galis.org (George Georgalis) Date: Mon, 11 Sep 2006 14:23:37 -0400 Subject: [nycbug-talk] dd /dev/null before imaging disk Message-ID: <20060911182337.GQ23014@run.galis.org> Before imaging a disk, I've been in the habbit of dd if=/dev/null >/usr/zeros ; rm /usr/zeros etc, for each partition, and from a mini-root cdrom, whatever. The idea is to make the media contain the most compressable bytes possible. Then I dd the entire disk to bzip2 and save the output to my image file. I assume this works, I have come up with some rather small image files. So now I want to image an NTFS XP install, is there any good and easy way to write out zeros before I image? // George -- George Georgalis, systems architect, administrator < From george at galis.org Mon Sep 11 16:39:46 2006 From: george at galis.org (George Georgalis) Date: Mon, 11 Sep 2006 16:39:46 -0400 Subject: [nycbug-talk] dd /dev/null before imaging disk In-Reply-To: <20060911182337.GQ23014@run.galis.org> References: <20060911182337.GQ23014@run.galis.org> Message-ID: <20060911203946.GR23014@run.galis.org> On Mon, Sep 11, 2006 at 02:23:37PM -0400, George Georgalis wrote: >Before imaging a disk, I've been in the habbit of > >dd if=/dev/null >/usr/zeros ; rm /usr/zeros > >etc, for each partition, and from a mini-root cdrom, whatever. >The idea is to make the media contain the most compressable bytes >possible. Then I dd the entire disk to bzip2 and save the output >to my image file. > >I assume this works, I have come up with some rather small image >files. > >So now I want to image an NTFS XP install, is there any good and >easy way to write out zeros before I image? ...so I tried setting up an apache alias. Alias /zero "/dev/zero" but I seem to be missing some magic. tried playing with the asis handler, and even AddType application/octet-stream zero but I'm not getting more than a zero byte file.... Any ideas on what I can do to apache to 'spill the beans'? // George -- George Georgalis, systems architect, administrator < From jonathan at kc8onw.net Mon Sep 11 17:34:11 2006 From: jonathan at kc8onw.net (Jonathan Stewart) Date: Mon, 11 Sep 2006 17:34:11 -0400 Subject: [nycbug-talk] dd /dev/null before imaging disk In-Reply-To: <20060911182337.GQ23014@run.galis.org> References: <20060911182337.GQ23014@run.galis.org> Message-ID: <4505D653.3040505@kc8onw.net> George Georgalis wrote: > Before imaging a disk, I've been in the habbit of > > dd if=/dev/null >/usr/zeros ; rm /usr/zeros > > etc, for each partition, and from a mini-root cdrom, whatever. > The idea is to make the media contain the most compressable bytes > possible. Then I dd the entire disk to bzip2 and save the output > to my image file. > > I assume this works, I have come up with some rather small image > files. > > So now I want to image an NTFS XP install, is there any good and > easy way to write out zeros before I image? See section 5.10 http://www.feyrer.de/g4u/ There are a couple of different ways mentioned Jonathan From lists at stringsutils.com Mon Sep 11 22:40:31 2006 From: lists at stringsutils.com (Francisco Reyes) Date: Mon, 11 Sep 2006 22:40:31 -0400 Subject: [nycbug-talk] FreeBSD minimum memory References: <45033C75.2060008@sddi.net> Message-ID: George R. writes: > Interesting post I found from the Richard B's Tao of Security blog: > http://lists.freebsd.org/pipermail/freebsd-doc/2006-August/011029.html Indeed very interesting. In particular Linux Binary Compat. 255 127 I had no idea that used so much disk. Not a big deal for a modern HD, but for anything minimalistic like working on doing a special purpose boot CD... 127MB is a lot. From trish at bsdunix.net Tue Sep 12 06:28:36 2006 From: trish at bsdunix.net (Trish Lynch) Date: Tue, 12 Sep 2006 10:28:36 +0000 Subject: [nycbug-talk] FreeBSD minimum memory In-Reply-To: References: <45033C75.2060008@sddi.net> Message-ID: <1013628568-1158056919-cardhu_blackberry.rim.net-1251282683-@bxe053-cell01.bisx.prod.on.blackberry> Yes, but if you have the knowhow to create a special boot cd, you've also got the knowhow to trim the linux compat libs down, they're so bloated because they contain a 'mini distribution' of binaries as well...... Essentially for it to REALLY work all you really need is the binary compat kernel module and some statically (and as such, self contained) compiled linux bins.... -Trish -- Trish Lynch M: 646-401-1405 H: 201-378-0434 -----Original Message----- From: Francisco Reyes Date: Mon, 11 Sep 2006 22:40:31 To:george at sddi.net Cc:NYCBUG Subject: Re: [nycbug-talk] FreeBSD minimum memory George R. writes: > Interesting post I found from the Richard B's Tao of Security blog: > http://lists.freebsd.org/pipermail/freebsd-doc/2006-August/011029.html Indeed very interesting. In particular Linux Binary Compat. 255 127 I had no idea that used so much disk. Not a big deal for a modern HD, but for anything minimalistic like working on doing a special purpose boot CD... 127MB is a lot. _______________________________________________ % NYC*BUG talk mailing list http://lists.nycbug.org/mailman/listinfo/talk %Be sure to check out our Jobs and NYCBUG-announce lists %We meet the first Wednesday of the month From lists at genoverly.net Tue Sep 12 09:53:51 2006 From: lists at genoverly.net (michael) Date: Tue, 12 Sep 2006 09:53:51 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts Message-ID: <20060912095351.4266acc9@wit.genoverly.com> http://www.securityfocus.com/infocus/1876 Analyzing malicious SSH login attempts Christian Seifert 2006-09-11 Introduction Malicious SSH login attempts have been appearing in some administrators' logs for several years. This article revisits the use of honeypots to analyze malicious SSH login attempts and see what can be learned about this activity. The article then offers recommendations on how to secure one's system against these attacks. ... Recommendations [snips] * Use the /etc/hosts.allow and /etc/hosts.deny files... * Install a firewall to restrict access to the SSH server... * Restrict the SSH server to only authenticate... * Move the listening port of the SSH server from 22... * Use an alternate authentication method... * disable remote access to root... I've read Hosts.[allow|deny] can be spoofed and besides, I can not predict where I'll be when I want to logon. Granted, I could leave a box open somewhere to logon to, and then hop to the target with that box as allowed.. but, what's the point? I still have a 'weak link' according to their logic. I am not a fan of port knocking, port shuffling, or any other port dance moves. It would only delay an attacker a few seconds but would wreak havoc on my muscle memory and any scripts that use scp, rsync, forwarding, or tunnelling. For years I used PermitRootLogin=No, but I am being swayed recently that that is false security. I also have found it to be really inconvenient. Recently, I have been moving toward keys vs. passwords (it makes logons fast and fun). But I still have lingering anxiety that once you have my desktop, you have my local network AND my datacenter network AND anywhere else I've dropped a key. Maybe I should, more seriously, consider the shear hassle of skeys. I'm curious, do NYCBUG talk subscribers consider this a "best practices" article? Is anything misleading, wrong, missing.. or right? I am also curious.. where do we draw the line and just *trust* our OS? -- Michael From dlavigne6 at sympatico.ca Tue Sep 12 10:33:49 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Tue, 12 Sep 2006 10:33:49 -0400 (EDT) Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060912095351.4266acc9@wit.genoverly.com> References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: <20060912103121.C624@dru.domain.org> On Tue, 12 Sep 2006, michael wrote: > Recommendations [snips] > * Use the /etc/hosts.allow and /etc/hosts.deny files... > * Install a firewall to restrict access to the SSH server... > * Restrict the SSH server to only authenticate... > * Move the listening port of the SSH server from 22... > * Use an alternate authentication method... > * disable remote access to root... > > I've read Hosts.[allow|deny] can be spoofed and besides, I can not > predict where I'll be when I want to logon. Granted, I could leave a > box open somewhere to logon to, and then hop to the target with that > box as allowed.. but, what's the point? I still have a 'weak link' > according to their logic. > > I am not a fan of port knocking, port shuffling, or any other port > dance moves. It would only delay an attacker a few seconds but would > wreak havoc on my muscle memory and any scripts that use scp, rsync, > forwarding, or tunnelling. > > For years I used PermitRootLogin=No, but I am being swayed recently > that that is false security. I also have found it to be really > inconvenient. > > Recently, I have been moving toward keys vs. passwords (it makes logons > fast and fun). But I still have lingering anxiety that once you have > my desktop, you have my local network AND my datacenter network AND > anywhere else I've dropped a key. > > Maybe I should, more seriously, consider the shear hassle of skeys. > > I'm curious, do NYCBUG talk subscribers consider this a "best > practices" article? Is anything misleading, wrong, missing.. or right? > > I am also curious.. where do we draw the line and just *trust* our OS? Here is what I do. Curious as to what works for others on the list. - restrict users with AllowUsers - reduce MaxAuthTries to 3 - user overload/flush in pf to keep the logs sane Dru From chsnyder at gmail.com Tue Sep 12 11:52:26 2006 From: chsnyder at gmail.com (csnyder) Date: Tue, 12 Sep 2006 11:52:26 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060912095351.4266acc9@wit.genoverly.com> References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: On 9/12/06, michael wrote: > I still have lingering anxiety that once you have > my desktop, you have my local network AND my datacenter network AND > anywhere else I've dropped a key. But you encrypted that key using a strong passphrase, right? They would have to get your desktop while ssh-agent was running. > Maybe I should, more seriously, consider the shear hassle of skeys. > > I'm curious, do NYCBUG talk subscribers consider this a "best > practices" article? Is anything misleading, wrong, missing.. or right? > > I am also curious.. where do we draw the line and just *trust* our OS? > I really wish the OpenSSH developers would address this issue in the server itself, by giving admins a lockout setting. I see absolutely no reason why hundreds of failed login attempts from the same IP address should be permitted as if it was standard procedure. Anyway, I use a php script that scans the log for multiple failed logins from a single IP, then sets a temporary firewall rule blocking access from that address. -- Chris Snyder http://chxo.com/ From lists at genoverly.net Tue Sep 12 12:29:53 2006 From: lists at genoverly.net (michael) Date: Tue, 12 Sep 2006 12:29:53 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: <20060912122953.76ac705b@wit.genoverly.com> On Tue, 12 Sep 2006 11:52:26 -0400 csnyder wrote: > But you encrypted that key using a strong passphrase, right? They > would have to get your desktop while ssh-agent was running. well.. I don't shut down my home PC when I walk away. It is usually running. But I do lock the apartment door [grin]. -- Michael From george at galis.org Tue Sep 12 13:12:15 2006 From: george at galis.org (George Georgalis) Date: Tue, 12 Sep 2006 13:12:15 -0400 Subject: [nycbug-talk] dd /dev/null before imaging disk In-Reply-To: <4505D653.3040505@kc8onw.net> References: <20060911182337.GQ23014@run.galis.org> <4505D653.3040505@kc8onw.net> Message-ID: <20060912171215.GD15283@run.galis.org> On Mon, Sep 11, 2006 at 05:34:11PM -0400, Jonathan Stewart wrote: >George Georgalis wrote: >> Before imaging a disk, I've been in the habbit of >> >> dd if=/dev/null >/usr/zeros ; rm /usr/zeros >> >> etc, for each partition, and from a mini-root cdrom, whatever. >> The idea is to make the media contain the most compressable bytes >> possible. Then I dd the entire disk to bzip2 and save the output >> to my image file. >> >> I assume this works, I have come up with some rather small image >> files. >> >> So now I want to image an NTFS XP install, is there any good and >> easy way to write out zeros before I image? > >See section 5.10 http://www.feyrer.de/g4u/ > >There are a couple of different ways mentioned I used "cipher /W:C:" and that worked great! (I just ran it a second time a little bit to fix the over-run of the first run) In the past g4u was not the right solution for me... but now it is the perfect tool, and worked like a champ. // George -- George Georgalis, systems architect, administrator < From george at galis.org Tue Sep 12 13:24:51 2006 From: george at galis.org (George Georgalis) Date: Tue, 12 Sep 2006 13:24:51 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060912122953.76ac705b@wit.genoverly.com> References: <20060912095351.4266acc9@wit.genoverly.com> <20060912122953.76ac705b@wit.genoverly.com> Message-ID: <20060912172451.GE15283@run.galis.org> On Tue, Sep 12, 2006 at 12:29:53PM -0400, michael wrote: >On Tue, 12 Sep 2006 11:52:26 -0400 >csnyder wrote: > >> But you encrypted that key using a strong passphrase, right? They >> would have to get your desktop while ssh-agent was running. > >well.. I don't shut down my home PC when I walk away. It is usually >running. But I do lock the apartment door [grin]. you do lock the screen, logout, or otherwise make the agent unavailable, right? UsePam No Use usb dive (with your private key), on your keychain, yes. There was some resolution (at openbsd I think) to encrypt the known_hosts entries with the remote host public key; so if your authentication was compromised, at least there wouldn't be a list a hosts for the attacker to look up. But I've not seen it in my OS yet. Maybe something similar should be done with .ssh/config? // George -- George Georgalis, systems architect, administrator < From af.dingo at gmail.com Tue Sep 12 13:46:20 2006 From: af.dingo at gmail.com (Jeff Quast) Date: Tue, 12 Sep 2006 13:46:20 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: On 9/12/06, csnyder wrote: > > > > I am also curious.. where do we draw the line and just *trust* our OS? > > As if password brute forcing is anything new.... Well OpenSSH is written by an OS project that can be trusted. This is not an issue of trusting an OS anyway, it is an issue of trusting that the legitmate accounts on your machine use strong passwords. This is usualy enforced with passwd, and can be monitored with john the ripper if you have a large and dynamic enviroment. If it takes 2 hours for a modern machine to crack a password, it would take 3 years for a remote attacker to brute both login and password over the network. It can be scripted and cron'd to disable and mail somebody when an account has been disabled due to weak passwords. > I really wish the OpenSSH developers would address this issue in the > server itself, by giving admins a lockout setting. I see absolutely no > reason why hundreds of failed login attempts from the same IP address > should be permitted as if it was standard procedure. OpenSSH developers are not responsible for making sure you use strong passwords and a secure OS or a good firewall. If OpenSSH kept adding features and knobs the community wants to see on it, it wouldn't be secure and simple anymore, much less portable. There is a trivial solution for blocking hosts that connect too many times, http://www.openbsd.org/faq/pf/filter.html#stateopts Hasn't made it to freebsd yet, of course, http://lists.freebsd.org/pipermail/freebsd-pf/2005-August/001409.html As you see in most of the examples from a google search of 'max-src-conn-rate overload' It works for more than just ssh brute forcing. Works great to block wget, nmap, web bots, spammers, etc. > Anyway, I use a php script that scans the log for multiple failed > logins from a single IP, then sets a temporary firewall rule blocking > access from that address. I think parsing logs and injecting rules is just plain ridiculous. Especialy using 3rd party languages not native to your OS. Its just more custom stuff to re-implement on the next os rebuild. Searching archives you will find simple shell scripts that can dump the pf table of blocked IP's and save to disk to be reloaded later. I've never bothered, as you only need to block them for a few hours until they've given up for eternity. This whole discussion has been beaten to death for years. pf is the solution. If you dont have pf, then don't use passwords. Use ssh keys. A little knowledge of ssh-agent can make using ssh keys more convenient and secure than using passwords. I just felt the need to reply to the line that this is OpenSSH's responsability to deal with. It made me mad. They do a great job dealing with this issue in the place it is meant to be dealt with. Password authentication should only be used once to add your public key to authorized_keys file anyway. I dont even know most of the passwords for my SSH accounts :0, they are too hard to remember, much less guess. From dlavigne6 at sympatico.ca Tue Sep 12 14:27:48 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Tue, 12 Sep 2006 14:27:48 -0400 (EDT) Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: <20060912142623.I624@dru.domain.org> On Tue, 12 Sep 2006, Jeff Quast wrote: > There is a trivial solution for blocking hosts that connect too many > times, http://www.openbsd.org/faq/pf/filter.html#stateopts > > Hasn't made it to freebsd yet, of course, > http://lists.freebsd.org/pipermail/freebsd-pf/2005-August/001409.html That changed in November 2005: http://www.freebsd.org/releases/6.0R/relnotes-i386.html I've been happily using max-src-conn-rate overload since January :-) Dru From chsnyder at gmail.com Tue Sep 12 14:54:20 2006 From: chsnyder at gmail.com (csnyder) Date: Tue, 12 Sep 2006 14:54:20 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: On 9/12/06, Jeff Quast wrote: > There is a trivial solution for blocking hosts that connect too many > times, http://www.openbsd.org/faq/pf/filter.html#stateopts > > Hasn't made it to freebsd yet, of course, > http://lists.freebsd.org/pipermail/freebsd-pf/2005-August/001409.html Or to Linux. Or to OSX. > I think parsing logs and injecting rules is just plain ridiculous. > Especialy using 3rd party languages not native to your OS. Its just > more custom stuff to re-implement on the next os rebuild. Look, I know it's ridiculous, but it's also more portable (for now) than pf. > I just felt the need to reply to the line that this is OpenSSH's > responsability to deal with. It made me mad. They do a great job > dealing with this issue in the place it is meant to be dealt with. It may not be the developer's responsibility to implement such a feature, but I feel no qualms about wishing that they would. Sshd is in a very good place to prevent this kind of abuse, and it could be prevented in a way that isn't OS and firewall dependent. -- Chris Snyder http://chxo.com/ From riegersteve at gmail.com Tue Sep 12 18:46:46 2006 From: riegersteve at gmail.com (Steve Rieger) Date: Tue, 12 Sep 2006 15:46:46 -0700 Subject: [nycbug-talk] etc/hosts Message-ID: <450738D6.9090103@gmail.com> on fbsd, www2# cat hosts 127.0.0.1 localhost.x.net localhost x.x.x.252 www.x.net www x.x.x.252 www.x.net. is the . at the end required ? me dont think so, looking for confirmation. -- -- eats the blues for breakfast, does unix for rent, plays harp for food, will play the flute for kicks rides for the freedom works hard, playes harder 310-883-5838 (cell) pager at up-south.com (pager) From dan at langille.org Tue Sep 12 19:56:40 2006 From: dan at langille.org (Dan Langille) Date: Tue, 12 Sep 2006 19:56:40 -0400 Subject: [nycbug-talk] etc/hosts In-Reply-To: <450738D6.9090103@gmail.com> Message-ID: <450710F8.26224.2F4BABC8@dan.langille.org> On 12 Sep 2006 at 15:46, Steve Rieger wrote: > on fbsd, > > www2# cat hosts > 127.0.0.1 localhost.x.net localhost > x.x.x.252 www.x.net www > x.x.x.252 www.x.net. > > > > is the . at the end required ? > > me dont think so, looking for confirmation. It is not, but it can be useful. If for example, you have this in /etc/resolv.conf: search unixathome.org And you're doing host names such as freebsd.org.unixathome.org, the freebsd.org. will take you to the FreeBSD website, and freebsd.org will take you to .... Ummm, well that doesn't work, but that's the example I was trying to give.... -- Dan Langille : Software Developer looking for work my resume: http://www.freebsddiary.org/dan_langille.php From spork at bway.net Tue Sep 12 23:58:38 2006 From: spork at bway.net (Charles Sprickman) Date: Tue, 12 Sep 2006 23:58:38 -0400 (EDT) Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060912172451.GE15283@run.galis.org> References: <20060912095351.4266acc9@wit.genoverly.com> <20060912122953.76ac705b@wit.genoverly.com> <20060912172451.GE15283@run.galis.org> Message-ID: On Tue, 12 Sep 2006, George Georgalis wrote: > On Tue, Sep 12, 2006 at 12:29:53PM -0400, michael wrote: >> On Tue, 12 Sep 2006 11:52:26 -0400 >> csnyder wrote: >> >>> But you encrypted that key using a strong passphrase, right? They >>> would have to get your desktop while ssh-agent was running. >> >> well.. I don't shut down my home PC when I walk away. It is usually >> running. But I do lock the apartment door [grin]. > > you do lock the screen, logout, or otherwise make the agent > unavailable, right? If you're running os-x as your desktop box, SSHKeychain has a number of handy features, including one to pull the key from the agent when the machine sleeps or the screensaver kicks in. Fairly handy. http://www.sshkeychain.org/ Charles > // George > > > -- > George Georgalis, systems architect, administrator < > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > From ike at lesmuug.org Wed Sep 13 10:35:00 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 13 Sep 2006 10:35:00 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060912095351.4266acc9@wit.genoverly.com> References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: Hi MW, All, On Sep 12, 2006, at 9:53 AM, michael wrote: > > Recently, I have been moving toward keys vs. passwords (it makes > logons > fast and fun). But I still have lingering anxiety that once you have > my desktop, you have my local network AND my datacenter network AND > anywhere else I've dropped a key. > > Maybe I should, more seriously, consider the shear hassle of skeys. To be straight, I've used and loved SSH keys for years- so I'm biased here, and I feel there's a lot of over-macho info out there about using keys, and little compiled on the important basics. I tend to use huge keys, and replace them often- and not muck about with ssh keychain and the like, its' served me very well. Basically, I feel that ssh-agent, SSHkeychain, and a other keychain apps people mention, can be really overwhelming for new key users to deal with. I've also found them to be 'cumbersome under pressure' for experienced key users, but that's a matter of taste perhaps. Since SSH is obviously a critical part of any of my infrastructure *gulp*, I prefer to keep it really simple. (Again, this is all about threat models and one's use contexts, right?) ---------------------------------------------------------------------- With that, I'll explain what I feel are the important basics of my key use (some of this obvious stuff to key users), I'm obviously making suggestions based on my use patterns. For the record, I manage 'a fistfull' of hardware machines in datacenter and office enviornments, and run lots and lots of jailed systems. My threat models are pretty much all based around attacks from the internet. -- First, use big keys, period (4096 bit are fabby). I believe many older systems have problems with anything larger than 4096 bit keys (32 bit architecture issues in older softwares?), but by today's CPU standards, this isn't really so big. Common discussion on key size is that it would take serious crackers, (people with supercomputers), 3-5 years to crack a 4096 bit key outright. This is assuming network auth. capture and replay. This is of course only speculation, and it can be assumed people with supercomputers can do much better these days. Using 4096 bit keys work just as fast as smaller keys in practical use. -- Lock a key with a passphrase, so you unlock them on your local computer when you use them. Some admins like the fact that unlocked keys let them jump to-and-fro between machines without having to enter passwords, I feel this is silly and have seen borderline irresponsible uses of keys in this manner. If you don't lock your keys, anyone can use it- and you have to spend an inordinate amount of time protecting them against overwhelming threats. This is where ssh-agent and the other keychain apps come into play, because they let you authenticate your keys once locally, but again this opens the door to various and common local threats. Is it really that cumbersome to enter a passphrase for each ssh login? **caveat** It does get messy to manage using multiple keys without ssh-agent. However, a caveat to that caveat, is that using many keys discourages admins from changing their keys regularly, which I see as a MUCH larger threat :) -- If you lock your private keys with a local passphrase, you can then happily toss them around to different trusted systems, keep them on your iPod, whatever you choose to trust. The threat here is that someone would run a dictionary attack against your keys themselves, so you still want to be very conservative with where they live. Also, one must of course trust any machine where they *unlock* those keys, (any machine which you ssh OUT of). -- Your public keys, can be tossed around pretty freely, and placed on remote systems you simply want to log IN to. It's a cool way to work, because between savvy admins, there's no need to deal with passwords when setting up shell account access. (These keys are ~/.ssh/id_dsa.pub or ~/.ssh/id_dsa.pub respectively) This is great for shell accounts on systems you don't use, or don't entirely trust based on it's use. If your not going to ever ssh out from said machine, there's no reason for you to have your private keys up there, weather they're locked with a passphrase or not. -- RSA or DSA keys? You decide, this is really a big ongoing debate by the hardcore crypto folks. RSA is far older, comes from MIT in the late 70's. DSA comes from the NSA in the mid 90's. Both are believed to be a secure signature algorithm given 'sufficently long keys'. I believe the relative age of RSA is what prompted the 'RSAAuthentication yes' line in sshd_config. In the late 90's, when DSA had become equally common, it was assumed RSA would be (or had been) cracked. Since then, it's held up quite well, and at the time of this writing RSA is only twice as old as DSA (15 years compared to 30 years), so hrmph- it's all all about calculated risk anyhow, eh? :) -- Dissalow password auth on your servers. On 'high-security' host machines, simply disallow password authentication alltogether in your sshd_config file- (and noteworthy, on some systems [FreeBSD 6.x onward for example] one needs to disable PAM auth. in the config file too, to completely disallow password logins). This is critical for any online system you wish to protect, password auth is low lying fruit. A practical personal experience with this was with the 'SSH1 CRC-32' exploit from 2001, which of course rendered SSH protocol V1 useless: http://www.cert.org/incident_notes/IN-2001-12.html So basically, my experience was that systems that disallowed password auth weren't low enough fruit for the ssh feeding frenzy that ensued- giving us plenty of time to reconfigure the systems to disallow SSHv1 alltogether, and run around unplugging any hardwired-ssh1 in network appliances of the time (PDU's, etc...). ---------------------------------------------------------------------- At the end of the day an exploit is an exploit- and everything is of course crackable (including one's brain), but ssh keys keep your system from being low-lying fruit- which is practical and important today on the internet. So my key philosophy is simple- change your keys often, and do whatever is easiest for you according to your threat and usage models. An awesome how-to, because it's simple: http://pkeck.myweb.uga.edu/ssh/ > > I'm curious, do NYCBUG talk subscribers consider this a "best > practices" article? Is anything misleading, wrong, missing.. or > right? Hope my rambling primer is useful to folks out there. > > I am also curious.. where do we draw the line and just *trust* our OS? When we turn our computers off, IMHO. :P Best, .ike From okan at demirmen.com Wed Sep 13 10:37:17 2006 From: okan at demirmen.com (Okan Demirmen) Date: Wed, 13 Sep 2006 10:37:17 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060912172451.GE15283@run.galis.org> References: <20060912095351.4266acc9@wit.genoverly.com> <20060912122953.76ac705b@wit.genoverly.com> <20060912172451.GE15283@run.galis.org> Message-ID: <20060913143717.GP25801@clam.khaoz.org> On Tue 2006.09.12 at 13:24 -0400, George Georgalis wrote: > There was some resolution (at openbsd I think) to encrypt > the known_hosts entries with the remote host public key; > so if your authentication was compromised, at least there > wouldn't be a list a hosts for the attacker to look up. > But I've not seen it in my OS yet. man ssh_config - see HashKnownHosts From dave at donnerjack.com Wed Sep 13 11:34:07 2006 From: dave at donnerjack.com (David Lawson) Date: Wed, 13 Sep 2006 11:34:07 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: On Sep 13, 2006, at 10:35 AM, Isaac Levy wrote: > Lock a key with a passphrase, so you unlock them on your local > computer when you use them. > > Some admins like the fact that unlocked keys let them jump to-and-fro > between machines without having to enter passwords, I feel this is > silly and have seen borderline irresponsible uses of keys in this > manner. If you don't lock your keys, anyone can use it- and you have > to spend an inordinate amount of time protecting them against > overwhelming threats. > This is where ssh-agent and the other keychain apps come into play, > because they let you authenticate your keys once locally, but again > this opens the door to various and common local threats. Is it > really that cumbersome to enter a passphrase for each ssh login? > **caveat** It does get messy to manage using multiple keys without > ssh-agent. However, a caveat to that caveat, is that using many keys > discourages admins from changing their keys regularly, which I see > as a MUCH larger threat :) > > -- > If you lock your private keys with a local passphrase, you can then > happily toss them around to different trusted systems, keep them on > your iPod, whatever you choose to trust. > > The threat here is that someone would run a dictionary attack against > your keys themselves, so you still want to be very conservative with > where they live. Also, one must of course trust any machine where > they *unlock* those keys, (any machine which you ssh OUT of). This is really the only part of what Ike has to say that I'd disagree with. Personally, I've found that, yes, it is cumbersome to be entering a passphrase for every login to a machine, and that negates a lot of the convenience that comes with using ssh keys and makes their added security attractive to admins. The various key management tools (SSHKeychain, ssh-agent) can all be configured securely, to time out the authorization of a key after a given period of time so the passphrase has to be re-entered, and a passphrase would be, to say the least, extremely difficult to dictionary attack, since the theory is, rather than a word, it's phrase. The only real option is to brute force the passphrase, which isn't going to be terribly effective if it's of a reasonable length. The flip side of this is that I can't think of any good reason, when using an agent to manage your keys, to have an un-passphrase protected private key. That would strike me as an extremely irresponsible way to manage access, since that really does depend entirely upon the security of they private key file. My experience has been that a passphrase protected ssh key with a management agent (SSHKeychain in my case), makes managing secure access to large numbers of machines vastly, vastly simpler than it would be using passwords. Some of that, I think, will vary depending on your working environment and needs, but in general I've become a huge fan of keys and agent forwarding over the last few years, so personally I can't really think of a good argument _against_ using keys to do authentication, though I'd be interested to hear one if one exists. --Dave From dave at donnerjack.com Wed Sep 13 11:34:07 2006 From: dave at donnerjack.com (David Lawson) Date: Wed, 13 Sep 2006 11:34:07 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: On Sep 13, 2006, at 10:35 AM, Isaac Levy wrote: > Lock a key with a passphrase, so you unlock them on your local > computer when you use them. > > Some admins like the fact that unlocked keys let them jump to-and-fro > between machines without having to enter passwords, I feel this is > silly and have seen borderline irresponsible uses of keys in this > manner. If you don't lock your keys, anyone can use it- and you have > to spend an inordinate amount of time protecting them against > overwhelming threats. > This is where ssh-agent and the other keychain apps come into play, > because they let you authenticate your keys once locally, but again > this opens the door to various and common local threats. Is it > really that cumbersome to enter a passphrase for each ssh login? > **caveat** It does get messy to manage using multiple keys without > ssh-agent. However, a caveat to that caveat, is that using many keys > discourages admins from changing their keys regularly, which I see > as a MUCH larger threat :) > > -- > If you lock your private keys with a local passphrase, you can then > happily toss them around to different trusted systems, keep them on > your iPod, whatever you choose to trust. > > The threat here is that someone would run a dictionary attack against > your keys themselves, so you still want to be very conservative with > where they live. Also, one must of course trust any machine where > they *unlock* those keys, (any machine which you ssh OUT of). This is really the only part of what Ike has to say that I'd disagree with. Personally, I've found that, yes, it is cumbersome to be entering a passphrase for every login to a machine, and that negates a lot of the convenience that comes with using ssh keys and makes their added security attractive to admins. The various key management tools (SSHKeychain, ssh-agent) can all be configured securely, to time out the authorization of a key after a given period of time so the passphrase has to be re-entered, and a passphrase would be, to say the least, extremely difficult to dictionary attack, since the theory is, rather than a word, it's phrase. The only real option is to brute force the passphrase, which isn't going to be terribly effective if it's of a reasonable length. The flip side of this is that I can't think of any good reason, when using an agent to manage your keys, to have an un-passphrase protected private key. That would strike me as an extremely irresponsible way to manage access, since that really does depend entirely upon the security of they private key file. My experience has been that a passphrase protected ssh key with a management agent (SSHKeychain in my case), makes managing secure access to large numbers of machines vastly, vastly simpler than it would be using passwords. Some of that, I think, will vary depending on your working environment and needs, but in general I've become a huge fan of keys and agent forwarding over the last few years, so personally I can't really think of a good argument _against_ using keys to do authentication, though I'd be interested to hear one if one exists. --Dave From ike at lesmuug.org Wed Sep 13 12:08:05 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 13 Sep 2006 12:08:05 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: Hi Dave, All, On Sep 13, 2006, at 11:34 AM, David Lawson wrote: > This is really the only part of what Ike has to say that I'd > disagree with. Personally, I've found that, yes, it is cumbersome > to be entering a passphrase for every login to a machine, and that > negates a lot of the convenience that comes with using ssh keys and > makes their added security attractive to admins. Dave, since I believe it was you and wintermute who taught me to use keys in the first place (sometime around 99'), and I *know* you regularly manage far more machines than I do, I'll happily nod with approval to this practice. > The flip side of this is that I can't think of any good reason, > when using an agent to manage your keys, to have an un-passphrase > protected private key. That would strike me as an extremely > irresponsible way to manage access, since that really does depend > entirely upon the security of they private key file. Actually, one cool use I've applied in a pinch with great success, is to use un-passphrase keys for 'robot users' to run quick and dirty operations between machines. Essentially, creating underprivileged user accounts who possess keys without passphrases, allows one to setup funky cron jobs to shuffle data, or run commands, over ssh. e.g. 'myrobotuser' can ssh files between machines nightly, or get ps statistics from the other machine every few minutes, or whatever. I've done this in suituations where 2+ servers are not multi-user systems, for if one machine is compromised, those private keys are unprotected, and the other system is easily compromised... So, this setup is really only useful when the threat lies in protecting network MITM between cron-powered robot rpc, and when the local machines are contextually trusted. Kindof a hack for rpc, but getting creative with the building blocks is what makes UNIX fun and powerful, to me. :) > I can't really think of a good argument _against_ using keys to do > authentication, though I'd be interested to hear one if one exists. Me too, my rpc-robot tangent above really is a different kind of key use- not daily admin login practices. Rocket- .ike From ike at lesmuug.org Wed Sep 13 13:02:20 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 13 Sep 2006 13:02:20 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060913143717.GP25801@clam.khaoz.org> References: <20060912095351.4266acc9@wit.genoverly.com> <20060912122953.76ac705b@wit.genoverly.com> <20060912172451.GE15283@run.galis.org> <20060913143717.GP25801@clam.khaoz.org> Message-ID: On Sep 13, 2006, at 10:37 AM, Okan Demirmen wrote: > On Tue 2006.09.12 at 13:24 -0400, George Georgalis wrote: >> There was some resolution (at openbsd I think) to encrypt >> the known_hosts entries with the remote host public key; >> so if your authentication was compromised, at least there >> wouldn't be a list a hosts for the attacker to look up. >> But I've not seen it in my OS yet. > > man ssh_config - see HashKnownHosts Niiiiice. Thx Okan. I learn something useful in every thread on this list :) Rocket- .ike From okan at demirmen.com Wed Sep 13 13:09:01 2006 From: okan at demirmen.com (Okan Demirmen) Date: Wed, 13 Sep 2006 13:09:01 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: <20060913170901.GU25801@clam.khaoz.org> On Wed 2006.09.13 at 12:08 -0400, Isaac Levy wrote: > Actually, one cool use I've applied in a pinch with great success, is > to use un-passphrase keys for 'robot users' to run quick and dirty > operations between machines. Essentially, creating underprivileged > user accounts who possess keys without passphrases, allows one to > setup funky cron jobs to shuffle data, or run commands, over ssh. > e.g. 'myrobotuser' can ssh files between machines nightly, or get ps > statistics from the other machine every few minutes, or whatever. and to simply add to this...use options within AuthorizedKeysFile! From ike at lesmuug.org Wed Sep 13 13:25:22 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 13 Sep 2006 13:25:22 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: <044D5671-B462-47F2-832F-97CD4ABFE129@lesmuug.org> Hi All, Some SSH food for thought, On Sep 12, 2006, at 2:54 PM, csnyder wrote: >> I think parsing logs and injecting rules is just plain ridiculous. >> Especialy using 3rd party languages not native to your OS. Its just >> more custom stuff to re-implement on the next os rebuild. > > Look, I know it's ridiculous, but it's also more portable (for now) > than pf. Forgive my possible naivet?, but how does any ssh/packet-filter incorporation strategy really secure anything, big picture (regardless of the implementation)? What happens when ssh passwords come under distributed dictionary attack by a botnet (many IP addresses)? Wouldn't it render the filter moot, and perhaps even create a resource attack as a side effect of dynamically loading gargantuan filter rulesets? What happens when an attacker spoofs the IP addresses you use, with the effect of blocking you from your own systems? -- Additionally, what happens when SSH itself meets it's inevitable zero- day (could be tomorrow, could be 50 years from now)? Doesn't any complicated intermingling with other parts of the system make ssh that much more difficult and error prone to replace quickly? I'm not lookin' to pick a flame-fight, I'm just discussing, and I feel many packet-filter strategies give a false sense of security. Convince me it's a sane strategy, and I'll likely go implement it tomorrow :) Rocket- .ike From dlavigne6 at sympatico.ca Wed Sep 13 13:55:01 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Wed, 13 Sep 2006 13:55:01 -0400 (EDT) Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <044D5671-B462-47F2-832F-97CD4ABFE129@lesmuug.org> References: <20060912095351.4266acc9@wit.genoverly.com> <044D5671-B462-47F2-832F-97CD4ABFE129@lesmuug.org> Message-ID: <20060913134517.P624@dru.domain.org> On Wed, 13 Sep 2006, Isaac Levy wrote: > Forgive my possible naivet?, but how does any ssh/packet-filter > incorporation strategy really secure anything, big picture > (regardless of the implementation)? Aaah, but isn't that the rub in security? Security after all is a myth, or at best, an arms race where you have to balance risk and effort :-) > What happens when ssh passwords come under distributed dictionary > attack by a botnet (many IP addresses)? Wouldn't it render the > filter moot, and perhaps even create a resource attack as a side > effect of dynamically loading gargantuan filter rulesets? I haven't experienced this problem and would be interested to hear if others have. My worst box experience was on a network where the ISP did absolutely no upstream filtering. The first time I activated a service on that system, I had to stop it within 30 seconds as the amount of crap traffic hitting the system was faster than syslog could keep up with. However some pf overload rules took care of the crap and even though the bad_hosts table I was overloading to had over 10,000 entries, it did not effect performance on the box. Being a bit cautious, I spent an afternoon whois'ing and combining network blocks for portions of the world that had no legit reason to contact that server--again, I'd be interested in hearing how large others' tables are without effecting performance. > What happens when an attacker spoofs the IP addresses you use, with > the effect of blocking you from your own systems? This I haven't experienced. But, again, I have addresses scattered throughout various networks I could come in from as I have been known to lock myself out on rare occasion :-) Dru From ike at lesmuug.org Wed Sep 13 14:11:04 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 13 Sep 2006 14:11:04 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060913134517.P624@dru.domain.org> References: <20060912095351.4266acc9@wit.genoverly.com> <044D5671-B462-47F2-832F-97CD4ABFE129@lesmuug.org> <20060913134517.P624@dru.domain.org> Message-ID: Hi Dru, All, On Sep 13, 2006, at 1:55 PM, Dru wrote: > > > On Wed, 13 Sep 2006, Isaac Levy wrote: > >> Forgive my possible naivet?, but how does any ssh/packet-filter >> incorporation strategy really secure anything, big picture >> (regardless of the implementation)? > > > Aaah, but isn't that the rub in security? Security after all is a > myth, or at best, an arms race where you have to balance risk and > effort :-) That statement gets the packet-fu award from me for this summer. > > >> What happens when ssh passwords come under distributed dictionary >> attack by a botnet (many IP addresses)? Wouldn't it render the >> filter moot, and perhaps even create a resource attack as a side >> effect of dynamically loading gargantuan filter rulesets? > > > I haven't experienced this problem and would be interested to hear > if others have. I haven't seen it with SSH, but I have experienced this with MTA's and web applications, simultaneous distributed dict. attacks, each originating from a different IP address. Ugly. Not sure of the scale or true nature of the attacker's systems, never investigated once the problem was solved- (the apps modified to limit particular auth. attempt scale, respectively). > My worst box experience was on a network where the ISP did > absolutely no upstream filtering. The first time I activated a > service on that system, I had to stop it within 30 seconds as the > amount of crap traffic hitting the system was faster than syslog > could keep up with. However > some pf overload rules took care of the crap and even though the > bad_hosts > table I was overloading to had over 10,000 entries, it did not effect > performance on the box. Being a bit cautious, I spent an afternoon > whois'ing and combining network blocks for portions of the world > that had no > legit reason to contact that server--again, I'd be interested in > hearing how large others' tables are without effecting performance. Ugh. > > >> What happens when an attacker spoofs the IP addresses you use, with >> the effect of blocking you from your own systems? > > > This I haven't experienced. But, again, I have addresses scattered > throughout various networks I could come in from as I have been > known to lock myself out on rare occasion :-) I've had this kind of attack attempted, the following all happened in about 20 seconds. During a large mail joe-back attack I was part of resolving, which also included an LTA attack on the mailserver- exploiting a vuln. in Cyrus involving oversize subject lines (over 256 characters). Bad day altogether. I made the mistake of running a traceroute on a particular host which was part of the attack, using my laptop behind a remote DSL line where I was ssh'd to the servers. Within seconds, around 100 ssh auth attempts were made to one mailserver *spoofing the IP address of my DSL line*, in effect locking me out of future SSH connections based on 'MaxAuthTries' in sshd_conf. I had a few shells already on that box, so I was able to continue working, (or I guess I could have figured out how to flush MaxAuthTries somehow if I needed to do it then). That was less important to me though, for suddenly my DSL line was ping-flooded with what must have been oversized packets, and I went dark for a few seconds. Startled the bejeziz out of me at the time though, bad day, won't forget it. :) > > Dru Rocket- .ike From jlam at pkgsrc.org Wed Sep 13 14:23:54 2006 From: jlam at pkgsrc.org (Johnny Lam) Date: Wed, 13 Sep 2006 14:23:54 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: <45084CBA.80106@pkgsrc.org> David Lawson wrote: > > My experience has been that a passphrase protected ssh key with a > management agent (SSHKeychain in my case), makes managing secure > access to large numbers of machines vastly, vastly simpler than it > would be using passwords. Some of that, I think, will vary > depending on your working environment and needs, but in general I've > become a huge fan of keys and agent forwarding over the last few > years, so personally I can't really think of a good argument > _against_ using keys to do authentication, though I'd be interested > to hear one if one exists. Given the way that ssh-agent works (using sockets in /tmp/ssh-XXXXXXX), the disadvantage is that you have to *really* trust every intermediate machine through which you do agent forwarding. This is because anyone with root access on any machine through which you do agent forwarding can simply use your forwarded credentials because he can access that socket file. I personally do use agent forwarding, but with the above understanding about trust. Cheers, -- Johnny Lam From dlavigne6 at sympatico.ca Wed Sep 13 14:37:12 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Wed, 13 Sep 2006 14:37:12 -0400 (EDT) Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> <044D5671-B462-47F2-832F-97CD4ABFE129@lesmuug.org> <20060913134517.P624@dru.domain.org> Message-ID: <20060913143215.Y624@dru.domain.org> On Wed, 13 Sep 2006, Isaac Levy wrote: >> Aaah, but isn't that the rub in security? Security after all is a myth, or >> at best, an arms race where you have to balance risk and effort :-) > > That statement gets the packet-fu award from me for this summer. I'll pick it up next time I'm in NYC ;-) > I made the mistake of running a traceroute on a particular host which was > part of the attack, using my laptop behind a remote DSL line where I was > ssh'd to the servers. > Within seconds, around 100 ssh auth attempts were made to one mailserver > *spoofing the IP address of my DSL line*, in effect locking me out of future > SSH connections based on 'MaxAuthTries' in sshd_conf. > I had a few shells already on that box, so I was able to continue working, > (or I guess I could have figured out how to flush MaxAuthTries somehow if I > needed to do it then). That was less important to me though, for suddenly my > DSL line was ping-flooded with what must have been oversized packets, and I > went dark for a few seconds. > > Startled the bejeziz out of me at the time though, bad day, won't forget it. I'm sure everyone on this list groaned and thought of at least one similar horror story of their own... Here's a totally wild idea: remember the BSD success stories pdf? How about we put together a collection of "my sysadmin horror stories and what I learned from them" and have the PDF launched at NYCBSDCon? I'll volunteer to play editor if we can collect enough stories. Ike, you'd make a great graphics layout person if you have the time and interest. Dru From af.dingo at gmail.com Wed Sep 13 14:43:52 2006 From: af.dingo at gmail.com (Jeff Quast) Date: Wed, 13 Sep 2006 14:43:52 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <044D5671-B462-47F2-832F-97CD4ABFE129@lesmuug.org> References: <20060912095351.4266acc9@wit.genoverly.com> <044D5671-B462-47F2-832F-97CD4ABFE129@lesmuug.org> Message-ID: On 9/13/06, Isaac Levy wrote: > Hi All, > > Some SSH food for thought, yummy > On Sep 12, 2006, at 2:54 PM, csnyder wrote: > > >> I think parsing logs and injecting rules is just plain ridiculous. > >> Especialy using 3rd party languages not native to your OS. Its just > >> more custom stuff to re-implement on the next os rebuild. > > > > Look, I know it's ridiculous, but it's also more portable (for now) > > than pf. > > Forgive my possible naivet?, but how does any ssh/packet-filter > incorporation strategy really secure anything, big picture > (regardless of the implementation)? > > What happens when ssh passwords come under distributed dictionary > attack by a botnet (many IP addresses)? Wouldn't it render the > filter moot, and perhaps even create a resource attack as a side > effect of dynamically loading gargantuan filter rulesets? your pf rule for always allowing pass in from your administrative machine would still alow you to log in. Your pf rule for allowing only so many connections per state per time, and only so many global connections globally would prevent this botnet from attempting any more than X per Y seconds. Serial ports are found on all server equipment for a good reason. Let sshd crash from resource exhaustion. You have other things to worry about at this point, but ssh is not the only point of entry from an administrator viewpoint. > What happens when an attacker spoofs the IP addresses you use, with > the effect of blocking you from your own systems? That wouldn't happen with the option i recommended. Its a stateful tracking option. You can't create state with a spoofed IP (syn+ack networking 101). > -- > Additionally, what happens when SSH itself meets it's inevitable zero- > day (could be tomorrow, could be 50 years from now)? Doesn't any > complicated intermingling with other parts of the system make ssh > that much more difficult and error prone to replace quickly? ssh is extremely easy to upgrade. If you're so paranoid then you're subscribed to all of the announcement lists? development lists? Your OS manages your software well, doesn't it? > I'm not lookin' to pick a flame-fight, I'm just discussing, and I > feel many packet-filter strategies give a false sense of security. > Convince me it's a sane strategy, and I'll likely go implement it > tomorrow :) Whats your alternative? Use telnet with no packet filter? > Rocket- > .ike I repeat, brute forcing is nothing new at all. There are very very very simple solutions to enforce strong passwords, I already listed these. Now this discussion has wandered off into brute forcing ssh keys. I would like to see this done successfully for even a smaller than default keysize. Brute forcing an ssh key would take a very strong machine a very long time to crack locally. Now add tcp/ip and the Internet to this, and we're talking several lifetimes... make a botnet and we're talking a single lifetime. Then we're talking a filled /var from the size of the logfiles of bad guesses. Make a botnet to guess stupidly easy and predictable passwords and you have thousands of more poorly administered machines to add to your botnet. That they were running ssh is of no correlation, they would have joined a botnet sooner or later (or may be part of many!). I've done an nmap on a few dozen of these "brute forcers" out of curiosity. I have found most of these machines to have telnet open, vulnerable sendmail versions, redhat linux from the mid-90's, vulnerable ftp servers with anonymous upload access (sometimes used as warez sites). Even stupidly old sun os running network services I haven't seen or heard of for nearly a decade. That these machines had an easily guessable password and were hacked in this method is totally unrelated. I have yet to see a secure or modern BSD machine as a victim. All but one (the SunOS) that I could identify were linux machines of old or poor distributions. I suspect many of these were not hacked through ssh passwords, they are often acting as web servers with commonly vulnerable php software etc. For some of these I have been able to find a contact and emailed or called to let them know. I have only done this a dozen times or so over the years, because as a result sometimes I am suddenly suspected as the person who broke into their box, or I get flood of mails from corporate management asking me how to fix it and how it happened etc. etc. Being nice can take too much time. I haven't done this in over a year now, I'm not so sure I'd do it again since the last time I was damn near made at fault. I really truly think it is not ssh's responsibility to have a "prevent brute forcing" option of some kind. The ssh daemon running while uid 0 almost does more than I am comfortable with as it is. Outside of an authenticated user's context (and so setuid()'d), I wouldn't like to see any more features added at all. There is snort, john the ripper, a plethora of unix tools to automate it all and communicate results and reporting. Advanced firewalls can limit heavy distributed brute forcing to a trickle, making the possibility of bruting within your lifetime a feat. Your pager and cellphone could go off long before they get the chance. There is disabling passwords all together, there are ssh keys+passphrases, there are ssh agents for every OS I've ever used that make keys easy to manage despite what others have said here. When I log in to my xdm I have it set to run ssh-add (and get the nifty x11 frontend!) for several keys. ssh-add -x locks those keys when I'm not around. I forward these keys to trusted machines. I type my passphrases only once in a great while. Like three times a week per key, yet I use them several times a day for authenticating. 10 minutes of work has made managing ssh accounts extraordinarily easy for me. As for worrying about a zero-day, sshd is the least of your problems. Using an OS with strong stack protection and memory allocation protections, many potential sshd vulnerabilities are thwarted, and the developers of ssh, though a few small embarrassing problems early on, have taken great care to write correct code. Some software code (or disassembly) is looked at by dozens of experienced hackers every day. ssh is no exception, having your own back-door hack to sshd is a bit of a holy grail these days. Since there seems to be so much discussion and interest on ssh, I recommend the O'Reilly book on it, http://www.oreilly.com/catalog/sshtdg/ jdq From nycbug at chrisbuechler.com Wed Sep 13 17:26:16 2006 From: nycbug at chrisbuechler.com (Chris Buechler) Date: Wed, 13 Sep 2006 17:26:16 -0400 Subject: [nycbug-talk] ipfw, ipf, pf comparison matrix In-Reply-To: References: <20060908174233.GF25206@cybertron.cyth.net> Message-ID: <45087778.7090709@chrisbuechler.com> Isaac Levy wrote: > So I've gotten some polite offlist replies to this already, > basically, I've been told the list is fairly out of date, especially > where PF is concerned. > Since Scott and I created that matrix, we realized we botched up a couple things and we never updated the presentation that Ike pulled that from. Actually we realized I think one of them during the presentation, and have caught at least a couple more since then. Creating that was a pretty adventurous undertaking, and we started it entirely too soon before the presentation (we didn't start that part until we were in Ottawa). pf has changed since then as well (and probably even more so because we were pulling features from the FreeBSD man pages for all that, and OpenBSD pf is always a bit ahead of the FreeBSD port). I'm swamped right now, but if anyone has any corrections, I'll gladly make note of them and update that matrix accordingly as soon as I have time. Cheers, -Chris From jonathan at kc8onw.net Wed Sep 13 19:00:52 2006 From: jonathan at kc8onw.net (Jonathan Stewart) Date: Wed, 13 Sep 2006 19:00:52 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] Message-ID: <45088DA4.6020208@kc8onw.net> Hello all, I set up a new server recently and transferred all the information from my old server over. I tried to use unison to synchronize the backup of pictures I have taken and noticed that a large number of pictures where marked as changed on the server. After checking the pictures by hand I confirmed that many of the pictures on the server were corrupted. I attempted to use unison to update the files on the server with the correct local copies but it would fail on almost all the files with the message "destination updated during synchronization." It appears the corruption happens during the read process because when I recompare the files in a graphical diff tool between cache flushes the differences move around!?!?!? The differences also appear to be very small for the most part, single bytes scattered throughout the file. I really have no idea what is causing the problem and would like to pin it down so I can either replace hardware if it's bad or fix whatever the bug is. The problem appears no matter how I read the file, unison, md5, etc. 1 out of maybe 100 times it will read correctly. I have another drive that I use for the OS and I have done many buildworlds/kernels without problems on that drive as well as compiling some very large software packages. I'm wondering if a possible cause is the controller ignoring read errors from the hard drive but I would think more than the occasional single byte would be changed? I cvsup-ed and rebuilt world and kernel recently hoping that it had been fixed but with no luck. I have not seen any error messages on the console at all either. I have a pair of 320GB SATA hard drives setup as RAID0 on a HighPoint RocketRaid 1520 card. The card BIOS is the latest revision as is the motherboard BIOS. This being a data corruption issue I can afford any amount of downtime needed for trouble shooting as it's not very useful to have the server up if everything is going to get corrupted. I'm thinking about maybe trying to dd the file from the raw device in an attempt to see if the problem is occurring in the filesystem code or is lower level yet. Any suggestions on how to locate the file on the disk or how to isolate the problem better are welcome. I don't mind doing the work I just have no idea where to look/what to try next. Thank you if you actually read all this :), Jonathan uname -a: FreeBSD XXXXX 6.1-STABLE FreeBSD 6.1-STABLE #0: Sun Sep 10 22:54:17 EDT 2006 root at XXXXX:/usr/obj/usr/src/sys/SERVER i386 dmesg: Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.1-STABLE #0: Sun Sep 10 22:54:17 EDT 2006 root at XXXXX:/usr/obj/usr/src/sys/SERVER mptable_probe: MP Config Table has bad signature: 4\^C\^_ Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Athlon(tm) XP 3200+ (2090.16-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x6a0 Stepping = 0 Features=0x383fbff AMD Features=0xc0400800 real memory = 1073676288 (1023 MB) avail memory = 1041698816 (993 MB) kbd1 at kbdmux0 ath_hal: 0.9.17.2 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 cpu0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 Correcting nForce2 C1 CPU disconnect hangs agp0: mem 0xd8000000-0xdbffffff at device 0.0 on pci0 pci0: at device 0.1 (no driver attached) pci0: at device 0.2 (no driver attached) pci0: at device 0.3 (no driver attached) pci0: at device 0.4 (no driver attached) pci0: at device 0.5 (no driver attached) isab0: at device 1.0 on pci0 isa0: on isab0 pci0: at device 1.1 (no driver attached) ohci0: mem 0xe1085000-0xe1085fff irq 5 at device 2.0 on pci0 ohci0: [GIANT-LOCKED] usb0: OHCI version 1.0, legacy support usb0: on ohci0 usb0: USB revision 1.0 uhub0: nVidia OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 3 ports with 3 removable, self powered ohci1: mem 0xe1082000-0xe1082fff irq 5 at device 2.1 on pci0 ohci1: [GIANT-LOCKED] usb1: OHCI version 1.0, legacy support usb1: on ohci1 usb1: USB revision 1.0 uhub1: nVidia OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 3 ports with 3 removable, self powered ehci0: mem 0xe1083000-0xe10830ff irq 12 at device 2.2 on pci0 ehci0: [GIANT-LOCKED] usb2: EHCI version 1.0 usb2: companion controllers, 4 ports each: usb0 usb1 usb2: on ehci0 usb2: USB revision 2.0 uhub2: nVidia EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub2: 6 ports with 6 removable, self powered nve0: port 0xe400-0xe407 mem 0xe1084000-0xe1084fff irq 12 at device 4.0 on pci0 nve0: Ethernet address 00:0c:6e:7d:e0:79 miibus0: on nve0 rlphy0: on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto nve0: Ethernet address: 00:0c:6e:7d:e0:79 pci0: at device 5.0 (no driver attached) pci0: at device 6.0 (no driver attached) pcib1: at device 8.0 on pci0 pci1: on pcib1 atapci0: port 0xa000-0xa007,0xa400-0xa403,0xa800-0xa807,0xac00-0xac03,0xb000-0xb0ff irq 11 at device 6.0 on pci1 ata2: on atapci0 ata3: on atapci0 pci1: at device 9.0 (no driver attached) pci1: at device 9.1 (no driver attached) atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 9.0 on pci0 ata0: on atapci1 ata1: on atapci1 pcib2: at device 12.0 on pci0 pci2: on pcib2 xl0: <3Com 3c920B-EMB Integrated Fast Etherlink XL> port 0xc000-0xc07f mem 0xdd000000-0xdd00007f irq 5 at device 1.0 on pci2 miibus1: on xl0 acphy0: on miibus1 acphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto xl0: Ethernet address: 00:26:54:10:8c:0f pcib3: at device 30.0 on pci0 pci3: on pcib3 pci3: at device 0.0 (no driver attached) fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A ppc0: port 0x378-0x37f,0x778-0x77b irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/16 bytes threshold ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 pmtimer0 on isa0 orm0: at iomem 0xd0000-0xd17ff,0xd6000-0xd67ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 2090164914 Hz quality 800 Timecounters tick every 1.000 msec ad0: 194481MB at ata0-master UDMA133 acd0: DVDROM at ata0-slave UDMA33 ad4: 305245MB at ata2-master UDMA133 ad6: 305245MB at ata3-master UDMA133 ar0: 610490MB status: READY ar0: disk0 READY using ad4 at ata2-master ar0: disk1 READY using ad6 at ata3-master Trying to mount root from ufs:/dev/ad0s1a _______________________________________________ freebsd-stable at freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org" From ike at lesmuug.org Wed Sep 13 20:50:25 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 13 Sep 2006 20:50:25 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <45088DA4.6020208@kc8onw.net> References: <45088DA4.6020208@kc8onw.net> Message-ID: Hi Jonathan, What timing. I've just run into the same reproducible problem (I'm having flashbacks to 5.x and sweating and gnashing my teeth). On Sep 13, 2006, at 7:00 PM, Jonathan Stewart wrote: > Hello all, > > I set up a new server recently and transferred all the information > from > my old server over. I tried to use unison to synchronize the > backup of > pictures I have taken and noticed that a large number of pictures > where > marked as changed on the server. After checking the pictures by > hand I > confirmed that many of the pictures on the server were corrupted. I > attempted to use unison to update the files on the server with the > correct local copies but it would fail on almost all the files with > the > message "destination updated during synchronization." > > It appears the corruption happens during the read process because > when I > recompare the files in a graphical diff tool between cache flushes the > differences move around!?!?!? The differences also appear to be very > small for the most part, single bytes scattered throughout the > file. I > really have no idea what is causing the problem and would like to > pin it > down so I can either replace hardware if it's bad or fix whatever the > bug is. > > The problem appears no matter how I read the file, unison, md5, > etc. 1 > out of maybe 100 times it will read correctly. I have another drive > that I use for the OS and I have done many buildworlds/kernels without > problems on that drive as well as compiling some very large software > packages. I'm wondering if a possible cause is the controller > ignoring > read errors from the hard drive but I would think more than the > occasional single byte would be changed? I've narrowed it down to the SATA drivers, something has changed since I burned the 6.1 bootonly media (tried installing from both freebsd.nycbug.org and ftp.freebsd.org to the same effect). When I cvsup the STABLE branch, the kernel seems to totally freak out on me. I originally thought it was the buildworld process, but after simply installing the new kernel and rebooting, (su mode or not), I get screens full of disk read errors; mostly: "error issuing READ_DMA command" From your DMESG: > ad4: 305245MB at ata2-master UDMA133 Ok- as soon as I get this box back up again, I'll post my dmesg as well for comparison. This box was ruunning fine for about a week with the 6.1 install media, so for me I'm just going to wipe it and install from ftp binaries once more. If this gets messy, I'll try to snag a spare SATA drive and replicate the problem again, but for now I have to get this box back to work... Best, .ike From ike at lesmuug.org Wed Sep 13 21:00:37 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 13 Sep 2006 21:00:37 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <45088DA4.6020208@kc8onw.net> References: <45088DA4.6020208@kc8onw.net> Message-ID: <36CDDE61-042B-4491-BADE-F85FC0C19597@lesmuug.org> Sidenote, Since everyone may get annoyed with this issue, it may be good to hit the FreeBSD-STABLE mailing list, here: http://lists.freebsd.org/mailman/listinfo/freebsd-stable And info about the STABLE branch can be found here: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current- stable.html#STABLE Rocket- .ike From ike at lesmuug.org Wed Sep 13 21:23:44 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 13 Sep 2006 21:23:44 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <45088DA4.6020208@kc8onw.net> References: <45088DA4.6020208@kc8onw.net> Message-ID: Hi Jonathan, Here's my dmesg, (posted in full below yours), you have: > ad4: 305245MB at ata2-master UDMA133 > ad6: 305245MB at ata3-master UDMA133 I have: ad4: 152627MB at ata2-master UDMA33 I believe it's the ad driver, something has changed. Lets take this over to the STABLE list? Best, .ike On Sep 13, 2006, at 7:00 PM, Jonathan Stewart wrote: > uname -a: > FreeBSD XXXXX 6.1-STABLE FreeBSD 6.1-STABLE #0: Sun Sep 10 22:54:17 > EDT > 2006 root at XXXXX:/usr/obj/usr/src/sys/SERVER i386 > > dmesg: > Copyright (c) 1992-2006 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, > 1994 > The Regents of the University of California. All rights > reserved. > FreeBSD 6.1-STABLE #0: Sun Sep 10 22:54:17 EDT 2006 > root at XXXXX:/usr/obj/usr/src/sys/SERVER > mptable_probe: MP Config Table has bad signature: 4\^C\^_ > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: AMD Athlon(tm) XP 3200+ (2090.16-MHz 686-class CPU) > Origin = "AuthenticAMD" Id = 0x6a0 Stepping = 0 > > Features=0x383fbff E,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE> > AMD Features=0xc0400800 > real memory = 1073676288 (1023 MB) > avail memory = 1041698816 (993 MB) > kbd1 at kbdmux0 > ath_hal: 0.9.17.2 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, > RF5413) > acpi0: on motherboard > acpi0: Power Button (fixed) > Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 > cpu0: on acpi0 > acpi_button0: on acpi0 > pcib0: port 0xcf8-0xcff on acpi0 > pci0: on pcib0 > Correcting nForce2 C1 CPU disconnect hangs > agp0: mem 0xd8000000-0xdbffffff at > device 0.0 on pci0 > pci0: at device 0.1 (no driver attached) > pci0: at device 0.2 (no driver attached) > pci0: at device 0.3 (no driver attached) > pci0: at device 0.4 (no driver attached) > pci0: at device 0.5 (no driver attached) > isab0: at device 1.0 on pci0 > isa0: on isab0 > pci0: at device 1.1 (no driver attached) > ohci0: mem 0xe1085000-0xe1085fff irq 5 > at device 2.0 on pci0 > ohci0: [GIANT-LOCKED] > usb0: OHCI version 1.0, legacy support > usb0: on ohci0 > usb0: USB revision 1.0 > uhub0: nVidia OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 > uhub0: 3 ports with 3 removable, self powered > ohci1: mem 0xe1082000-0xe1082fff irq 5 > at device 2.1 on pci0 > ohci1: [GIANT-LOCKED] > usb1: OHCI version 1.0, legacy support > usb1: on ohci1 > usb1: USB revision 1.0 > uhub1: nVidia OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 > uhub1: 3 ports with 3 removable, self powered > ehci0: mem > 0xe1083000-0xe10830ff irq > 12 at device 2.2 on pci0 > ehci0: [GIANT-LOCKED] > usb2: EHCI version 1.0 > usb2: companion controllers, 4 ports each: usb0 usb1 > usb2: on ehci0 > usb2: USB revision 2.0 > uhub2: nVidia EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 > uhub2: 6 ports with 6 removable, self powered > nve0: port 0xe400-0xe407 mem > 0xe1084000-0xe1084fff irq 12 at device 4.0 on pci0 > nve0: Ethernet address 00:0c:6e:7d:e0:79 > miibus0: on nve0 > rlphy0: on miibus0 > rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > nve0: Ethernet address: 00:0c:6e:7d:e0:79 > pci0: at device 5.0 (no driver attached) > pci0: at device 6.0 (no driver attached) > pcib1: at device 8.0 on pci0 > pci1: on pcib1 > atapci0: port > 0xa000-0xa007,0xa400-0xa403,0xa800-0xa807,0xac00-0xac03,0xb000-0xb0ff > irq 11 at device 6.0 on pci1 > ata2: on atapci0 > ata3: on atapci0 > pci1: at device 9.0 (no driver attached) > pci1: at device 9.1 (no driver attached) > atapci1: port > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 9.0 on > pci0 > ata0: on atapci1 > ata1: on atapci1 > pcib2: at device 12.0 on pci0 > pci2: on pcib2 > xl0: <3Com 3c920B-EMB Integrated Fast Etherlink XL> port 0xc000-0xc07f > mem 0xdd000000-0xdd00007f irq 5 at device 1.0 on pci2 > miibus1: on xl0 > acphy0: on miibus1 > acphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > xl0: Ethernet address: 00:26:54:10:8c:0f > pcib3: at device 30.0 on pci0 > pci3: on pcib3 > pci3: at device 0.0 (no driver attached) > fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 > on acpi0 > fdc0: does not respond > device_attach: fdc0 attach returned 6 > sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags > 0x10 on > acpi0 > sio0: type 16550A > sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 > sio1: type 16550A > ppc0: port 0x378-0x37f,0x778-0x77b irq > 7 drq > 3 on acpi0 > ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode > ppc0: FIFO with 16/16/16 bytes threshold > ppbus0: on ppc0 > plip0: on ppbus0 > lpt0: on ppbus0 > lpt0: Interrupt-driven port > ppi0: on ppbus0 > atkbdc0: port 0x60,0x64 irq 1 on acpi0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 > on acpi0 > fdc0: does not respond > device_attach: fdc0 attach returned 6 > pmtimer0 on isa0 > orm0: at iomem 0xd0000-0xd17ff,0xd6000-0xd67ff on > isa0 > sc0: at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=0x300> > vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff > on isa0 > Timecounter "TSC" frequency 2090164914 Hz quality 800 > Timecounters tick every 1.000 msec > ad0: 194481MB at ata0-master UDMA133 > acd0: DVDROM at ata0-slave UDMA33 > ad4: 305245MB at ata2-master UDMA133 > ad6: 305245MB at ata3-master UDMA133 > ar0: 610490MB > status: READY > ar0: disk0 READY using ad4 at ata2-master > ar0: disk1 READY using ad6 at ata3-master > Trying to mount root from ufs:/dev/ad0s1a $ uname -a && dmesg FreeBSD monkey.jmg.lan 6.1-RELEASE FreeBSD 6.1-RELEASE #0: Sun May 7 04:42:56 UTC 2006 root at opus.cse.buffalo.edu:/usr/obj/usr/src/sys/ SMP i386 Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.1-RELEASE #0: Sun May 7 04:32:43 UTC 2006 root at opus.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC ACPI APIC Table: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) D CPU 3.00GHz (3000.14-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf62 Stepping = 2 Features=0xbfebfbff Features2=0xe43d,> AMD Features=0x20000000 AMD Features2=0x1 Cores per package: 2 real memory = 2147155968 (2047 MB) avail memory = 2092068864 (1995 MB) ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 acpi_throttle0: on cpu0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: at device 0.5 (no driver attached) pcib1: at device 1.0 on pci0 pci1: on pcib1 pcib2: irq 27 at device 2.0 on pci0 pci2: on pcib2 pci2: at device 0.0 (no driver attached) atapci0: port 0xdc00-0xdc07,0xd880-0xd883,0xd800-0xd807,0xd480-0xd483,0xd400-0xd40f, 0xd000-0xd0ff irq 21 at device 15.0 on pci0 ata2: on atapci0 ata3: on atapci0 atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 15.1 on pci0 ata0: on atapci1 ata1: on atapci1 uhci0: port 0xcc00-0xcc1f irq 20 at device 16.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xc880-0xc89f irq 22 at device 16.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xc800-0xc81f irq 21 at device 16.2 on pci0 uhci2: [GIANT-LOCKED] usb2: on uhci2 usb2: USB revision 1.0 uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered uhci3: port 0xc480-0xc49f irq 23 at device 16.3 on pci0 uhci3: [GIANT-LOCKED] usb3: on uhci3 usb3: USB revision 1.0 uhub3: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xf8fffc00-0xf8fffcff irq 21 at device 16.4 on pci0 ehci0: [GIANT-LOCKED] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub4: 8 ports with 8 removable, self powered isab0: at device 17.0 on pci0 isa0: on isab0 vr0: port 0xc000-0xc0ff mem 0xf8fff800-0xf8fff8ff irq 23 at device 18.0 on pci0 miibus0: on vr0 rlphy0: on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto vr0: Ethernet address: 00:17:31:ee:7f:cb pcib3: at device 19.0 on pci0 pci3: on pcib3 pci3: at device 1.0 (no driver attached) pcib4: at device 19.1 on pci0 pci4: on pcib4 re0: port 0xe800-0xe8ff mem 0xfbfffc00-0xfbfffcff irq 17 at device 5.0 on pci4 miibus1: on re0 rgephy0: on miibus1 rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto re0: Ethernet address: 00:14:6c:c0:14:cc re1: port 0xe400-0xe4ff mem 0xfbfff800-0xfbfff8ff irq 18 at device 6.0 on pci4 miibus2: on re1 rgephy1: on miibus2 rgephy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto re1: Ethernet address: 00:14:6c:c0:13:29 acpi_button0: on acpi0 acpi_button1: on acpi0 ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A pmtimer0 on isa0 orm0: at iomem 0xc0000-0xcefff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 3000143580 Hz quality 800 Timecounters tick every 1.000 msec md0: Preloaded image 4423680 bytes at 0xc0a91928 acd0: DVDR at ata0-slave UDMA33 ad4: 152627MB at ata2-master UDMA33 Trying to mount root from ufs:/dev/md0 Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.1-RELEASE #0: Sun May 7 04:32:43 UTC 2006 root at opus.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC ACPI APIC Table: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) D CPU 3.00GHz (3000.14-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf62 Stepping = 2 Features=0xbfebfbff Features2=0xe43d,> AMD Features=0x20000000 AMD Features2=0x1 Cores per package: 2 real memory = 2147155968 (2047 MB) avail memory = 2092068864 (1995 MB) ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 acpi_throttle0: on cpu0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: at device 0.5 (no driver attached) pcib1: at device 1.0 on pci0 pci1: on pcib1 pcib2: irq 27 at device 2.0 on pci0 pci2: on pcib2 pci2: at device 0.0 (no driver attached) atapci0: port 0xdc00-0xdc07,0xd880-0xd883,0xd800-0xd807,0xd480-0xd483,0xd400-0xd40f, 0xd000-0xd0ff irq 21 at device 15.0 on pci0 ata2: on atapci0 ata3: on atapci0 atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 15.1 on pci0 ata0: on atapci1 ata1: on atapci1 uhci0: port 0xcc00-0xcc1f irq 20 at device 16.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xc880-0xc89f irq 22 at device 16.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xc800-0xc81f irq 21 at device 16.2 on pci0 uhci2: [GIANT-LOCKED] usb2: on uhci2 usb2: USB revision 1.0 uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered uhci3: port 0xc480-0xc49f irq 23 at device 16.3 on pci0 uhci3: [GIANT-LOCKED] usb3: on uhci3 usb3: USB revision 1.0 uhub3: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xf8fffc00-0xf8fffcff irq 21 at device 16.4 on pci0 ehci0: [GIANT-LOCKED] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub4: 8 ports with 8 removable, self powered isab0: at device 17.0 on pci0 isa0: on isab0 vr0: port 0xc000-0xc0ff mem 0xf8fff800-0xf8fff8ff irq 23 at device 18.0 on pci0 miibus0: on vr0 rlphy0: on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto vr0: Ethernet address: 00:17:31:ee:7f:cb pcib3: at device 19.0 on pci0 pci3: on pcib3 pci3: at device 1.0 (no driver attached) pcib4: at device 19.1 on pci0 pci4: on pcib4 re0: port 0xe800-0xe8ff mem 0xfbfffc00-0xfbfffcff irq 17 at device 5.0 on pci4 miibus1: on re0 rgephy0: on miibus1 rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto re0: Ethernet address: 00:14:6c:c0:14:cc re1: port 0xe400-0xe4ff mem 0xfbfff800-0xfbfff8ff irq 18 at device 6.0 on pci4 miibus2: on re1 rgephy1: on miibus2 rgephy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto re1: Ethernet address: 00:14:6c:c0:13:29 acpi_button0: on acpi0 acpi_button1: on acpi0 ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A pmtimer0 on isa0 orm0: at iomem 0xc0000-0xcefff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 3000140205 Hz quality 800 Timecounters tick every 1.000 msec md0: Preloaded image 4423680 bytes at 0xc0a91928 acd0: DVDR at ata0-slave UDMA33 ad4: 152627MB at ata2-master UDMA33 Trying to mount root from ufs:/dev/md0 Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining...10 8 0 0 0 done All buffers synced. Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.1-RELEASE #0: Sun May 7 04:42:56 UTC 2006 root at opus.cse.buffalo.edu:/usr/obj/usr/src/sys/SMP Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) D CPU 3.00GHz (3000.13-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf62 Stepping = 2 Features=0xbfebfbff Features2=0xe43d,> AMD Features=0x20000000 AMD Features2=0x1 Cores per package: 2 real memory = 2147155968 (2047 MB) avail memory = 2096103424 (1999 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 acpi_throttle0: on cpu0 cpu1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: at device 0.5 (no driver attached) pcib1: at device 1.0 on pci0 pci1: on pcib1 pcib2: irq 27 at device 2.0 on pci0 pci2: on pcib2 pci2: at device 0.0 (no driver attached) atapci0: port 0xdc00-0xdc07,0xd880-0xd883,0xd800-0xd807,0xd480-0xd483,0xd400-0xd40f, 0xd000-0xd0ff irq 21 at device 15.0 on pci0 ata2: on atapci0 ata3: on atapci0 atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 15.1 on pci0 ata0: on atapci1 ata1: on atapci1 uhci0: port 0xcc00-0xcc1f irq 20 at device 16.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xc880-0xc89f irq 22 at device 16.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xc800-0xc81f irq 21 at device 16.2 on pci0 uhci2: [GIANT-LOCKED] usb2: on uhci2 usb2: USB revision 1.0 uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered uhci3: port 0xc480-0xc49f irq 23 at device 16.3 on pci0 uhci3: [GIANT-LOCKED] usb3: on uhci3 usb3: USB revision 1.0 uhub3: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xf8fffc00-0xf8fffcff irq 21 at device 16.4 on pci0 ehci0: [GIANT-LOCKED] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub4: 8 ports with 8 removable, self powered isab0: at device 17.0 on pci0 isa0: on isab0 vr0: port 0xc000-0xc0ff mem 0xf8fff800-0xf8fff8ff irq 23 at device 18.0 on pci0 miibus0: on vr0 rlphy0: on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto vr0: Ethernet address: 00:17:31:ee:7f:cb pcib3: at device 19.0 on pci0 pci3: on pcib3 pci3: at device 1.0 (no driver attached) pcib4: at device 19.1 on pci0 pci4: on pcib4 re0: port 0xe800-0xe8ff mem 0xfbfffc00-0xfbfffcff irq 17 at device 5.0 on pci4 miibus1: on re0 rgephy0: on miibus1 rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto re0: Ethernet address: 00:14:6c:c0:14:cc re1: port 0xe400-0xe4ff mem 0xfbfff800-0xfbfff8ff irq 18 at device 6.0 on pci4 miibus2: on re1 rgephy1: on miibus2 rgephy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto re1: Ethernet address: 00:14:6c:c0:13:29 acpi_button0: on acpi0 acpi_button1: on acpi0 ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A pmtimer0 on isa0 orm0: at iomem 0xc0000-0xcefff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec acd0: DVDR at ata0-slave UDMA33 ad4: 152627MB at ata2-master UDMA33 SMP: AP CPU #1 Launched! Trying to mount root from ufs:/dev/ad4s1a $ From george at galis.org Wed Sep 13 21:53:12 2006 From: george at galis.org (George Georgalis) Date: Wed, 13 Sep 2006 21:53:12 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <044D5671-B462-47F2-832F-97CD4ABFE129@lesmuug.org> References: <20060912095351.4266acc9@wit.genoverly.com> <044D5671-B462-47F2-832F-97CD4ABFE129@lesmuug.org> Message-ID: <20060914015312.GI3349@run.galis.org> On Wed, Sep 13, 2006 at 01:25:22PM -0400, Isaac Levy wrote: >Hi All, > >Some SSH food for thought, > >On Sep 12, 2006, at 2:54 PM, csnyder wrote: > >>> I think parsing logs and injecting rules is just plain ridiculous. >>> Especialy using 3rd party languages not native to your OS. Its just >>> more custom stuff to re-implement on the next os rebuild. >> >> Look, I know it's ridiculous, but it's also more portable (for now) >> than pf. > >Forgive my possible naivet?, but how does any ssh/packet-filter >incorporation strategy really secure anything, big picture >(regardless of the implementation)? > >What happens when ssh passwords come under distributed dictionary >attack by a botnet (many IP addresses)? Wouldn't it render the >filter moot, and perhaps even create a resource attack as a side >effect of dynamically loading gargantuan filter rulesets? > >What happens when an attacker spoofs the IP addresses you use, with >the effect of blocking you from your own systems? UsePam No lets you login from anywhere, using something you have and something you know. and, ever try connecting without your private key? good luck. // George -- George Georgalis, systems architect, administrator < From jonathan at kc8onw.net Thu Sep 14 07:20:10 2006 From: jonathan at kc8onw.net (Jonathan Stewart) Date: Thu, 14 Sep 2006 07:20:10 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: References: <45088DA4.6020208@kc8onw.net> Message-ID: <45093AEA.5010501@kc8onw.net> Isaac Levy wrote: > Hi Jonathan, > > Here's my dmesg, (posted in full below yours), > > you have: >> ad4: 305245MB at ata2-master UDMA133 >> ad6: 305245MB at ata3-master UDMA133 > > I have: > ad4: 152627MB at ata2-master UDMA33 > > I believe it's the ad driver, something has changed. Lets take this > over to the STABLE list? I posted twice about it on the STABLE list already so hopefully someone else posting about it will generate a bit more response. My second thread starts at [1] I have a second post in that thread with more detail. I don't get any errors on the console though :P Maybe because I'm going through a PCI RAID card? If you want I can write up the list of steps I took to pin down the problem as much as I could. Thanks, Jonathan P.S. How did you get your dmesg because there where a couple of them together? I normally just use /var/run/dmesg.boot [1] http://lists.freebsd.org/pipermail/freebsd-stable/2006-September/028293.html From dan at langille.org Thu Sep 14 07:43:41 2006 From: dan at langille.org (Dan Langille) Date: Thu, 14 Sep 2006 07:43:41 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <45093AEA.5010501@kc8onw.net> References: Message-ID: <4509082D.31915.54B803D@dan.langille.org> On 14 Sep 2006 at 7:20, Jonathan Stewart wrote: > P.S. How did you get your dmesg because there where a couple of them > together? I normally just use /var/run/dmesg.boot Presumably via dmesg(8), which can append information as the system keeps running. /var/run/dmesg.boot is always the output from the most recent boot. -- Dan Langille : Software Developer looking for work my resume: http://www.freebsddiary.org/dan_langille.php From george at galis.org Thu Sep 14 07:48:41 2006 From: george at galis.org (George Georgalis) Date: Thu, 14 Sep 2006 07:48:41 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060913143717.GP25801@clam.khaoz.org> References: <20060912095351.4266acc9@wit.genoverly.com> <20060912122953.76ac705b@wit.genoverly.com> <20060912172451.GE15283@run.galis.org> <20060913143717.GP25801@clam.khaoz.org> Message-ID: <20060914114841.GJ3349@run.galis.org> On Wed, Sep 13, 2006 at 10:37:17AM -0400, Okan Demirmen wrote: >On Tue 2006.09.12 at 13:24 -0400, George Georgalis wrote: >> There was some resolution (at openbsd I think) to encrypt >> the known_hosts entries with the remote host public key; >> so if your authentication was compromised, at least there >> wouldn't be a list a hosts for the attacker to look up. >> But I've not seen it in my OS yet. > >man ssh_config - see HashKnownHosts nice, looks like it is in my upgrade path. has there been any discussion of hashing .ssh/config? maybe requiring a private key and passphrase/agent? // George -- George Georgalis, systems architect, administrator < From george at galis.org Thu Sep 14 08:10:31 2006 From: george at galis.org (George Georgalis) Date: Thu, 14 Sep 2006 08:10:31 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <45088DA4.6020208@kc8onw.net> References: <45088DA4.6020208@kc8onw.net> Message-ID: <20060914121031.GK3349@run.galis.org> On Wed, Sep 13, 2006 at 07:00:52PM -0400, Jonathan Stewart wrote: >ad4: 305245MB at ata2-master UDMA133 >ad6: 305245MB at ata3-master UDMA133 Sorry I didn't take a close look at your issue, but please check sector 0xfffffff and 0x10000000. the on their larger (?) drives seagate doc says use LBA32 for sectors <= 0xfffffff and LBA48 for sectors >0xfffffff when the drives actually require LBA32 < 0xfffffff and LBA48 for sectors =>0xfffffff I haven't tested yet, but I think these numbers are correct. In netbsd many seagate drives are in a quirk table, but last I checked my 400 and 500Gb ones where not. // George -- George Georgalis, systems architect, administrator < From jonathan at kc8onw.net Thu Sep 14 08:20:47 2006 From: jonathan at kc8onw.net (Jonathan Stewart) Date: Thu, 14 Sep 2006 08:20:47 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <20060914121031.GK3349@run.galis.org> References: <45088DA4.6020208@kc8onw.net> <20060914121031.GK3349@run.galis.org> Message-ID: <4509491F.1070802@kc8onw.net> George Georgalis wrote: > On Wed, Sep 13, 2006 at 07:00:52PM -0400, Jonathan Stewart wrote: >> ad4: 305245MB at ata2-master UDMA133 >> ad6: 305245MB at ata3-master UDMA133 > > Sorry I didn't take a close look at your issue, but please check > sector 0xfffffff and 0x10000000. > > the on their larger (?) drives seagate doc says use LBA32 for > sectors <= 0xfffffff and LBA48 for sectors >0xfffffff when the > drives actually require LBA32 < 0xfffffff and LBA48 for sectors > =>0xfffffff > > I haven't tested yet, but I think these numbers are correct. In > netbsd many seagate drives are in a quirk table, but last I > checked my 400 and 500Gb ones where not. > Did you mean doing something like sudo dd if=/dev/ad4 of=/tmp/temp skip=268435455 count=1 sudo dd if=/dev/ad4 of=/tmp/temp skip=268435456 count=1 sudo dd if=/dev/ad4 of=/tmp/temp2 skip=268435456 count=1 to make sure I can read the sectors and the reads come back the same each time? If so then yes to both. Thanks, Jonathan From lists at genoverly.net Thu Sep 14 08:50:25 2006 From: lists at genoverly.net (michael) Date: Thu, 14 Sep 2006 08:50:25 -0400 Subject: [nycbug-talk] we've been blogged! Message-ID: <20060914085025.5ef2b3c8@wit.genoverly.com> http://blogs.ittoolbox.com/unix/bsd/archives/bsd-bytes-11662 Thanks, Dru. Also, "thank you" to those that have posted to other online sites. -- Michael From george at galis.org Thu Sep 14 09:09:04 2006 From: george at galis.org (George Georgalis) Date: Thu, 14 Sep 2006 09:09:04 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <4509491F.1070802@kc8onw.net> References: <45088DA4.6020208@kc8onw.net> <20060914121031.GK3349@run.galis.org> <4509491F.1070802@kc8onw.net> Message-ID: <20060914130904.GM3349@run.galis.org> On Thu, Sep 14, 2006 at 08:20:47AM -0400, Jonathan Stewart wrote: >George Georgalis wrote: >> On Wed, Sep 13, 2006 at 07:00:52PM -0400, Jonathan Stewart wrote: >>> ad4: 305245MB at ata2-master UDMA133 >>> ad6: 305245MB at ata3-master UDMA133 >> >> Sorry I didn't take a close look at your issue, but please check >> sector 0xfffffff and 0x10000000. >> >> the on their larger (?) drives seagate doc says use LBA32 for >> sectors <= 0xfffffff and LBA48 for sectors >0xfffffff when the >> drives actually require LBA32 < 0xfffffff and LBA48 for sectors >> =>0xfffffff >> >> I haven't tested yet, but I think these numbers are correct. In >> netbsd many seagate drives are in a quirk table, but last I >> checked my 400 and 500Gb ones where not. >> > >Did you mean doing something like >sudo dd if=/dev/ad4 of=/tmp/temp skip=268435455 count=1 >sudo dd if=/dev/ad4 of=/tmp/temp skip=268435456 count=1 >sudo dd if=/dev/ad4 of=/tmp/temp2 skip=268435456 count=1 > >to make sure I can read the sectors and the reads come back the same >each time? If so then yes to both. not sure about the base change or your device names, but if=/dev/rwd0d skip=0xffffffe count=3 is what I had in mind, is ad4 a raw device? // George -- George Georgalis, systems architect, administrator < From jonathan at kc8onw.net Thu Sep 14 09:17:53 2006 From: jonathan at kc8onw.net (Jonathan Stewart) Date: Thu, 14 Sep 2006 09:17:53 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <20060914130904.GM3349@run.galis.org> References: <45088DA4.6020208@kc8onw.net> <20060914121031.GK3349@run.galis.org> <4509491F.1070802@kc8onw.net> <20060914130904.GM3349@run.galis.org> Message-ID: <45095681.1030302@kc8onw.net> George Georgalis wrote: > On Thu, Sep 14, 2006 at 08:20:47AM -0400, Jonathan Stewart wrote: >> Did you mean doing something like >> sudo dd if=/dev/ad4 of=/tmp/temp skip=268435455 count=1 >> sudo dd if=/dev/ad4 of=/tmp/temp skip=268435456 count=1 >> sudo dd if=/dev/ad4 of=/tmp/temp2 skip=268435456 count=1 >> >> to make sure I can read the sectors and the reads come back the same >> each time? If so then yes to both. > > > not sure about the base change > or your device names, but > if=/dev/rwd0d skip=0xffffffe count=3 > is what I had in mind, is ad4 a raw device? No errors, ad4 and ad6 are the 2 base drives used for the RAID stripe. They are on a highpoint rocketraid controller card. I didn't know dd would take hex, learn something new everyday :) Thanks, Jonathan From george at galis.org Thu Sep 14 09:42:06 2006 From: george at galis.org (George Georgalis) Date: Thu, 14 Sep 2006 09:42:06 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <45095681.1030302@kc8onw.net> References: <45088DA4.6020208@kc8onw.net> <20060914121031.GK3349@run.galis.org> <4509491F.1070802@kc8onw.net> <20060914130904.GM3349@run.galis.org> <45095681.1030302@kc8onw.net> Message-ID: <20060914134206.GN3349@run.galis.org> On Thu, Sep 14, 2006 at 09:17:53AM -0400, Jonathan Stewart wrote: >George Georgalis wrote: >> On Thu, Sep 14, 2006 at 08:20:47AM -0400, Jonathan Stewart wrote: >>> Did you mean doing something like >>> sudo dd if=/dev/ad4 of=/tmp/temp skip=268435455 count=1 >>> sudo dd if=/dev/ad4 of=/tmp/temp skip=268435456 count=1 >>> sudo dd if=/dev/ad4 of=/tmp/temp2 skip=268435456 count=1 >>> >>> to make sure I can read the sectors and the reads come back the same >>> each time? If so then yes to both. >> >> >> not sure about the base change >> or your device names, but >> if=/dev/rwd0d skip=0xffffffe count=3 >> is what I had in mind, is ad4 a raw device? > >No errors, ad4 and ad6 are the 2 base drives used for the RAID stripe. >They are on a highpoint rocketraid controller card. Are we talking about the same blocks as if the drives where not in a raid? >I didn't know dd would take hex, learn something new everyday :) It was a nice surprise when I saw somebody do that too. :) // George -- George Georgalis, systems architect, administrator < From jonathan at kc8onw.net Thu Sep 14 09:48:09 2006 From: jonathan at kc8onw.net (Jonathan Stewart) Date: Thu, 14 Sep 2006 09:48:09 -0400 Subject: [nycbug-talk] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <20060914134206.GN3349@run.galis.org> References: <45088DA4.6020208@kc8onw.net> <20060914121031.GK3349@run.galis.org> <4509491F.1070802@kc8onw.net> <20060914130904.GM3349@run.galis.org> <45095681.1030302@kc8onw.net> <20060914134206.GN3349@run.galis.org> Message-ID: <45095D99.8050402@kc8onw.net> George Georgalis wrote: > On Thu, Sep 14, 2006 at 09:17:53AM -0400, Jonathan Stewart wrote: >> George Georgalis wrote: >>> On Thu, Sep 14, 2006 at 08:20:47AM -0400, Jonathan Stewart wrote: >>>> Did you mean doing something like >>>> sudo dd if=/dev/ad4 of=/tmp/temp skip=268435455 count=1 >>>> sudo dd if=/dev/ad4 of=/tmp/temp skip=268435456 count=1 >>>> sudo dd if=/dev/ad4 of=/tmp/temp2 skip=268435456 count=1 >>>> >>>> to make sure I can read the sectors and the reads come back the same >>>> each time? If so then yes to both. >>> >>> not sure about the base change >>> or your device names, but >>> if=/dev/rwd0d skip=0xffffffe count=3 >>> is what I had in mind, is ad4 a raw device? >> No errors, ad4 and ad6 are the 2 base drives used for the RAID stripe. >> They are on a highpoint rocketraid controller card. > > Are we talking about the same blocks as if the drives where not in a raid? Yes, the raid device is ar0 while the drives show up as standard ad devices with this card. Jonathan From trish at bsdunix.net Thu Sep 14 09:07:04 2006 From: trish at bsdunix.net (Trish Lynch) Date: Thu, 14 Sep 2006 09:07:04 -0400 (EDT) Subject: [nycbug-talk] we've been blogged! In-Reply-To: <20060914085025.5ef2b3c8@wit.genoverly.com> References: <20060914085025.5ef2b3c8@wit.genoverly.com> Message-ID: <20060914090425.M7416@daemon.bsdunix.net> On Thu, 14 Sep 2006, michael wrote: > http://blogs.ittoolbox.com/unix/bsd/archives/bsd-bytes-11662 > > Thanks, Dru. > > Also, "thank you" to those that have posted to other online sites. > thats awesome. I wish my personal life had not gotten in the way of a submission this year. Failing health and employer legal issues can be a bitch.... Next Year... or maybe George'll just have to have me come speak again *grin* Its about the only excuse I can use to get away from my family during the week (unless you guys switches the week which the meeting is....) -Trish -- Trish Lynch trish at bsdunix.net Key fingerprint = 781D 2B47 AA4B FC88 B919 0CD6 26B2 1D62 6FC1 FF16 From trish at bsdunix.net Tue Sep 12 15:46:34 2006 From: trish at bsdunix.net (Trish Lynch) Date: Tue, 12 Sep 2006 15:46:34 -0400 (EDT) Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: <20060912154214.D7416@daemon.bsdunix.net> On Tue, 12 Sep 2006, Jeff Quast wrote: > On 9/12/06, csnyder wrote: >>> >>> I am also curious.. where do we draw the line and just *trust* our OS? >>> > > > I just felt the need to reply to the line that this is OpenSSH's > responsability to deal with. It made me mad. They do a great job > dealing with this issue in the place it is meant to be dealt with. > I 100% disagree with this, since OpenSSH is in fact partially responsible for handling the connection and authenticating it, including keys... if its failed to authenticate within OpenSSH, its not any other program or tool's responssibility to handle it. IMO you've got it 100% wrong... but then we can agree to disgaree on this. If OpenSSH wasn;t handling part of the auth layer, I'd agree, but since it does, inclduing what kind of auth you use (key or password) it needs to work for both password and key based auth. OpenSSH is the place to gracefully handle this without having to implement a specific firewall to make it work. > Password authentication should only be used once to add your public > key to authorized_keys file anyway. I dont even know most of the > passwords for my SSH accounts :0, they are too hard to remember, much > less guess. That I'd agree on, but remember you can have failed key attempts as well, while brute forcing keys is difficult, remember that its not impossible to crack lesser key auths.... one of these days its going to work. Besides connection based attacks aren;t always based on authentication.... you can tie up resources by spamming key based auth failures. -Trish > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > -- Trish Lynch trish at bsdunix.net Key fingerprint = 781D 2B47 AA4B FC88 B919 0CD6 26B2 1D62 6FC1 FF16 From trish at bsdunix.net Tue Sep 12 15:39:06 2006 From: trish at bsdunix.net (Trish Lynch) Date: Tue, 12 Sep 2006 15:39:06 -0400 (EDT) Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060912122953.76ac705b@wit.genoverly.com> References: <20060912095351.4266acc9@wit.genoverly.com> <20060912122953.76ac705b@wit.genoverly.com> Message-ID: <20060912153745.E7416@daemon.bsdunix.net> On Tue, 12 Sep 2006, michael wrote: > On Tue, 12 Sep 2006 11:52:26 -0400 > csnyder wrote: > >> But you encrypted that key using a strong passphrase, right? They >> would have to get your desktop while ssh-agent was running. > > well.. I don't shut down my home PC when I walk away. It is usually > running. But I do lock the apartment door [grin]. > Of course, I guess what was asking is when is Paranoid too paranoid? when you are living by yourself and never have people over, and lock your doors, and have never given your address out, and you STILL LOCK YOUR COMPUTER and never use ssh-agent :) *grin* ((I never use ssh-agent anyway, but this is an extreme example) -Trish -- Trish Lynch trish at bsdunix.net Key fingerprint = 781D 2B47 AA4B FC88 B919 0CD6 26B2 1D62 6FC1 FF16 From trish at bsdunix.net Tue Sep 12 15:37:11 2006 From: trish at bsdunix.net (Trish Lynch) Date: Tue, 12 Sep 2006 15:37:11 -0400 (EDT) Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: References: <20060912095351.4266acc9@wit.genoverly.com> Message-ID: <20060912153445.Q7416@daemon.bsdunix.net> On Tue, 12 Sep 2006, csnyder wrote: > > I really wish the OpenSSH developers would address this issue in the > server itself, by giving admins a lockout setting. I see absolutely no > reason why hundreds of failed login attempts from the same IP address > should be permitted as if it was standard procedure. > I 100% agree with this, its frustrating to have to rely upon self-made scripts and third-party apps to get penSSH to do what it should, which is lock out an IP/username after a certain amount of failed logins. Its not too hard to implement, and I'm sure we're not the only ones asking for it. > Anyway, I use a php script that scans the log for multiple failed > logins from a single IP, then sets a temporary firewall rule blocking > access from that address. > Yes, there are plenty of "log watcher" type programs out there, but why not build this functionality within the daemon itself. Many other daemons have it.... -Trish -- Trish Lynch trish at bsdunix.net Key fingerprint = 781D 2B47 AA4B FC88 B919 0CD6 26B2 1D62 6FC1 FF16 From trish at bsdunix.net Sun Sep 10 19:25:18 2006 From: trish at bsdunix.net (Trish Lynch) Date: Sun, 10 Sep 2006 19:25:18 -0400 (EDT) Subject: [nycbug-talk] ipfw, ipf, pf comparison matrix In-Reply-To: <20060909183115.V624@dru.domain.org> References: <20060908174233.GF25206@cybertron.cyth.net> <20060909164530.F624@dru.domain.org> <20060909183115.V624@dru.domain.org> Message-ID: <20060910192231.Y7416@daemon.bsdunix.net> On Sat, 9 Sep 2006, Dru wrote: > > Okay, so I'm into firewalls and incomplete charts bug me... > > Here's a start at a table that only compares ipfw and pf. Functionality > has been alphabetized. Comparisons were interesting as similar > functionality was described using different terminology in the > documentation for the two firewalls. > > I haven't had a need to make firewall rules that included the IP fields > with ipfw keywords (man ipfw) and would appreciate anyone confirming if pf > also allows you to refer to those fields and how to do so. > > I'd also like feedback on further functionality that should be added to > the chart and a reference proving that a missing * is indeed possible in > that firewall. > > Have fun :-) > > Dru > What do you mean by "Flush", as ipfw has 'ipfw flush', if it means to flush rulesets 'in place'. -Trish -- Trish Lynch trish at bsdunix.net Key fingerprint = 781D 2B47 AA4B FC88 B919 0CD6 26B2 1D62 6FC1 FF16 From nycbug at cyth.net Thu Sep 14 15:43:24 2006 From: nycbug at cyth.net (Ray Lai) Date: Thu, 14 Sep 2006 15:43:24 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060914114841.GJ3349@run.galis.org> References: <20060912095351.4266acc9@wit.genoverly.com> <20060912122953.76ac705b@wit.genoverly.com> <20060912172451.GE15283@run.galis.org> <20060913143717.GP25801@clam.khaoz.org> <20060914114841.GJ3349@run.galis.org> Message-ID: <20060914194347.GC25206@cybertron.cyth.net> On Thu, Sep 14, 2006 at 07:48:41AM -0400, George Georgalis wrote: > On Wed, Sep 13, 2006 at 10:37:17AM -0400, Okan Demirmen wrote: > >On Tue 2006.09.12 at 13:24 -0400, George Georgalis wrote: > >> There was some resolution (at openbsd I think) to encrypt > >> the known_hosts entries with the remote host public key; > >> so if your authentication was compromised, at least there > >> wouldn't be a list a hosts for the attacker to look up. > >> But I've not seen it in my OS yet. > > > >man ssh_config - see HashKnownHosts > > nice, looks like it is in my upgrade path. > > has there been any discussion of hashing .ssh/config? > maybe requiring a private key and passphrase/agent? Hashing the known_hosts file allows you to keep your list of accessible hosts hidden, but still accessible if you already know the hostname. This prevents ssh worms from connecting to your machine, grabbing the list of hosts that you have connected to, and connecting to those hosts using any passphrase-less keys you have set up. I don't see the point of hashing your config, unless you really mean encrypting your config, to which I respond: don't specify any hosts in your config! -Ray- From george at galis.org Thu Sep 14 16:03:20 2006 From: george at galis.org (George Georgalis) Date: Thu, 14 Sep 2006 16:03:20 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060914194347.GC25206@cybertron.cyth.net> References: <20060912095351.4266acc9@wit.genoverly.com> <20060912122953.76ac705b@wit.genoverly.com> <20060912172451.GE15283@run.galis.org> <20060913143717.GP25801@clam.khaoz.org> <20060914114841.GJ3349@run.galis.org> <20060914194347.GC25206@cybertron.cyth.net> Message-ID: <20060914200320.GB143@run.galis.org> On Thu, Sep 14, 2006 at 03:43:24PM -0400, Ray Lai wrote: > >I don't see the point of hashing your config, unless you really mean >encrypting your config, to which I respond: don't specify any hosts in >your config! > Well pretty much the only options I have in my config are per host configs, ie use identity file a, b or c; port x y or z ; user 1 2 or 3 and so on. so for the same reason I'd hash known hosts, I'd also like to hash (encrypt) config. -- but I've not given much thought about implementation, might be difficult/impossible. // George -- George Georgalis, systems architect, administrator < From george at galis.org Thu Sep 14 16:16:04 2006 From: george at galis.org (George Georgalis) Date: Thu, 14 Sep 2006 16:16:04 -0400 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <20060914200320.GB143@run.galis.org> References: <20060912095351.4266acc9@wit.genoverly.com> <20060912122953.76ac705b@wit.genoverly.com> <20060912172451.GE15283@run.galis.org> <20060913143717.GP25801@clam.khaoz.org> <20060914114841.GJ3349@run.galis.org> <20060914194347.GC25206@cybertron.cyth.net> <20060914200320.GB143@run.galis.org> Message-ID: <20060914201604.GC143@run.galis.org> On Thu, Sep 14, 2006 at 04:03:20PM -0400, George Georgalis wrote: >On Thu, Sep 14, 2006 at 03:43:24PM -0400, Ray Lai wrote: >> >>I don't see the point of hashing your config, unless you really mean >>encrypting your config, to which I respond: don't specify any hosts in >>your config! >> > >Well pretty much the only options I have in my config are per host >configs, ie use identity file a, b or c; port x y or z ; user 1 2 >or 3 and so on. so for the same reason I'd hash known hosts, I'd also >like to hash (encrypt) config. -- but I've not given much thought >about implementation, might be difficult/impossible. on second thought, a one to many hash (one way) might fit the bill. I know where I'm going, just don't want the config file to show where I go. // George -- George Georgalis, systems architect, administrator < From ike at lesmuug.org Fri Sep 15 01:19:44 2006 From: ike at lesmuug.org (Isaac Levy) Date: Fri, 15 Sep 2006 01:19:44 -0400 Subject: [nycbug-talk] ucarp question Message-ID: Wordemup All, I've been mucking about with ucarp for the night, and come to a standstill, hitting my head against a wall. Basically, I'm not sure if it's my NIC's, or what- but I can't get it to bring up the IP address? FreeBSD monkey.jmg.lan 6.1-RELEASE FreeBSD 6.1-RELEASE #0: Sun May 7 04:42:56 UTC 2006 root at opus.cse.buffalo.edu:/usr/obj/usr/src/sys/ SMP i386 My ucarp line looks like this: -- ucarp interface=rl0 --srcip=10.0.1.242 --vhid=1 --pass=mypassword \ --preempt --addr=10.1.1.240 \ --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh Running it looks like this: -- [root at monkey ~]# ucarp interface=rl0 --srcip=10.0.1.242 --vhid=1 -- pass=mypassword --preempt --addr=10.1.1.240 --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh [INFO] Using [re0] as a network interface [INFO] Local advertised ethernet address is [00:14:6c:c0:14:cc] [WARNING] Switching to state: MASTER [WARNING] Spawning [/etc/vip-up.sh re0] hello up I'm not daemonizing it, so it just hangs there. No external ping to my new IP, and the IP address is not brought up on the machine on any interface. I've tried this on boxes using rl(4) (RealTek), re(4) (again, RealTek) and xl0 (3com) cards- all doing the same thing. Any urls or other suggestions? I thought ucarp would just add the IP address as an alias, and start speaking CARP... ? Sadly, there's not much documentation out there on ucarp. Rocket- .ike -- Sidenote: PLEASE pretty please don't send any answers like 'just use carp(4)', I'm trying to get ucarp working for a definite reason here. If you haven't heard of ucarp, check it out here: http://www.ucarp.org/ From trish at bsdunix.net Fri Sep 15 07:37:11 2006 From: trish at bsdunix.net (Trish Lynch) Date: Fri, 15 Sep 2006 07:37:11 -0400 (EDT) Subject: [nycbug-talk] spread and wackamole, (Was Re: ucarp question) In-Reply-To: References: Message-ID: <20060915072502.K7416@daemon.bsdunix.net> On Fri, 15 Sep 2006, Isaac Levy wrote: > Sidenote: PLEASE pretty please don't send any answers like 'just use > carp(4)', I'm trying to get ucarp working for a definite reason > here. If you haven't heard of ucarp, check it out here: > http://www.ucarp.org/ Curious as to the reason, as I may actually have something else that might meet your requirement.... Even if it doesn't, its good to get the word out as not many are using it, and its rather cool in both implementation and the fact that its seems more reliable than anything else I've used. http://www.dsn.jhu.edu/pastProjects.html Has two projects, spread, and wackamole, I use almost exclusively in projects that need network redundancy and also, I've been playing with WALRUS for load balancing as well (though an external load balancer such as Linux Virtual Server, serves me better usually, it depends on the requirements of the project). I used to work for Theo Schlossnagle at OmniTI and the guy IMO is a genius and its sad that the most used OSS projects he's worked on are mod_backhand and the SecureID integration into OpenSSH, though I had to leave because of illness (back in 2003 when my fibromyalgia first was diagnosed and I couldn;t physically deal with the symptoms, and I had no decent "medical help" like I do now...) I still respect the work we did there, including the ecelerity MTA which is totally amazing. -Trish -- Trish Lynch trish at bsdunix.net Key fingerprint = 781D 2B47 AA4B FC88 B919 0CD6 26B2 1D62 6FC1 FF16 From trish at bsdunix.net Fri Sep 15 07:48:43 2006 From: trish at bsdunix.net (Trish Lynch) Date: Fri, 15 Sep 2006 07:48:43 -0400 (EDT) Subject: [nycbug-talk] spread and wackamole, (Was Re: ucarp question) In-Reply-To: <20060915072502.K7416@daemon.bsdunix.net> References: <20060915072502.K7416@daemon.bsdunix.net> Message-ID: <20060915074716.B7416@daemon.bsdunix.net> On Fri, 15 Sep 2006, Trish Lynch wrote: Also forgot to mention that mod_backhand does a pretty good job of load balancing web sites without the need for external hardware, but its limited to web sites (or anything apache can handle as a daemon, FTP I'm sure has been implemented within apache as a module somewhere) -Trish -- Trish Lynch trish at bsdunix.net Key fingerprint = 781D 2B47 AA4B FC88 B919 0CD6 26B2 1D62 6FC1 FF16 From ike at lesmuug.org Fri Sep 15 11:11:08 2006 From: ike at lesmuug.org (Isaac Levy) Date: Fri, 15 Sep 2006 11:11:08 -0400 Subject: [nycbug-talk] spread and wackamole, (Was Re: ucarp question) In-Reply-To: <20060915072502.K7416@daemon.bsdunix.net> References: <20060915072502.K7416@daemon.bsdunix.net> Message-ID: Trish, Ya' got good taste man. The Spread Toolkit I know, it's used in Zope to provide replication services, which is where I first saw it in action. It's got a great Python wrapper, as well as other high-level languages. wackamole looks *very* interesting... Rocket- .ike From dlavigne6 at sympatico.ca Fri Sep 15 13:23:11 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Fri, 15 Sep 2006 13:23:11 -0400 (EDT) Subject: [nycbug-talk] BSD Chapter in HLE Message-ID: <20060915130725.S628@dru.domain.org> Hacking Linux Exposed is going to its third edition and I've been asked to write a chapter on BSD security for this edition. I only get one chapter and am supposed to provide an overview of the security features available in *BSD. A draft outline is appended. I plan to showcase the features common to FreeBSD, NetBSD, and OpenBSD as well as point out any features which may not be currently available in all 3. My question to the list is: is this draft missing any features which should be mentioned? Should I mention the ability to strip kernels and build world/build.sh? What about OpenBSD propolice? What about Coverity audits being integrated into engineering processes? Cheers, Dru --- Overview of BSD Projects - brief history (2-3 sentences) - overview of NetBSD, FreeBSD, OpenBSD projects - brief note of FreeBSD forks (PC-BSD, DesktopBSD) Built-in security features - minimal install (secure by default) - periodic security scripts - sysctl - chflags - PAM - /etc/ttys - /etc/ssh/sshd_config - blowfish support - encrypted (filesystem) support (cfs, cgd, gbde, geli) - veriexec - securelevel - system accounting - rc.conf TrustedBSD Extensions - ACLs - MAC policies - OpenBSM pf Firewall Features - CARP - ALTQ - stateful tracking (connection limiting, synproxy) - direct manipulation of state table - OS fingerprinting - traffic normalization - state modulation Securing Applications - jail (sysjail) - portaudit, audit-packages - vuxml BSD Security Advisories - overview of advisory format - overview of security officer/team - URLs to advisory lists Additional BSD Resources - URLs to FreeBSD Handbook, NetBSD Guide, OpenBSD Guide From pete at nomadlogic.org Fri Sep 15 13:37:08 2006 From: pete at nomadlogic.org (Peter Wright) Date: Fri, 15 Sep 2006 10:37:08 -0700 (PDT) Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915130725.S628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> Message-ID: <36932.160.33.20.11.1158341828.squirrel@webmail.nomadlogic.org> > > Hacking Linux Exposed is going to its third edition and I've been asked to > write a chapter on BSD security for this edition. I only get one chapter > and am supposed to provide an overview of the security features available > in *BSD. > > A draft outline is appended. I plan to showcase the features common to > FreeBSD, NetBSD, and OpenBSD as well as point out any features which may > not > be currently available in all 3. > > My question to the list is: is this draft missing any features which > should be mentioned? Should I mention the ability to strip kernels and > build world/build.sh? What about OpenBSD propolice? What about Coverity > audits being integrated into engineering processes? > One thing that I think many linux people overlook, or don't understand, regarding the "bsd way" is that *BSD is an operating system - not a kernel. i think this cohesiveness has a *huge* impact in stability and security. -pete > Cheers, > > Dru > > --- > > Overview of BSD Projects > - brief history (2-3 sentences) > - overview of NetBSD, FreeBSD, OpenBSD projects > - brief note of FreeBSD forks (PC-BSD, DesktopBSD) > > Built-in security features > - minimal install (secure by default) > - periodic security scripts > - sysctl > - chflags > - PAM > - /etc/ttys > - /etc/ssh/sshd_config > - blowfish support > - encrypted (filesystem) support (cfs, cgd, gbde, geli) > - veriexec > - securelevel > - system accounting > - rc.conf > > TrustedBSD Extensions > - ACLs > - MAC policies > - OpenBSM > > pf Firewall Features > - CARP > - ALTQ > - stateful tracking (connection limiting, synproxy) > - direct manipulation of state table > - OS fingerprinting > - traffic normalization > - state modulation > > Securing Applications > - jail (sysjail) > - portaudit, audit-packages > - vuxml > > BSD Security Advisories > - overview of advisory format > - overview of security officer/team > - URLs to advisory lists > > Additional BSD Resources > - URLs to FreeBSD Handbook, NetBSD Guide, OpenBSD Guide > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > -- ~~oO00Oo~~ Peter Wright pete at nomadlogic.org www.nomadlogic.org/~pete 310.869.9459 From lists at genoverly.net Fri Sep 15 13:40:07 2006 From: lists at genoverly.net (michael) Date: Fri, 15 Sep 2006 13:40:07 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915130725.S628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> Message-ID: <20060915134007.4f06402c@wit.genoverly.com> On Fri, 15 Sep 2006 13:23:11 -0400 (EDT) Dru wrote: > Overview of BSD Projects > - brief history (2-3 sentences) > - overview of NetBSD, FreeBSD, OpenBSD projects > - brief note of FreeBSD forks (PC-BSD, DesktopBSD) > > Built-in security features > - minimal install (secure by default) > - periodic security scripts > - sysctl > - chflags > - PAM > - /etc/ttys > - /etc/ssh/sshd_config > - blowfish support > - encrypted (filesystem) support (cfs, cgd, gbde, geli) > - veriexec > - securelevel > - system accounting > - rc.conf ssh? (linux users should learn where it comes from) strlcpy() and strlcat() Memory protection purify * W^X * .rodata segment * Guard pages * Randomized malloc() * Randomized mmap() * atexit() and stdio protection Privilege separation Privilege revocation Chroot jailing New uids ProPolice cryptography! Pseudo Random Number Generators Cryptographic Hash Functions Cryptographic Transforms Cryptographic Hardware Support > TrustedBSD Extensions > - ACLs > - MAC policies > - OpenBSM > > pf Firewall Features > - CARP > - ALTQ > - stateful tracking (connection limiting, synproxy) > - direct manipulation of state table > - OS fingerprinting > - traffic normalization > - state modulation block, pass, nat, rdr, ftp-proxy, authpf, logging > Securing Applications > - jail (sysjail) > - portaudit, audit-packages > - vuxml chroot! > BSD Security Advisories > - overview of advisory format > - overview of security officer/team > - URLs to advisory lists > > Additional BSD Resources > - URLs to FreeBSD Handbook, NetBSD Guide, OpenBSD Guide talk at nycbug [grin] > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month -- Michael From dlavigne6 at sympatico.ca Fri Sep 15 13:58:28 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Fri, 15 Sep 2006 13:58:28 -0400 (EDT) Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <36932.160.33.20.11.1158341828.squirrel@webmail.nomadlogic.org> References: <20060915130725.S628@dru.domain.org> <36932.160.33.20.11.1158341828.squirrel@webmail.nomadlogic.org> Message-ID: <20060915135742.P628@dru.domain.org> On Fri, 15 Sep 2006, Peter Wright wrote: > One thing that I think many linux people overlook, or don't understand, > regarding the "bsd way" is that *BSD is an operating system - not a > kernel. i think this cohesiveness has a *huge* impact in stability and > security. Noted. I'll definitely provide an overview of the release engineering processes in the BSD overview section. Dru From george at sddi.net Fri Sep 15 13:58:37 2006 From: george at sddi.net (George R.) Date: Fri, 15 Sep 2006 13:58:37 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915130725.S628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> Message-ID: <450AE9CD.3080506@sddi.net> Dru wrote: > Hacking Linux Exposed is going to its third edition and I've been asked to > write a chapter on BSD security for this edition. I only get one chapter > and am supposed to provide an overview of the security features available > in *BSD. so it's a focus on "features" and not the os itself? > > A draft outline is appended. I plan to showcase the features common to > FreeBSD, NetBSD, and OpenBSD as well as point out any features which may not > be currently available in all 3. > > My question to the list is: is this draft missing any features which > should be mentioned? Should I mention the ability to strip kernels and > build world/build.sh? What about OpenBSD propolice? What about Coverity > audits being integrated into engineering processes? > > Cheers, > > Dru > > --- > > Overview of BSD Projects > - brief history (2-3 sentences) > - overview of NetBSD, FreeBSD, OpenBSD projects > - brief note of FreeBSD forks (PC-BSD, DesktopBSD) I think the pete point is important . . kernel v everything else is a huge issue. . . the hierarchy of development (v. the anarchy of linux!) it's worth mentioning the scarcity of kernel vulnerbilities v linux also. i know you don't want to compare too much. . . but. . . and add in ports/pkg_src, etc. . . checksum checks. . . > > Built-in security features > - minimal install (secure by default) compare a top output from new install. . . particularly obsd. > - periodic security scripts > - sysctl > - chflags > - PAM do all have PAM support now? > - /etc/ttys > - /etc/ssh/sshd_config question of root enabled by default, although I think this has changed now with obsd. > - blowfish support > - encrypted (filesystem) support (cfs, cgd, gbde, geli) > - veriexec > - securelevel > - system accounting > - rc.conf > > TrustedBSD Extensions > - ACLs > - MAC policies > - OpenBSM > > pf Firewall Features > - CARP > - ALTQ > - stateful tracking (connection limiting, synproxy) > - direct manipulation of state table > - OS fingerprinting > - traffic normalization > - state modulation > you should probably put in *some* discussion of ipf and ipfw. .. but then break into pf as not your ordinary packet filter. > Securing Applications > - jail (sysjail) jails, yes, but is sysjail anywhere yet? and chroot? > - portaudit, audit-packages > - vuxml > > BSD Security Advisories > - overview of advisory format > - overview of security officer/team > - URLs to advisory lists > > Additional BSD Resources > - URLs to FreeBSD Handbook, NetBSD Guide, OpenBSD Guide add swap encryption . . . right? tcp-wrappers. . . let me think a bit more about this... g From dlavigne6 at sympatico.ca Fri Sep 15 14:09:57 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Fri, 15 Sep 2006 14:09:57 -0400 (EDT) Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915134007.4f06402c@wit.genoverly.com> References: <20060915130725.S628@dru.domain.org> <20060915134007.4f06402c@wit.genoverly.com> Message-ID: <20060915140043.I628@dru.domain.org> On Fri, 15 Sep 2006, michael wrote: > strlcpy() and strlcat() > Memory protection purify > * W^X > * .rodata segment > * Guard pages > * Randomized malloc() > * Randomized mmap() > * atexit() and stdio protection Are you aware of any good (preferably "for dummies") URLs explaining these and their benefits? http://www.openbsd.org/security.html isn't a good reference for this book's target audience... And Theo's announcement email on propolice (if you don't know what propolice is go back to kindergarten) isn't useful either ;-) You can get an idea of the audience (yup, it's not techy geeks, but it is large) by taking a look at the free chapter for edition 2: http://www.hackinglinuxexposed.com/samples/hlev2-chapter1.pdf > Pseudo Random Number Generators Hmmm, reminds me I should mention IPSec... > authpf Added. > chroot! Added >> Additional BSD Resources >> - URLs to FreeBSD Handbook, NetBSD Guide, OpenBSD Guide > > talk at nycbug [grin] Perhaps the library: http://www.nycbug.org/index.php?NAV=Library Too bad the SANS reading room doesn't have a BSD section... Dru From lists at genoverly.net Fri Sep 15 14:16:49 2006 From: lists at genoverly.net (michael) Date: Fri, 15 Sep 2006 14:16:49 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915140043.I628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> <20060915134007.4f06402c@wit.genoverly.com> <20060915140043.I628@dru.domain.org> Message-ID: <20060915141649.3c5f14eb@wit.genoverly.com> On Fri, 15 Sep 2006 14:09:57 -0400 (EDT) Dru wrote: > > > On Fri, 15 Sep 2006, michael wrote: > > > strlcpy() and strlcat() > > Memory protection purify > > * W^X > > * .rodata segment > > * Guard pages > > * Randomized malloc() > > * Randomized mmap() > > * atexit() and stdio protection > > > Are you aware of any good (preferably "for dummies") URLs explaining > these and their benefits? http://www.openbsd.org/security.html isn't > a good reference for this book's target audience... you caught me.. that is where I plucked it. We have an OpenBSD developer on the list.. Ray? little help? > Hmmm, reminds me I should mention IPSec... ah, good catch. > > talk at nycbug [grin] > > > Perhaps the library: > > http://www.nycbug.org/index.php?NAV=Library I was kidding around.. but mentioning "your local user group" as a resource, may be a good call. I believe we have them on our Links page. > Too bad the SANS reading room doesn't have a BSD section... > > Dru Yea, we were trying to 'round out' our offerings as a user group. BTW [shameless plug] everyone is welcome to add to the Library! -- Michael From jlam at pkgsrc.org Fri Sep 15 16:13:57 2006 From: jlam at pkgsrc.org (Johnny Lam) Date: Fri, 15 Sep 2006 16:13:57 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915130725.S628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> Message-ID: <450B0985.4060804@pkgsrc.org> Dru wrote: > > TrustedBSD Extensions > - ACLs > - MAC policies > - OpenBSM I think these are FreeBSDisms? NetBSD just incorporated a clean-room implementation of kauth, which is Apple's kernel authorization module, and which shares some common ground with these topics. I have spoken to the author of kauth, Elad Efrat, and and said he is willing to provide any information that you may need. The following URLs provide some information about kauth: http://blog.bsd.org.il/index.php/2005/12/23/kernel-authorization-for-netbsd/ http://mail-index.netbsd.org/tech-security/2006/04/18/0000.html http://developer.apple.com/technotes/tn2005/tn2127.html I've bcc:ed Elad on this message, and I hope that he'll get in touch with you. Cheers, -- Johnny Lam References: <20060915130725.S628@dru.domain.org> <450B0985.4060804@pkgsrc.org> Message-ID: <8c50a3c30609151413s2aa27b17kd3430a4fe74292c1@mail.gmail.com> what about what is in the next version, dtrace is targeted for 7, I think, http://people.freebsd.org/~jb/dtrace/ On 9/15/06, Johnny Lam wrote: > Dru wrote: > > > > TrustedBSD Extensions > > - ACLs > > - MAC policies > > - OpenBSM > > I think these are FreeBSDisms? NetBSD just incorporated a clean-room > implementation of kauth, which is Apple's kernel authorization module, > and which shares some common ground with these topics. I have spoken to > the author of kauth, Elad Efrat, and and said he is willing to provide > any information that you may need. > > The following URLs provide some information about kauth: > > http://blog.bsd.org.il/index.php/2005/12/23/kernel-authorization-for-netbsd/ > http://mail-index.netbsd.org/tech-security/2006/04/18/0000.html > http://developer.apple.com/technotes/tn2005/tn2127.html > > I've bcc:ed Elad on this message, and I hope that he'll get in touch > with you. > > Cheers, > > -- Johnny Lam _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > -- "We trained very hard, but it seemed that every time we were beginning to form into teams we would be reorganized. I was to learn later in life that we tend to meet any new situation by reorganizing, and a wonderful method it can be for creating the illusion of progress, while producing confusion, inefficiency and demoralization." -Gaius Petronius, 1st Century AD From jlam at pkgsrc.org Fri Sep 15 17:28:06 2006 From: jlam at pkgsrc.org (Johnny Lam) Date: Fri, 15 Sep 2006 17:28:06 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <8c50a3c30609151413s2aa27b17kd3430a4fe74292c1@mail.gmail.com> References: <20060915130725.S628@dru.domain.org> <450B0985.4060804@pkgsrc.org> <8c50a3c30609151413s2aa27b17kd3430a4fe74292c1@mail.gmail.com> Message-ID: <450B1AE6.4090200@pkgsrc.org> Marc Spitzer wrote: > what about what is in the next version, dtrace is targeted for 7, I > think, http://people.freebsd.org/~jb/dtrace/ I think DTrace is cool technology, but I don't think it relates to BSD security? Cheers, -- Johnny Lam From nycbug at cyth.net Fri Sep 15 17:51:20 2006 From: nycbug at cyth.net (Ray Lai) Date: Fri, 15 Sep 2006 17:51:20 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915130725.S628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> Message-ID: <20060915215143.GG25206@cybertron.cyth.net> On Fri, Sep 15, 2006 at 01:23:11PM -0400, Dru wrote: > Hacking Linux Exposed is going to its third edition and I've been asked to > write a chapter on BSD security for this edition. I only get one chapter > and am supposed to provide an overview of the security features available > in *BSD. > > A draft outline is appended. I plan to showcase the features common to > FreeBSD, NetBSD, and OpenBSD as well as point out any features which may not > be currently available in all 3. I can't really speak for the other BSDs, but I'll try to discuss OpenBSD-specific features worthy of mention. > My question to the list is: is this draft missing any features which > should be mentioned? Should I mention the ability to strip kernels and > build world/build.sh? I don't see how stripping kernels is a feature, since it is helpful to debug kernel panics. make build is a nice and easy way to keep your system up to date. Just cvs up or apply patches, make build, and go to sleep. > What about OpenBSD propolice? It would be nice to describe some of these security enhancements in depth (more in depth than Theo's slides, less in depth than the author's web pages). > What about Coverity > audits being integrated into engineering processes? Coverity is a nice tool, but its suggested fixes should not be committed wholesale without checking if they are correct. This is true for just about every other tool. Don't overlook lint, either. Chad Loder has been improving our lint to quiet it down and to concentrate on real issues. It is pretty useful to run these tools on the source code and look carefully at areas they point out, concentrating on new findings. Be careful not to change code just to silence the tools, however; this can introduce bugs or silence legitimate ones. > Built-in security features > - minimal install (secure by default) I'd like to mention that the GENERIC kernel has all the usable devices enabled by default, so users don't need to configure and recompile anything to get their devices working. The less there is to configure, the less chances users have of messing up. > - periodic security scripts Checks for changed suid/sgid files, changed devices, and wrong permissions. > - sysctl Enable holes in the kernel, if you'd like. Mainly, machdep.allowaperture=2 is required for i386 for the X Window System. This requirement is being worked on; I am currently running an unaccelerated X with machdep.allowaperture=0 on my X40, but unfortunately this won't make the 4.0 release. 4.1 will probably have it. > - chflags chflags (and read-only mounts) can be used to prevent files from being overwritten, but can be maintenance nightmares. Log rolling and updating system files suddenly requires rebooting, sometimes in single- user mode. > - /etc/ssh/sshd_config Privilege separation, delayed compression, and no forwarding by default are worthy of mention. > - blowfish support Be sure to mention the techniques described in "Future-Adaptable Password Scheme", by Niels Provos and David Mazieres. It would be nice to mention all the different sources of entropy used to generate random data as well. > - encrypted (filesystem) support (cfs, cgd, gbde, geli) vnconfig(8) supports encrypted filesystem images. Also, encrypted swap is enabled by default. > - securelevel Like chflags and read-only mounts, securelevels can be maintenance nightmares, requiring booting to single-user mode for firewall or system updates. > - rc.conf By default, almost everything is turned off but can easily be turned on by adding lines to rc.conf.local. > pf Firewall Features > - CARP pfsync and CARP allow firewalls to be upgraded without having downtime. > - stateful tracking (connection limiting, synproxy) Many complicated firewall filtering rules can be condensed into simple keep state rules, leaving all the packet validation to the pf instead. synproxy is nice, but does anyone still do syn floods? I guess it's useful for protecting operating systems that are vulnerable to them, though. > - direct manipulation of state table Useful for writing simple, secure userland proxies, such as ftp-proxy. > - OS fingerprinting Redirect mail sent from Windows and Linux machines to spamd(8) works wonders, I hear. > - traffic normalization > - state modulation Like synproxy, these two are helpful for protecting other operating systems. > Additional BSD Resources > - URLs to FreeBSD Handbook, NetBSD Guide, OpenBSD Guide We strive to have correct, up-to-date, and useful manuals, so please don't ignore them! For more extensive coverage of certain topics, the FAQ is useful as well. I hope everything I've stated here is accurate; any corrections are welcome. -Ray- From dlavigne6 at sympatico.ca Fri Sep 15 18:02:08 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Fri, 15 Sep 2006 18:02:08 -0400 (EDT) Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <450AE9CD.3080506@sddi.net> References: <20060915130725.S628@dru.domain.org> <450AE9CD.3080506@sddi.net> Message-ID: <20060915175754.L628@dru.domain.org> On Fri, 15 Sep 2006, George R. wrote: > and add in ports/pkg_src, etc. . . checksum checks. . . Added. > compare a top output from new install. . . particularly obsd. That and a netstat... > do all have PAM support now? Open is still debating the necessity IIRC. > you should probably put in *some* discussion of ipf and ipfw. .. but > then break into pf as not your ordinary packet filter. I'll mention them. > tcp-wrappers. . . Do people still use inetd? Dru From nycbug at cyth.net Fri Sep 15 17:58:31 2006 From: nycbug at cyth.net (Ray Lai) Date: Fri, 15 Sep 2006 17:58:31 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <450AE9CD.3080506@sddi.net> References: <20060915130725.S628@dru.domain.org> <450AE9CD.3080506@sddi.net> Message-ID: <20060915215854.GH25206@cybertron.cyth.net> On Fri, Sep 15, 2006 at 01:58:37PM -0400, George R. wrote: > and add in ports/pkg_src, etc. . . checksum checks. . . systrace can be used during ports builds to contain trojaned sources. > > - PAM > > do all have PAM support now? Not OpenBSD. > > - /etc/ssh/sshd_config > > question of root enabled by default, although I think this has changed > now with obsd. Nope, still enabled. > > Securing Applications > > - jail (sysjail) > > jails, yes, but is sysjail anywhere yet? > > and chroot? chroot and dropping privileges is important. root can break out of a chroot, so you must change to an unprivileged user. Additionally, OpenBSD creates new users and groups for each privilege-revoking program, so one cannot another. > tcp-wrappers. . . I think packet filters have largely replaced tcp-wrappers. -Ray- From george at sddi.net Fri Sep 15 18:05:41 2006 From: george at sddi.net (George R.) Date: Fri, 15 Sep 2006 18:05:41 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915215854.GH25206@cybertron.cyth.net> References: <20060915130725.S628@dru.domain.org> <450AE9CD.3080506@sddi.net> <20060915215854.GH25206@cybertron.cyth.net> Message-ID: <450B23B5.7050000@sddi.net> Ray Lai wrote: > On Fri, Sep 15, 2006 at 01:58:37PM -0400, George R. wrote: >> and add in ports/pkg_src, etc. . . checksum checks. . . > > systrace can be used during ports builds to contain trojaned sources. systrace is certainly worth putting in, and it ups the control that an admin or developer has. . . IMHO, it also is the open source reply to much of the IPS functionality. > >>> - PAM >> do all have PAM support now? > > Not OpenBSD. that's what i thought. > >>> - /etc/ssh/sshd_config >> question of root enabled by default, although I think this has changed >> now with obsd. > > Nope, still enabled. double negative time. . . I don't have a recent obsd box to look at, but I am stating that I think that obsd *now* enabled default root access as per sshd_conf.. . am i correct or wrong? I remember the arguments around this. . . > >>> Securing Applications >>> - jail (sysjail) >> jails, yes, but is sysjail anywhere yet? >> >> and chroot? > > chroot and dropping privileges is important. root can break out of a > chroot, so you must change to an unprivileged user. Additionally, > OpenBSD creates new users and groups for each privilege-revoking > program, so one cannot another. > >> tcp-wrappers. . . > > I think packet filters have largely replaced tcp-wrappers. > Mostly . . . but there is a certain continued relevance to both linux and the bsds. . . and besides, Wietse is speaking at NYCBSDCon . . . ;-) and dru, don't forget your mtree-as-poorman's-tripwire. . . but again, found both in linux and the bsds. g From dlavigne6 at sympatico.ca Fri Sep 15 18:24:28 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Fri, 15 Sep 2006 18:24:28 -0400 (EDT) Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915215143.GG25206@cybertron.cyth.net> References: <20060915130725.S628@dru.domain.org> <20060915215143.GG25206@cybertron.cyth.net> Message-ID: <20060915180745.I628@dru.domain.org> On Fri, 15 Sep 2006, Ray Lai wrote: >> My question to the list is: is this draft missing any features which >> should be mentioned? Should I mention the ability to strip kernels and >> build world/build.sh? > > I don't see how stripping kernels is a feature, since it is helpful to > debug kernel panics. make build is a nice and easy way to keep your > system up to date. Just cvs up or apply patches, make build, and go to > sleep. Which is why I asked ;-) This list is chock-full of admins/security folks, what are your best practices for preparing production systems? I can setup this portion of the chapter to show the flexibility/differing philosophies and capabilities of the various BSDs while showing how the tools are available to easily create a secure production system suited to a org's specific requirements. Myself, I always cvsup, build world and strip custom kernel on FreeBSD systems. OpenBSD systems I leave the world/kernel as-is. >> What about OpenBSD propolice? > > It would be nice to describe some of these security enhancements in > depth (more in depth than Theo's slides, less in depth than the author's > web pages). Anyone aware of a succint, easy to read paragraph or two or have the time to contribute one for propolice? >> What about Coverity >> audits being integrated into engineering processes? > > Coverity is a nice tool, but its suggested fixes should not be committed > wholesale without checking if they are correct. This is true for just > about every other tool. Don't overlook lint, either. Chad Loder has > been improving our lint to quiet it down and to concentrate on real > issues. It is pretty useful to run these tools on the source code and > look carefully at areas they point out, concentrating on new findings. > Be careful not to change code just to silence the tools, however; this > can introduce bugs or silence legitimate ones. I'd like to stress the quality of code and the release engineering, commit bit processes as this is a big difference between the BSDs and Linux. I'm also not a committer so it would be interesting to have a paragraph or so from each project explaining how their processes promote secure and quality code. > I'd like to mention that the GENERIC kernel has all the usable devices > enabled by default, so users don't need to configure and recompile > anything to get their devices working. The less there is to configure, > the less chances users have of messing up. Good point. >> - blowfish support > > Be sure to mention the techniques described in "Future-Adaptable > Password Scheme", by Niels Provos and David Mazieres. I will look this up. > vnconfig(8) supports encrypted filesystem images. Also, encrypted swap > is enabled by default. Knew I forgot one... > pfsync and CARP allow firewalls to be upgraded without having downtime. Good point. > Redirect mail sent from Windows and Linux machines to spamd(8) works > wonders, I hear. Should add spamd as well. > We strive to have correct, up-to-date, and useful manuals, so please > don't ignore them! For more extensive coverage of certain topics, the > FAQ is useful as well. Yes, links to online manpages is good, as well as the FAQs. Good stuff. Dru From nycbug at cyth.net Fri Sep 15 18:21:27 2006 From: nycbug at cyth.net (Ray Lai) Date: Fri, 15 Sep 2006 18:21:27 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915141649.3c5f14eb@wit.genoverly.com> References: <20060915130725.S628@dru.domain.org> <20060915134007.4f06402c@wit.genoverly.com> <20060915140043.I628@dru.domain.org> <20060915141649.3c5f14eb@wit.genoverly.com> Message-ID: <20060915222150.GI25206@cybertron.cyth.net> On Fri, Sep 15, 2006 at 02:16:49PM -0400, michael wrote: > On Fri, 15 Sep 2006 14:09:57 -0400 (EDT) > Dru wrote: > > On Fri, 15 Sep 2006, michael wrote: > > > strlcpy() and strlcat() strlcpy and strlcat are safe and easy-to-use versions of strncpy and strncat. They are described in "strlcpy and strlcat -- consistent, safe, string copy and concatenation." by Todd C. Miller and Theo de Raadt. The paper is at http://www.openbsd.org/papers/strlcpy-paper.ps and the slides are at http://www.openbsd.org/papers/strlcpy-slides.ps > > > Memory protection purify > > > * W^X Separates memory regions to be either writable or executable, but not both. This prevents exploits from writing code they want to execute into memory, then causing the program to execute that code. > > > * .rodata segment Segments of memory where data is known not to be writable, such as constants or executable code. Writing to this area causes the program to abort. > > > * Guard pages Places no-access pages of memory at buffer boundaries, so if programs try to read or write outside the buffer the program aborts. Prevents programs that miscalculate the number of elements in an array or do poor pointer arithmetic from continuing unnoticed. > > > * Randomized mmap() Randomizes the location of each mmap allocated memory, which reduces predictability for exploits and leaves gaps between each mmapped region. Access to these gaps causes the program to abort, so overruns and underruns are detected. > > > * Randomized malloc() Uses the randomized mmap instead of the traditional brk/sbrk system calls, which basically forced all the memory allocated to be continguous. Accesses to previously freed memory would not cause core dumps. > > > * atexit() and stdio protection I think there are protections added to atexit, but generally its use is discouraged. I'm not sure what the stdio protection is. > > Are you aware of any good (preferably "for dummies") URLs explaining > > these and their benefits? http://www.openbsd.org/security.html isn't > > a good reference for this book's target audience... > > you caught me.. that is where I plucked it. We have an OpenBSD > developer on the list.. Ray? little help? For a general overview look at Theo's Exploit Mitigation Techniques slides: http://www.openbsd.org/papers/ven05-deraadt/ > > Hmmm, reminds me I should mention IPSec... Yes, and in 4.0 IPsec configuring has been greatly simplified. Check out http://www.openbsd.org/cgi-bin/man.cgi?query=ipsec.conf for example uses. -Ray- From dlavigne6 at sympatico.ca Fri Sep 15 18:32:24 2006 From: dlavigne6 at sympatico.ca (Dru) Date: Fri, 15 Sep 2006 18:32:24 -0400 (EDT) Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915215854.GH25206@cybertron.cyth.net> References: <20060915130725.S628@dru.domain.org> <450AE9CD.3080506@sddi.net> <20060915215854.GH25206@cybertron.cyth.net> Message-ID: <20060915182642.L628@dru.domain.org> On Fri, 15 Sep 2006, Ray Lai wrote: > systrace can be used during ports builds to contain trojaned sources. I see this is in Net and Open. Anyone know of a Free equivalent? > chroot and dropping privileges is important. root can break out of a > chroot, so you must change to an unprivileged user. Additionally, > OpenBSD creates new users and groups for each privilege-revoking > program, so one cannot another. Privilege separation is good and something I'd like to learn more about. Is this always on a per-application basis (e.g. openssh, tcpdump)? Other than Neils' paper, are there other good explanatory references, preferably not at an overly technical level I can use as a resource to refer to. Otherwise, I'll try to "dumb down" a technical reference to a paragraph or so to explain the concept. Dru From nycbug at cyth.net Fri Sep 15 18:27:18 2006 From: nycbug at cyth.net (Ray Lai) Date: Fri, 15 Sep 2006 18:27:18 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <450B23B5.7050000@sddi.net> References: <20060915130725.S628@dru.domain.org> <450AE9CD.3080506@sddi.net> <20060915215854.GH25206@cybertron.cyth.net> <450B23B5.7050000@sddi.net> Message-ID: <20060915222741.GJ25206@cybertron.cyth.net> On Fri, Sep 15, 2006 at 06:05:41PM -0400, George R. wrote: > Ray Lai wrote: > > On Fri, Sep 15, 2006 at 01:58:37PM -0400, George R. wrote: > >>> - /etc/ssh/sshd_config > >> question of root enabled by default, although I think this has changed > >> now with obsd. > > > > Nope, still enabled. > > double negative time. . . I don't have a recent obsd box to look at, but > I am stating that I think that obsd *now* enabled default root access as > per sshd_conf.. . am i correct or wrong? > > I remember the arguments around this. . . root has always been enabled in sshd. > and dru, don't forget your mtree-as-poorman's-tripwire. . . but again, > found both in linux and the bsds. Example usage can be found in security(8): Check for permission changes in special files and system binaries listed in /etc/mtree/special. security also provides hooks for administrators to create their own lists. These lists should be kept in /etc/mtree/ and filenames must have the suffix ``.secure''. The following example shows how to create such a list, to protect the home directory of user ``bob'': # mtree -cx -p /home/bob -K md5digest,type >/etc/mtree/bob.secure # chown root:wheel /etc/mtree/bob.secure # chmod 600 /etc/mtree/bob.secure Note: These checks do not provide complete protection against Trojan horsed binaries, as the miscreant can modify the tree specification to match the replaced binary. For details on really protecting yourself against modified binaries, see mtree(8). -Ray- From lists at genoverly.net Fri Sep 15 18:42:21 2006 From: lists at genoverly.net (michael) Date: Fri, 15 Sep 2006 18:42:21 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915180745.I628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> <20060915215143.GG25206@cybertron.cyth.net> <20060915180745.I628@dru.domain.org> Message-ID: <20060915184221.3c2c5de6@wit.genoverly.com> On Fri, 15 Sep 2006 18:24:28 -0400 (EDT) Dru wrote: > Myself, I always cvsup, build world and strip custom kernel on > FreeBSD systems. OpenBSD systems I leave the world/kernel as-is. OpenBSD binary upgrades (snapshots) are superfast. Compared to the FreeBSD boxen I was keeping, I was startled at how easy it was. You can boot into bsd.rd (ram disk kernel) or upgrade in-place; update etc files (mergemaster); upgrade packages (binary, no compile time); then reboot to freshness. It barely takes a few minutes. -- Michael From njt at ayvali.org Sat Sep 16 14:23:19 2006 From: njt at ayvali.org (N.J. Thomas) Date: Sat, 16 Sep 2006 14:23:19 -0400 Subject: [nycbug-talk] colored DEC terminal for Soekris net4501 Message-ID: <20060916182319.GC14220@ayvali.org> Got my hands on a Soekris net4501 recently. Will be slapping OpenBSD on it and using it as a router and firewall. So I'd like to get a DEC terminal that supports color: my main box gets borrowed and moved around a lot and 99% of the time I'm in a shell, so I don't necessarily need a PC around all the time. All the DEC terminals I've seen online cost $150 and up, and those are mostly the VT100 and VT220 models. EBay has a couple, but I'd rather go local first (a VT340 is going for $400 right now, what is up with that?). Is there any mom & pop place in the metro area where I can snag one of these at a decent price? $150 seems to be a bit too pricey, or have DEC terminals become a collectors items in the past 10 years? Thomas -- N.J. Thomas njt at ayvali.org Etiamsi occiderit me, in ipso sperabo From nycbug at cyth.net Sun Sep 17 02:20:31 2006 From: nycbug at cyth.net (Ray Lai) Date: Sun, 17 Sep 2006 02:20:31 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915182642.L628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> <450AE9CD.3080506@sddi.net> <20060915215854.GH25206@cybertron.cyth.net> <20060915182642.L628@dru.domain.org> Message-ID: <20060917062054.GQ25206@cybertron.cyth.net> On Fri, Sep 15, 2006 at 06:32:24PM -0400, Dru wrote: > On Fri, 15 Sep 2006, Ray Lai wrote: > >chroot and dropping privileges is important. root can break out of a > >chroot, so you must change to an unprivileged user. Additionally, > >OpenBSD creates new users and groups for each privilege-revoking > >program, so one cannot another. > > > Privilege separation is good and something I'd like to learn more about. > Is this always on a per-application basis (e.g. openssh, tcpdump)? Other > than Neils' paper, are there other good explanatory references, preferably > not at an overly technical level I can use as a resource to refer to. > Otherwise, I'll try to "dumb down" a technical reference to a paragraph or > so to explain the concept. Privilege separation limits each application's abilities to the bare minimum. This is done by first opening the necessary files, binding to the necessary ports, and doing anything else that is required while it is still unhindered. The application then removes access to all files in the filesystem by chrooting to an empty directory, typically /var/empty. Finally, it removes root's abilities (breaking out of chroot, binding to privileged ports, et cetera) by changing to an unused user and group. It is important to change to an unused user because processes can send signals to other processes of the same user, such as the kill signal. -Ray- From nycbug at cyth.net Sun Sep 17 02:53:01 2006 From: nycbug at cyth.net (Ray Lai) Date: Sun, 17 Sep 2006 02:53:01 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915180745.I628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> <20060915215143.GG25206@cybertron.cyth.net> <20060915180745.I628@dru.domain.org> Message-ID: <20060917065324.GR25206@cybertron.cyth.net> On Fri, Sep 15, 2006 at 06:24:28PM -0400, Dru wrote: > On Fri, 15 Sep 2006, Ray Lai wrote: > >>What about OpenBSD propolice? > > > >It would be nice to describe some of these security enhancements in > >depth (more in depth than Theo's slides, less in depth than the author's > >web pages). > > Anyone aware of a succint, easy to read paragraph or two or have the time > to contribute one for propolice? Here's a good explanation by Marc Espie, who probably knows more about ProPolice and W^X than me: http://www.onlamp.com/lpt/a/4676 One thing Marc briefly mentions in the interview is that there has been substantial suid and sgid removal. If removal was not possible, suid programs were changed to sgid (xterm is now sgid instead of suid). -Ray- From nycbug at cyth.net Sun Sep 17 03:31:34 2006 From: nycbug at cyth.net (Ray Lai) Date: Sun, 17 Sep 2006 03:31:34 -0400 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915180745.I628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> <20060915215143.GG25206@cybertron.cyth.net> <20060915180745.I628@dru.domain.org> Message-ID: <20060917073157.GS25206@cybertron.cyth.net> On Fri, Sep 15, 2006 at 06:24:28PM -0400, Dru wrote: > On Fri, 15 Sep 2006, Ray Lai wrote: > >>What about Coverity > >>audits being integrated into engineering processes? > > > >Coverity is a nice tool, but its suggested fixes should not be committed > >wholesale without checking if they are correct. This is true for just > >about every other tool. Don't overlook lint, either. Chad Loder has > >been improving our lint to quiet it down and to concentrate on real > >issues. It is pretty useful to run these tools on the source code and > >look carefully at areas they point out, concentrating on new findings. > >Be careful not to change code just to silence the tools, however; this > >can introduce bugs or silence legitimate ones. > > I'd like to stress the quality of code and the release engineering, commit > bit processes as this is a big difference between the BSDs and Linux. I'm > also not a committer so it would be interesting to have a paragraph or so > from each project explaining how their processes promote secure and > quality code. Here is my interpretation of the OpenBSD process: We generally require other developers to okay commits, so obvious bugs and other questionable changes don't creep in. The tree must never be broken. It's better to make small, verifiable changes to achieve a larger goal than to make huge, difficult to understand commits. Manual or tool-aided audits should be performed occasionally; I like doing these on old code because there are usually obvious bugs to fix and because I am unfamiliar with the code, I am forced to learn what it does (and not what the author intended it to do). It's also important to do this on new code, of course. -Ray- From joshmccormack at travelersdiary.com Sun Sep 17 16:22:54 2006 From: joshmccormack at travelersdiary.com (Josh McCormack) Date: Sun, 17 Sep 2006 16:22:54 -0400 Subject: [nycbug-talk] laptop? Message-ID: I'm shopping for a laptop to use OpenBSD on, likely dual booting with Windows XP (yeah, I know). Two things - 1) http://www.openbsd.org/i386-laptop.html is so woefully out of date and thin on info. It should be more like http://jcs.org/laptops/ and have tons more info. NYC BUG's dmesg is helpful, but I think more people sharing this info would be great. Were there a need I'd even possibly help in compiling and updating that info. 2) If anyone has a laptop they no longer need I'd prefer to give my money to people in this community rather than shifty_dude at aol.com. I'm not a big resource hog, so machines with as little as 256mb of RAM are optional. Let me know how much you want, or we can figure it out. Thanks, Josh From yds at CoolRat.org Sun Sep 17 20:02:59 2006 From: yds at CoolRat.org (Yarema) Date: Sun, 17 Sep 2006 20:02:59 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: References: Message-ID: --On Friday, September 15, 2006 1:19 AM -0400 Isaac Levy wrote: > My ucarp line looks like this: > -- > ucarp interface=rl0 --srcip=10.0.1.242 --vhid=1 --pass=mypassword \ > --preempt --addr=10.1.1.240 \ > --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh > > Running it looks like this: > -- > [root at monkey ~]# ucarp interface=rl0 --srcip=10.0.1.242 --vhid=1 -- > pass=mypassword --preempt --addr=10.1.1.240 --upscript=/etc/vip-up.sh > --downscript=/etc/vip-down.sh > [INFO] Using [re0] as a network interface > [INFO] Local advertised ethernet address is [00:14:6c:c0:14:cc] > [WARNING] Switching to state: MASTER > [WARNING] Spawning [/etc/vip-up.sh re0] > hello up Ike, What does your /etc/vip-up.sh script actually look like? Does it have the requisite ifconfig ... alias command to actually assign the virtual IP to the interface in question? As an aside I've found renaming interfaces akin to: ifconfig_fxp0_name="wan0" ifconfig_vge0_name="lan0" ifconfig_vr0_name="sync0" in /etc/rc.conf extremely useful. This way all scripts, ifconfig commands and pf rules can reference the descriptive name rather than the driver name of the interface. -- Yarema http://yds.CoolRat.org/ From nycbug at cyth.net Sun Sep 17 21:00:17 2006 From: nycbug at cyth.net (Ray Lai) Date: Sun, 17 Sep 2006 21:00:17 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: References: Message-ID: <20060918010040.GX25206@cybertron.cyth.net> On Sun, Sep 17, 2006 at 04:22:54PM -0400, Josh McCormack wrote: > I'm shopping for a laptop to use OpenBSD on, likely dual booting with > Windows XP (yeah, I know). If you can, try booting an OpenBSD live CD (such as anonym.os) at the store to make sure it works. > Two things - > > 1) http://www.openbsd.org/i386-laptop.html is so woefully out of date > and thin on info. It should be more like http://jcs.org/laptops/ and That's not really true, people have been submitting updated laptop descriptions. Sure, the descriptions are scant, but not everyone is a writer. Many contain dmesgs and xorg.confs as well. -Ray- From lists at genoverly.net Mon Sep 18 07:22:56 2006 From: lists at genoverly.net (michael) Date: Mon, 18 Sep 2006 07:22:56 -0400 Subject: [nycbug-talk] Charles M. Hannum on NetBSD In-Reply-To: <20060831090240.4aadb83a@wit.genoverly.com> References: <20060831090240.4aadb83a@wit.genoverly.com> Message-ID: <20060918072256.3a5d5ae1@wit.genoverly.com> More press on this. Looks like Hannum *really* wants to make a point. http://www.onlamp.com/pub/a/bsd/2006/09/14/netbsd_future.html -- Michael From mspitzer at gmail.com Mon Sep 18 15:57:19 2006 From: mspitzer at gmail.com (Marc Spitzer) Date: Mon, 18 Sep 2006 15:57:19 -0400 Subject: [nycbug-talk] sans is running a course in manhatten end of october Message-ID: <8c50a3c30609181257w4d7a307bm802d149017bbe7a@mail.gmail.com> if you are interested: http://www.sans.org/manhattan06/ marc -- "We trained very hard, but it seemed that every time we were beginning to form into teams we would be reorganized. I was to learn later in life that we tend to meet any new situation by reorganizing, and a wonderful method it can be for creating the illusion of progress, while producing confusion, inefficiency and demoralization." -Gaius Petronius, 1st Century AD From nycbug-list at 2xlp.com Mon Sep 18 16:25:01 2006 From: nycbug-list at 2xlp.com (Jonathan Vanasco) Date: Mon, 18 Sep 2006 16:25:01 -0400 Subject: [nycbug-talk] sans is running a course in manhatten end of october In-Reply-To: <8c50a3c30609181257w4d7a307bm802d149017bbe7a@mail.gmail.com> References: <8c50a3c30609181257w4d7a307bm802d149017bbe7a@mail.gmail.com> Message-ID: <616DA9F5-34A1-461C-B2E3-67389A2EB4B3@2xlp.com> On Sep 18, 2006, at 3:57 PM, Marc Spitzer wrote: > if you are interested: > > http://www.sans.org/manhattan06/ it seems overly windows centric From michael.bubb at gmail.com Mon Sep 18 17:11:43 2006 From: michael.bubb at gmail.com (Michael Bubb) Date: Mon, 18 Sep 2006 21:11:43 +0000 Subject: [nycbug-talk] help setting up NetBSD on a laptop In-Reply-To: <534a4cab0609181409m28423f39s7669214e49e0bc8a@mail.gmail.com> References: <534a4cab0609181409m28423f39s7669214e49e0bc8a@mail.gmail.com> Message-ID: <534a4cab0609181411o561f10baqed9b3e7c6543e48e@mail.gmail.com> Hello all, I have been using NetBSD for the past year as it is the OS of the University's UNIX lab where I am studying. A few months ago I got a Thinkpad T30 and put NETBSD 3.0 (GENERIC_LAPTOP) on it. I have been using pkgsrc to maintain the software on it. Overall it has been very good. I have come to like NetBSD very much and have gotten very used to it. However there are a number of fairly basic issues that I have not resolved by myself. I've hit a bit of a plateau with the documentation (which is generally very good). A loose 'laundry-list' would include: - wireless card setup (compiling kernel, etc) - proper pkgsrc setup (cvs, etc) - linux emul setup (firefox under suse, etc) - X irregularities ( I have fluxbox working pretty well but some X programs not well) I would like to meet with someone more knowledgeable than myself to work through some of these issues. I am local to NYC and would be willing to meet anywhere there is a usable wireless signal. If this is of any interest pls email me offlist with hourly rates and times available. I am most available on Fridays but can be flexible. Thanks -- Michael Bubb optPart, LLC - Hoboken, NJ cel 201.736.0870 fax 201.377.1717 -- Michael Bubb optPart, LLC - Hoboken, NJ cel 201.736.0870 fax 201.377.1717 From pete at nomadlogic.org Mon Sep 18 19:04:46 2006 From: pete at nomadlogic.org (Pete Wright) Date: Mon, 18 Sep 2006 19:04:46 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: <20060918010040.GX25206@cybertron.cyth.net> References: <20060918010040.GX25206@cybertron.cyth.net> Message-ID: <20060918230443.GA95707@sunset.nomadlogic.org> On Sun, Sep 17, 2006 at 09:00:17PM -0400, Ray Lai wrote: > On Sun, Sep 17, 2006 at 04:22:54PM -0400, Josh McCormack wrote: > > I'm shopping for a laptop to use OpenBSD on, likely dual booting with > > Windows XP (yeah, I know). > > If you can, try booting an OpenBSD live CD (such as anonym.os) at the > store to make sure it works. > > > Two things - > > > > 1) http://www.openbsd.org/i386-laptop.html is so woefully out of date > > and thin on info. It should be more like http://jcs.org/laptops/ and > > That's not really true, people have been submitting updated laptop > descriptions. Sure, the descriptions are scant, but not everyone is a > writer. Many contain dmesgs and xorg.confs as well. > i'd checkout the nycbug.org dmesgd page. i know i've submitted a fair amount of laptops there. i've had good luck running open on Xseries thikpads as well as a dell C400. -p -- ~~oO00Oo~~ Peter Wright pete at nomadlogic.org www.nomadlogic.org/~pete 310.869.9459 From lists at genoverly.net Mon Sep 18 19:23:27 2006 From: lists at genoverly.net (michael) Date: Mon, 18 Sep 2006 19:23:27 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: <20060918230443.GA95707@sunset.nomadlogic.org> References: <20060918010040.GX25206@cybertron.cyth.net> <20060918230443.GA95707@sunset.nomadlogic.org> Message-ID: <20060918192327.1eea8575@wit.genoverly.com> On Mon, 18 Sep 2006 19:04:46 -0400 pete at nomadlogic.org (Pete Wright) wrote: > i'd checkout the nycbug.org dmesgd page. i know i've submitted a fair > amount of laptops there. i've had good luck running open on Xseries > thikpads as well as a dell C400. I can vouch for OpenBSD on a ThinkPad T21 & T43.. no problems. OpenBSD on any IBM/Lenovo laptop is a pretty good bet. While Craigs list has it fair share of shifty_dudes, it is a little more personal (local) than ebay. There is even a slight chance you get an opportunity to slip in a CD to see if it boots well. Your specs are in the sweet spot for a bargain. This brings 74 items: http://newyork.craigslist.org/search/sys?query=thinkpad -- Michael From joshmccormack at travelersdiary.com Mon Sep 18 23:38:14 2006 From: joshmccormack at travelersdiary.com (Josh McCormack) Date: Mon, 18 Sep 2006 23:38:14 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: <20060918192327.1eea8575@wit.genoverly.com> References: <20060918010040.GX25206@cybertron.cyth.net> <20060918230443.GA95707@sunset.nomadlogic.org> <20060918192327.1eea8575@wit.genoverly.com> Message-ID: On 9/18/06, michael wrote: > On Mon, 18 Sep 2006 19:04:46 -0400 > pete at nomadlogic.org (Pete Wright) wrote: > > > i'd checkout the nycbug.org dmesgd page. i know i've submitted a fair > > amount of laptops there. i've had good luck running open on Xseries > > thikpads as well as a dell C400. > > I can vouch for OpenBSD on a ThinkPad T21 & T43.. no problems. OpenBSD > on any IBM/Lenovo laptop is a pretty good bet. > > While Craigs list has it fair share of shifty_dudes, it is a little > more personal (local) than ebay. There is even a slight chance you get > an opportunity to slip in a CD to see if it boots well. > > Your specs are in the sweet spot for a bargain. This brings 74 items: > http://newyork.craigslist.org/search/sys?query=thinkpad > > -- > > Michael Definitely the sweet spot. I picked an X23 with maxed out RAM (640MB) and was very happy with the deal. Tomorrow I'll have to ask the desktop support guys for a USB floppy &/or CD drive to borrow. Josh From pete at nomadlogic.org Tue Sep 19 12:23:20 2006 From: pete at nomadlogic.org (Pete Wright) Date: Tue, 19 Sep 2006 12:23:20 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: References: <20060918010040.GX25206@cybertron.cyth.net> <20060918230443.GA95707@sunset.nomadlogic.org> <20060918192327.1eea8575@wit.genoverly.com> Message-ID: <20060919162317.GA99159@sunset.nomadlogic.org> On Mon, Sep 18, 2006 at 11:38:14PM -0400, Josh McCormack wrote: > On 9/18/06, michael wrote: > > On Mon, 18 Sep 2006 19:04:46 -0400 > > pete at nomadlogic.org (Pete Wright) wrote: > > > > > i'd checkout the nycbug.org dmesgd page. i know i've submitted a fair > > > amount of laptops there. i've had good luck running open on Xseries > > > thikpads as well as a dell C400. > > > > I can vouch for OpenBSD on a ThinkPad T21 & T43.. no problems. OpenBSD > > on any IBM/Lenovo laptop is a pretty good bet. > > > > While Craigs list has it fair share of shifty_dudes, it is a little > > more personal (local) than ebay. There is even a slight chance you get > > an opportunity to slip in a CD to see if it boots well. > > > > Your specs are in the sweet spot for a bargain. This brings 74 items: > > http://newyork.craigslist.org/search/sys?query=thinkpad > > > > -- > > > > Michael > > Definitely the sweet spot. I picked an X23 with maxed out RAM (640MB) > and was very happy with the deal. Tomorrow I'll have to ask the > desktop support guys for a USB floppy &/or CD drive to borrow. > i had an x23 and it ran great. i'd suggest just PXE booting and doing an nfs install on the lappy. -pete -- ~~oO00Oo~~ Peter Wright pete at nomadlogic.org www.nomadlogic.org/~pete 310.869.9459 From joshmccormack at travelersdiary.com Tue Sep 19 12:42:53 2006 From: joshmccormack at travelersdiary.com (Josh McCormack) Date: Tue, 19 Sep 2006 12:42:53 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: <20060919162317.GA99159@sunset.nomadlogic.org> References: <20060918010040.GX25206@cybertron.cyth.net> <20060918230443.GA95707@sunset.nomadlogic.org> <20060918192327.1eea8575@wit.genoverly.com> <20060919162317.GA99159@sunset.nomadlogic.org> Message-ID: On 9/19/06, Pete Wright wrote: > On Mon, Sep 18, 2006 at 11:38:14PM -0400, Josh McCormack wrote: > > On 9/18/06, michael wrote: > > > On Mon, 18 Sep 2006 19:04:46 -0400 > > > pete at nomadlogic.org (Pete Wright) wrote: > > > > > > > i'd checkout the nycbug.org dmesgd page. i know i've submitted a fair > > > > amount of laptops there. i've had good luck running open on Xseries > > > > thikpads as well as a dell C400. > > > > > > I can vouch for OpenBSD on a ThinkPad T21 & T43.. no problems. OpenBSD > > > on any IBM/Lenovo laptop is a pretty good bet. > > > > > > While Craigs list has it fair share of shifty_dudes, it is a little > > > more personal (local) than ebay. There is even a slight chance you get > > > an opportunity to slip in a CD to see if it boots well. > > > > > > Your specs are in the sweet spot for a bargain. This brings 74 items: > > > http://newyork.craigslist.org/search/sys?query=thinkpad > > > > > > -- > > > > > > Michael > > > > Definitely the sweet spot. I picked an X23 with maxed out RAM (640MB) > > and was very happy with the deal. Tomorrow I'll have to ask the > > desktop support guys for a USB floppy &/or CD drive to borrow. > > > > i had an x23 and it ran great. i'd suggest just PXE booting and doing > an nfs install on the lappy. > > > -pete > > -- > ~~oO00Oo~~ > Peter Wright > pete at nomadlogic.org > www.nomadlogic.org/~pete > 310.869.9459 > > Awesome. I'll look up how to do it. Thanks, Josh From ike at lesmuug.org Tue Sep 19 18:12:14 2006 From: ike at lesmuug.org (Isaac Levy) Date: Tue, 19 Sep 2006 18:12:14 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: References: Message-ID: On Sep 17, 2006, at 8:02 PM, Yarema wrote: > --On Friday, September 15, 2006 1:19 AM -0400 Isaac Levy > wrote: > >> My ucarp line looks like this: >> -- >> ucarp interface=rl0 --srcip=10.0.1.242 --vhid=1 --pass=mypassword \ >> --preempt --addr=10.1.1.240 \ >> --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh >> >> Running it looks like this: >> -- >> [root at monkey ~]# ucarp interface=rl0 --srcip=10.0.1.242 --vhid=1 -- >> pass=mypassword --preempt --addr=10.1.1.240 --upscript=/etc/vip-up.sh >> --downscript=/etc/vip-down.sh >> [INFO] Using [re0] as a network interface >> [INFO] Local advertised ethernet address is [00:14:6c:c0:14:cc] >> [WARNING] Switching to state: MASTER >> [WARNING] Spawning [/etc/vip-up.sh re0] >> hello up > > Ike, > > What does your /etc/vip-up.sh script actually look like? Does it > have the requisite ifconfig ... alias command to actually assign > the virtual IP to the interface in question? > > As an aside I've found renaming interfaces akin to: > > ifconfig_fxp0_name="wan0" > ifconfig_vge0_name="lan0" > ifconfig_vr0_name="sync0" > > in /etc/rc.conf extremely useful. This way all scripts, ifconfig > commands and pf rules can reference the descriptive name rather > than the driver name of the interface. > > -- > Yarema > http://yds.CoolRat.org/ Oh. Boy I feel stupid. So UCarp is REALLY just the Carp, I feel like an idiot, I thought UCarp brought up the ipalias all by itself. (*doh!*) But it ONLY does the Carp. Boy I feel stupid. So, I'll do it again when I'm back touching that machine again, adding the appropriate ifconfig alias statements to the up and down scripts. Rocket- .ike From swygue at gmail.com Tue Sep 19 23:48:16 2006 From: swygue at gmail.com (swygue) Date: Tue, 19 Sep 2006 23:48:16 -0400 Subject: [nycbug-talk] What is fdescfs ? Message-ID: Playing around with ezjail and noticed fdescfs in ezjail.conf. What is it ? And what are the benefits of using it with JAIL(8). I read the manpage but the description does not make any sense to me. -- swygue neron --->> -------------- next part -------------- An HTML attachment was scrubbed... URL: From yds at CoolRat.org Wed Sep 20 00:49:39 2006 From: yds at CoolRat.org (Yarema) Date: Wed, 20 Sep 2006 00:49:39 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: References: Message-ID: <834B350BD84A95E8CD136EFD@[192.168.1.69]> --On Tuesday, September 19, 2006 6:12 PM -0400 Isaac Levy wrote: > On Sep 17, 2006, at 8:02 PM, Yarema wrote: > >> What does your /etc/vip-up.sh script actually look like? Does >> it have the requisite ifconfig ... alias command to actually >> assign the virtual IP to the interface in question? > > Oh. Boy I feel stupid. So UCarp is REALLY just the Carp, I feel > like an idiot, I thought UCarp brought up the ipalias all by > itself. (*doh!*) But it ONLY does the Carp. Boy I feel stupid. > > So, I'll do it again when I'm back touching that machine again, > adding the appropriate ifconfig alias statements to the up and > down scripts. Well, like you said.. the documentation is a little on the thin side with ucarp.. there's not even a man page.. you'll have to experiment with this, but I think the /etc/vip-{up,down}.sh scripts should add/remove the virtual IP aliases on *all* the interfaces for which you want failover. Not just the interface which stops responding to the heartbeat. I believe that's how the kernel carp(4) does it. That being said I think more apropos names for the up/down scripts would be /etc/start_if.carp as documented in rc.conf(5). If you write the scripts just right they might work for both ucarp and carp(4).. But I think the main advantage of ucarp over carp(4) is that you can do whatever you need in the up/down scripts. carp(4) does not have this type of flexibility. -- Yarema http://yds.CoolRat.org/ From freebsd-listen at fabiankeil.de Wed Sep 20 04:56:10 2006 From: freebsd-listen at fabiankeil.de (Fabian Keil) Date: Wed, 20 Sep 2006 10:56:10 +0200 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915182642.L628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> <450AE9CD.3080506@sddi.net> <20060915215854.GH25206@cybertron.cyth.net> <20060915182642.L628@dru.domain.org> Message-ID: <20060920105610.41d4a05b@localhost> Dru wrote: > On Fri, 15 Sep 2006, Ray Lai wrote: > > > systrace can be used during ports builds to contain trojaned sources. > > > I see this is in Net and Open. Anyone know of a Free equivalent? There is no exact equivalent, but in many cases you can use a FreeBSD jail instead. Experimental systrace patches for FreeBSD 4.x and 5.x are floating around the net. Fabian -- http://www.fabiankeil.de/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 187 bytes Desc: not available URL: From freebsd-listen at fabiankeil.de Wed Sep 20 05:04:56 2006 From: freebsd-listen at fabiankeil.de (Fabian Keil) Date: Wed, 20 Sep 2006 11:04:56 +0200 Subject: [nycbug-talk] BSD Chapter in HLE In-Reply-To: <20060915130725.S628@dru.domain.org> References: <20060915130725.S628@dru.domain.org> Message-ID: <20060920110456.4336cea8@localhost> Dru wrote: > My question to the list is: is this draft missing any features which > should be mentioned? Should I mention the ability to strip kernels and > build world/build.sh? What about OpenBSD propolice? ProPolice isn't specific to OpenBSD, I'm using it with FreeBSD as well. It's not in the base system, but adding it is not a big deal. http://tataz.chchile.org/~tataz/FreeBSD/SSP/README.html Fabian -- http://www.fabiankeil.de/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 187 bytes Desc: not available URL: From joshmccormack at travelersdiary.com Wed Sep 20 09:12:13 2006 From: joshmccormack at travelersdiary.com (Josh McCormack) Date: Wed, 20 Sep 2006 09:12:13 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: <20060919162317.GA99159@sunset.nomadlogic.org> References: <20060918010040.GX25206@cybertron.cyth.net> <20060918230443.GA95707@sunset.nomadlogic.org> <20060918192327.1eea8575@wit.genoverly.com> <20060919162317.GA99159@sunset.nomadlogic.org> Message-ID: On 9/19/06, Pete Wright wrote: > On Mon, Sep 18, 2006 at 11:38:14PM -0400, Josh McCormack wrote: > > On 9/18/06, michael wrote: > > > On Mon, 18 Sep 2006 19:04:46 -0400 > > > pete at nomadlogic.org (Pete Wright) wrote: > i had an x23 and it ran great. i'd suggest just PXE booting and doing > an nfs install on the lappy. > > > -pete > > -- > ~~oO00Oo~~ > Peter Wright > pete at nomadlogic.org > www.nomadlogic.org/~pete > 310.869.9459 > > I read up on it, and don't have anything I could use for the PXE booting. I don't have a router I could put anything on or an even a somewhat modern OpenBSD box. The computer has a CF card reader built in, so I tried making a CF card work like the boot floppy or CD, but the windows programs for writing those wouldn't do it. Anyone have a USB floppy or CD drive I could borrow, or want to help me do the PXE booting? Thanks, Josh From okan at demirmen.com Wed Sep 20 09:28:02 2006 From: okan at demirmen.com (Okan Demirmen) Date: Wed, 20 Sep 2006 09:28:02 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: References: <20060918010040.GX25206@cybertron.cyth.net> <20060918230443.GA95707@sunset.nomadlogic.org> <20060918192327.1eea8575@wit.genoverly.com> <20060919162317.GA99159@sunset.nomadlogic.org> Message-ID: <20060920132802.GI11341@clam.khaoz.org> On Wed 2006.09.20 at 09:12 -0400, Josh McCormack wrote: > I read up on it, and don't have anything I could use for the PXE > booting. I don't have a router I could put anything on or an even a > somewhat modern OpenBSD box. you just need 2 files to pxeboot(8), dhcpd(8) and tftpd(8) - very simple even if you have an older openbsd box. read the manpage online to see how to set it up if you don't have pxeboot(8) on your older openbsd box. > The computer has a CF card reader built in, so I tried making a CF > card work like the boot floppy or CD, but the windows programs for > writing those wouldn't do it. also you'd have to make sure that your bios can boot off the cf card. my x40 will not boot from its built-in cf card - so heads up. > Anyone have a USB floppy or CD drive I could borrow, or want to help > me do the PXE booting? also consider a usb key. good luck. cheers, okan From af.dingo at gmail.com Wed Sep 20 09:38:06 2006 From: af.dingo at gmail.com (Jeff Quast) Date: Wed, 20 Sep 2006 09:38:06 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: <834B350BD84A95E8CD136EFD@192.168.1.69> References: <834B350BD84A95E8CD136EFD@192.168.1.69> Message-ID: On 9/20/06, Yarema wrote: > > But I think the main advantage of ucarp over carp(4) is that you can do > whatever you need in the up/down scripts. carp(4) does not have this type > of flexibility. > Of course not. Thats what ifstated(8) is for! From af.dingo at gmail.com Wed Sep 20 09:44:41 2006 From: af.dingo at gmail.com (Jeff Quast) Date: Wed, 20 Sep 2006 09:44:41 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: <20060920132802.GI11341@clam.khaoz.org> References: <20060918010040.GX25206@cybertron.cyth.net> <20060918230443.GA95707@sunset.nomadlogic.org> <20060918192327.1eea8575@wit.genoverly.com> <20060919162317.GA99159@sunset.nomadlogic.org> <20060920132802.GI11341@clam.khaoz.org> Message-ID: On 9/20/06, Okan Demirmen wrote: > On Wed 2006.09.20 at 09:12 -0400, Josh McCormack wrote: > > Anyone have a USB floppy or CD drive I could borrow, or want to help > > me do the PXE booting? > > also consider a usb key. > Or a laptop 2.5" IDE to 3.5" IDE converter. About $2-$5, probobly can find it localy. Install it as if it were its own local disk on a desktop machine, then swap it back into the laptop. Of course, windows will puke on this, but any BSD would purr along happily inside a completely different machine provided it is the same architecture (i386) Worse case, you might have to boot bsd.rd and modify /etc/fstab with ed if your hard drive isn't installed using wd0 on the desktop machine. From ike at lesmuug.org Wed Sep 20 10:12:00 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 20 Sep 2006 10:12:00 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: References: <834B350BD84A95E8CD136EFD@192.168.1.69> Message-ID: <8424B177-D022-4747-BC3A-1B804D0C897E@lesmuug.org> On Sep 20, 2006, at 9:38 AM, Jeff Quast wrote: > On 9/20/06, Yarema wrote: >> >> But I think the main advantage of ucarp over carp(4) is that you >> can do >> whatever you need in the up/down scripts. carp(4) does not have >> this type >> of flexibility. >> > > Of course not. Thats what ifstated(8) is for! That's a new-ish OpenBSD thing? Sounds interesting... For the record, just looked it up: http://www.freebsd.org/cgi/man.cgi? query=ifstated&apropos=0&sektion=0&manpath=OpenBSD+3.9&format=html -or- http://tinyurl.com/nc9ud Hrm.... Niiiiice. Rocket- .ike From okan at demirmen.com Wed Sep 20 10:28:40 2006 From: okan at demirmen.com (Okan Demirmen) Date: Wed, 20 Sep 2006 10:28:40 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: <8424B177-D022-4747-BC3A-1B804D0C897E@lesmuug.org> References: <834B350BD84A95E8CD136EFD@192.168.1.69> <8424B177-D022-4747-BC3A-1B804D0C897E@lesmuug.org> Message-ID: <20060920142840.GL11341@clam.khaoz.org> On Wed 2006.09.20 at 10:12 -0400, Isaac Levy wrote: > On Sep 20, 2006, at 9:38 AM, Jeff Quast wrote: > > > On 9/20/06, Yarema wrote: > >> > >> But I think the main advantage of ucarp over carp(4) is that you > >> can do > >> whatever you need in the up/down scripts. carp(4) does not have > >> this type > >> of flexibility. > >> > > > > Of course not. Thats what ifstated(8) is for! > > That's a new-ish OpenBSD thing? Sounds interesting... well, if 2 years and 7 months is "new-ish", then yes ;) > Hrm.... Niiiiice. yes it is. From ike at lesmuug.org Wed Sep 20 10:38:48 2006 From: ike at lesmuug.org (Isaac Levy) Date: Wed, 20 Sep 2006 10:38:48 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: <20060920142840.GL11341@clam.khaoz.org> References: <834B350BD84A95E8CD136EFD@192.168.1.69> <8424B177-D022-4747-BC3A-1B804D0C897E@lesmuug.org> <20060920142840.GL11341@clam.khaoz.org> Message-ID: On Sep 20, 2006, at 10:28 AM, Okan Demirmen wrote: > On Wed 2006.09.20 at 10:12 -0400, Isaac Levy wrote: >> On Sep 20, 2006, at 9:38 AM, Jeff Quast wrote: >>> On 9/20/06, Yarema wrote: >>>> >>>> But I think the main advantage of ucarp over carp(4) is that you >>>> can do >>>> whatever you need in the up/down scripts. carp(4) does not have >>>> this type >>>> of flexibility. >>>> >>> >>> Of course not. Thats what ifstated(8) is for! >> >> That's a new-ish OpenBSD thing? Sounds interesting... > > well, if 2 years and 7 months is "new-ish", then yes ;) Well, in the scope of the big picture, I'll call that new :) > >> Hrm.... Niiiiice. > > yes it is. Q for you then: How do *you* use it, what kinds of problems have you solved with it? Does the codebase look portable, (I'd assume it's tied to the network stack pretty closely)? Rocket- .ike From dave at donnerjack.com Wed Sep 20 10:48:32 2006 From: dave at donnerjack.com (David Lawson) Date: Wed, 20 Sep 2006 10:48:32 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: References: <834B350BD84A95E8CD136EFD@192.168.1.69> <8424B177-D022-4747-BC3A-1B804D0C897E@lesmuug.org> <20060920142840.GL11341@clam.khaoz.org> Message-ID: <912EE08A-DC72-4B7F-90BB-0A15C6B0A2BA@donnerjack.com> On Sep 20, 2006, at 10:38 AM, Isaac Levy wrote: > On Sep 20, 2006, at 10:28 AM, Okan Demirmen wrote: >> On Wed 2006.09.20 at 10:12 -0400, Isaac Levy wrote: >>> On Sep 20, 2006, at 9:38 AM, Jeff Quast wrote: >>>> On 9/20/06, Yarema wrote: >>>>> >>>>> But I think the main advantage of ucarp over carp(4) is that you >>>>> can do >>>>> whatever you need in the up/down scripts. carp(4) does not have >>>>> this type >>>>> of flexibility. >>>>> >>>> >>>> Of course not. Thats what ifstated(8) is for! >>> >>> That's a new-ish OpenBSD thing? Sounds interesting... >> >> well, if 2 years and 7 months is "new-ish", then yes ;) > > Well, in the scope of the big picture, I'll call that new :) > >> >>> Hrm.... Niiiiice. >> >> yes it is. Huh, that looks really interesting. I ran into some problems with a heavily loaded box running CARP interfaces where some interfaces would fail to the secondary and others wouldn't, leading to some obvious routing problems. I couldn't find a pure CARP way of dealing with the problem, it looks like this might be something that'd solve that issue reasonably elegantly. --Dave From yds at CoolRat.org Wed Sep 20 13:36:29 2006 From: yds at CoolRat.org (Yarema) Date: Wed, 20 Sep 2006 13:36:29 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: <8424B177-D022-4747-BC3A-1B804D0C897E@lesmuug.org> References: <834B350BD84A95E8CD136EFD@192.168.1.69> <8424B177-D022-4747-BC3A-1B804D0C897E@lesmuug.org> Message-ID: --On Wednesday, September 20, 2006 10:12 AM -0400 Isaac Levy wrote: > On Sep 20, 2006, at 9:38 AM, Jeff Quast wrote: > >> On 9/20/06, Yarema wrote: >>> >>> But I think the main advantage of ucarp over carp(4) is >>> that you can do whatever you need in the up/down scripts. >>> carp(4) does not have this type of flexibility. >>> >> >> Of course not. Thats what ifstated(8) is for! > > That's a new-ish OpenBSD thing? Sounds interesting... > > For the record, just looked it up: > http://www.freebsd.org/cgi/man.cgi? > query=ifstated&apropos=0&sektion=0&manpath=OpenBSD+3.9&format=html > -or- > http://tinyurl.com/nc9ud > > Hrm.... Niiiiice. ifstated(8) does seem very nice. I suppose on the FreeBSD side of the fence we might be able to hack something similar with net-snmp, but that's far more complex. Actually the issue where ucarp vs. carp(8) makes a difference is that with carp(4) one ends up with the same IP nets being assigned to more than one interface -- the physical interface and the virtual carp(4) interface. This gets in the way running dhcpd(8). Using ucarp should help get around this by keeping all the IP addresses on the same physical interface. -- Yarema http://yds.CoolRat.org/ From yds at CoolRat.org Wed Sep 20 13:48:34 2006 From: yds at CoolRat.org (Yarema) Date: Wed, 20 Sep 2006 13:48:34 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: <912EE08A-DC72-4B7F-90BB-0A15C6B0A2BA@donnerjack.com> References: <834B350BD84A95E8CD136EFD@192.168.1.69> <8424B177-D022-4747-BC3A-1B804D0C897E@lesmuug.org> <20060920142840.GL11341@clam.khaoz.org> <912EE08A-DC72-4B7F-90BB-0A15C6B0A2BA@donnerjack.com> Message-ID: <7F73A9086256CECBA458244B@[192.168.1.69]> --On Wednesday, September 20, 2006 10:48 AM -0400 David Lawson wrote: > Huh, that looks really interesting. I ran into some problems > with a heavily loaded box running CARP interfaces where some > interfaces would fail to the secondary and others wouldn't, > leading to some obvious routing problems. I couldn't find a > pure CARP way of dealing with the problem, it looks like this > might be something that'd solve that issue reasonably elegantly. from the FreeBSD carp(4) man page: net.inet.carp.preempt -- Allow virtual hosts to preempt each other. It is also used to failover carp interfaces as a group. When the option is enabled and one of the carp enabled physical interfaces goes down, advskew is changed to 240 on all carp interfaces. See also the first example. Disabled by default. I've had other problems with carp(4), but with net.inet.carp.preempt set the failover does happen as a group in a "pure CARP way". -- Yarema http://yds.CoolRat.org/ From joshmccormack at travelersdiary.com Wed Sep 20 16:46:44 2006 From: joshmccormack at travelersdiary.com (Josh McCormack) Date: Wed, 20 Sep 2006 16:46:44 -0400 Subject: [nycbug-talk] laptop? In-Reply-To: References: <20060918010040.GX25206@cybertron.cyth.net> <20060918230443.GA95707@sunset.nomadlogic.org> <20060918192327.1eea8575@wit.genoverly.com> <20060919162317.GA99159@sunset.nomadlogic.org> <20060920132802.GI11341@clam.khaoz.org> Message-ID: On 9/20/06, Jeff Quast wrote: > On 9/20/06, Okan Demirmen wrote: > > On Wed 2006.09.20 at 09:12 -0400, Josh McCormack wrote: > > > Anyone have a USB floppy or CD drive I could borrow, or want to help > > > me do the PXE booting? > > > > also consider a usb key. > > > > Or a laptop 2.5" IDE to 3.5" IDE converter. About $2-$5, probobly can > find it localy. Install it as if it were its own local disk on a > desktop machine, then swap it back into the laptop. > > Of course, windows will puke on this, but any BSD would purr along > happily inside a completely different machine provided it is the same > architecture (i386) > > Worse case, you might have to boot bsd.rd and modify /etc/fstab with > ed if your hard drive isn't installed using wd0 on the desktop > machine. > My continuing attempt at making my laptop dual boot with preinstalled XP. I got my hands on a USB floppy drive and a box of floppies. I found a NTFS resizing program that can run off a floppy (ntfsresize), once a Linux environment is up. I've tried a bunch of floppy distros, but half of them aren't writing to the floppies do to bad sectors, the other half won't be read by the other computer b/c of bad sectors (I/0 errors). I've tried: lepton smart alcolix can't figure out how to make one from tomsrtbt from XP. Can I run ntfsresize from the OpenBSD floppy install shell? It's staticly linked, so I thought perhaps, but honestly I can't even figure out how to mount the floppy from the shell. ntfsresize: http://mlf.linux.rulez.org/mlf/ezaz/ntfsresize.html#static Josh From okan at demirmen.com Wed Sep 20 17:15:31 2006 From: okan at demirmen.com (Okan Demirmen) Date: Wed, 20 Sep 2006 17:15:31 -0400 Subject: [nycbug-talk] ucarp question In-Reply-To: References: <834B350BD84A95E8CD136EFD@192.168.1.69> <8424B177-D022-4747-BC3A-1B804D0C897E@lesmuug.org> <20060920142840.GL11341@clam.khaoz.org> Message-ID: <20060920211531.GX11341@clam.khaoz.org> On Wed 2006.09.20 at 10:38 -0400, Isaac Levy wrote: > >well, if 2 years and 7 months is "new-ish", then yes ;) > > Well, in the scope of the big picture, I'll call that new :) well, ok ;) > Q for you then: > How do *you* use it, what kinds of problems have you solved with it? > Does the codebase look portable, (I'd assume it's tied to the network > stack pretty closely)? i think the manpage and example ifstated.conf gives you an idea for what i do with it; but mainly to help fail devices not based on link state. i believe the code is relatively portable, but it does use event(3), which i'm not sure if freebsd has, at least in base. there may be other things, but i haven't really looked closely. that being said, most times you don't need it. okan From george at sddi.net Wed Sep 20 18:37:23 2006 From: george at sddi.net (George R.) Date: Wed, 20 Sep 2006 18:37:23 -0400 Subject: [nycbug-talk] NYCBSDCon Update Message-ID: <4511C2A3.7080105@sddi.net> Just to keep everyone updated. . Things are progressing along nicely for the second NYCBSDCon. Our sponsor page should be updated soon, with a number of well-known sponsors that is allowing us to bring out some pretty impressive speakers. I don't think anyone will be disappointed. Some additional surprise speakers are also being added, making this conference pretty heavy, once again. The site will be updated shortly. Registration is now open at nycbsdcon.org. We strongly encourage everyone to register ASAP, as the price is very reasonable right now, but will increase significantly. The price includes continental breakfast and lunch for both days, to facilitate the networking part of the event. . . no more being forced to wander Broadway trying to find a slice. Also, if any small firms are interested in doing the small business sponsorship again this year, email me off-list about it. You can find more information about this under the sponsors tab. Finally, our next NYCBUG meeting on October 4th will be a discussion and overview of the upcoming conference. We strongly encourage everyone interested in taking an active role in the operation of the conference to attend. . . and this includes the (local) speakers, board members, etc. And yes, if anyone was wondering, Ike will be MC'g the event again this year. I mean, come on, how could Ike *not*? George From george at sddi.net Thu Sep 21 18:52:47 2006 From: george at sddi.net (George R.) Date: Thu, 21 Sep 2006 18:52:47 -0400 Subject: [nycbug-talk] cheap SATA drives Message-ID: <451317BF.8020609@sddi.net> As an FYI. . .Compusa has some Samsung 160 gig SATA 7200 RPM 3.5 drives on sale for $70. This was at the 5th Ave/37th Street location. . . g From scottro at nyc.rr.com Thu Sep 21 19:41:39 2006 From: scottro at nyc.rr.com (Scott Robbins) Date: Thu, 21 Sep 2006 19:41:39 -0400 Subject: [nycbug-talk] HP4550 printer needs good home Message-ID: <20060921234139.GA86233@mail.scottro.net> Howdy folks, In the course of our company doing a major move, they are getting rid of some things. One of these is an HP4550 color laserjet printer, rather old, but in reasonable shape. It would require a JetDirect (unless you were planning to connect it directly.) It has some consumables in it, I'm not sure how much of any are remaining. It's a moderately heavy duty printer, and speaking of that, it's quite heavy. At any rate, it's free if someone wants to take it away. It's at 1001 6th Avenue, between 38th and 39th. Anyone interested, please email me off-list. We are open at least part of the day tomorrow and they really want it out of there. There's also a low end older 3com switch with no power cord--we'll throw it in. :) Thanks. -- Scott Robbins PGP keyID EB3467D6 ( 1B48 077D 66F6 9DB0 FDC2 A409 FA54 EB34 67D6 ) gpg --keyserver pgp.mit.edu --recv-keys EB3467D6 Xander: So, we Bronzin' it tonight? Willow: Wednesdays, kinda beat... Xander: Well, we could grind our enemies into talcum powder with a sledgehammer, but, gosh, we did that last night. From lists at genoverly.net Fri Sep 22 16:56:54 2006 From: lists at genoverly.net (michael) Date: Fri, 22 Sep 2006 16:56:54 -0400 Subject: [nycbug-talk] conference schedule Message-ID: <20060922165654.10758a02@wit.genoverly.com> NYCBSDCon 2006 The speaker schedule for the conference has gone up. http://nycbsdcon.org/schedule Also check out the updated speakers list. http://nycbsdcon.org/speakers This is a really exciting line-up. -- Michael From spork at bway.net Fri Sep 22 17:13:38 2006 From: spork at bway.net (Charles Sprickman) Date: Fri, 22 Sep 2006 17:13:38 -0400 (EDT) Subject: [nycbug-talk] conference schedule In-Reply-To: <20060922165654.10758a02@wit.genoverly.com> References: <20060922165654.10758a02@wit.genoverly.com> Message-ID: On Fri, 22 Sep 2006, michael wrote: > NYCBSDCon 2006 > > The speaker schedule for the conference has gone up. > http://nycbsdcon.org/schedule What an excellent lineup. But where are these geeks that wake up before 10 a.m.??? :) I've yet to meet one... C > Also check out the updated speakers list. > http://nycbsdcon.org/speakers > > This is a really exciting line-up. > > -- > > Michael > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > From tux at penguinnetwerx.net Fri Sep 22 17:35:31 2006 From: tux at penguinnetwerx.net (Kevin Reiter) Date: Fri, 22 Sep 2006 17:35:31 -0400 Subject: [nycbug-talk] conference schedule In-Reply-To: References: <20060922165654.10758a02@wit.genoverly.com> Message-ID: <45145723.9020501@penguinnetwerx.net> Charles Sprickman wrote: > What an excellent lineup. But where are these geeks that wake up before > 10 a.m.??? :) I've yet to meet one... ..especially the geeks living in Jersey that have an hour+ commute with traffic. Might as well just stay awake Friday night into Saturday :) From trish at bsdunix.net Fri Sep 22 17:46:02 2006 From: trish at bsdunix.net (Trish Lynch) Date: Fri, 22 Sep 2006 17:46:02 -0400 (EDT) Subject: [nycbug-talk] conference schedule In-Reply-To: <45145723.9020501@penguinnetwerx.net> References: <20060922165654.10758a02@wit.genoverly.com> <45145723.9020501@penguinnetwerx.net> Message-ID: <20060922174437.O7416@daemon.bsdunix.net> On Fri, 22 Sep 2006, Kevin Reiter wrote: > Charles Sprickman wrote: >> What an excellent lineup. But where are these geeks that wake up before >> 10 a.m.??? :) I've yet to meet one... > > ..especially the geeks living in Jersey that have an hour+ commute with > traffic. Might as well just stay awake Friday night into Saturday :) > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > I have room for one or two at my place in the Bronx. Not sure if I'll have my son though, so no partying. As I'm also very ill, I go to bed early, but I do offer wifi (secured and "unsecured") and a comfy fold out couch, and if I don;t have my son, a futon as well. -Trish -- Trish Lynch trish at bsdunix.net Key fingerprint = 781D 2B47 AA4B FC88 B919 0CD6 26B2 1D62 6FC1 FF16 From jonathan at kc8onw.net Fri Sep 22 18:00:08 2006 From: jonathan at kc8onw.net (Jonathan Stewart) Date: Fri, 22 Sep 2006 18:00:08 -0400 Subject: [nycbug-talk] [SOLVED] Reproducible data corruption on 6.1-Stable [Long but please read] In-Reply-To: <45088DA4.6020208@kc8onw.net> References: <45088DA4.6020208@kc8onw.net> Message-ID: <45145CE8.5010903@kc8onw.net> Just as a follow up I did a dd from the raw device and piping it to md5 with the filesystem unmounted. Since the md5 changed everytime I'm fairly certain it is a hardware issue and have called highpoint and gotten an RMA number. It took me long enough to realize it but I didn't have to dd that exact file as long as I could reproduce the issue with a dd from anywhere on the device. Thanks, Jonathan From lists at genoverly.net Sun Sep 24 11:26:09 2006 From: lists at genoverly.net (michael) Date: Sun, 24 Sep 2006 11:26:09 -0400 Subject: [nycbug-talk] conference schedule In-Reply-To: References: <20060922165654.10758a02@wit.genoverly.com> Message-ID: <20060924112609.09085d6c@wit.genoverly.com> On Fri, 22 Sep 2006 17:13:38 -0400 (EDT) Charles Sprickman wrote: > What an excellent lineup. But where are these geeks that wake up > before 10 a.m.??? :) I've yet to meet one... Real geeks sleep in late and have long uptimes, right? Myths and machismo perpetuated.. [grin] The conference has arranged to bring many of the BSD greats, one-by-one, onto the stage to talk to you about BSD.. all day Saturday and into the afternoon on Sunday. Some are traveling a fair distance to do this for you. There was a great response when we called for presenters. It would hardly feel fair to the audience to deny speakers who want to come and talk BSD because we couldn't start before noon. There is so much material that the schedule demanded starting in the AM and is kept pretty tight all day. While it may seem onerous to drag yourself out of bed on a weekend morning, but.. could it be for a better reason? [grin] So, get up, brush your teeth, comb your hair, and get up to Columbia. There will be hot coffee waiting for you... not to mention a staff that got up way earlier than you to make sure it all happened! -- Michael From tux at penguinnetwerx.net Mon Sep 25 09:38:36 2006 From: tux at penguinnetwerx.net (Kevin Reiter) Date: Mon, 25 Sep 2006 09:38:36 -0400 Subject: [nycbug-talk] conference schedule In-Reply-To: <20060922174437.O7416@daemon.bsdunix.net> References: <20060922165654.10758a02@wit.genoverly.com> <45145723.9020501@penguinnetwerx.net> <20060922174437.O7416@daemon.bsdunix.net> Message-ID: <4517DBDC.20805@penguinnetwerx.net> Trish Lynch wrote: > On Fri, 22 Sep 2006, Kevin Reiter wrote: > >> Charles Sprickman wrote: >>> What an excellent lineup. But where are these geeks that wake up before >>> 10 a.m.??? :) I've yet to meet one... >> >> ..especially the geeks living in Jersey that have an hour+ commute with >> traffic. Might as well just stay awake Friday night into Saturday :) >> _______________________________________________ >> % NYC*BUG talk mailing list >> http://lists.nycbug.org/mailman/listinfo/talk >> %Be sure to check out our Jobs and NYCBUG-announce lists >> %We meet the first Wednesday of the month >> > > I have room for one or two at my place in the Bronx. Not sure if I'll > have my son though, so no partying. As I'm also very ill, I go to bed > early, but I do offer wifi (secured and "unsecured") and a comfy fold > out couch, and if I don;t have my son, a futon as well. Thanks, I'll keep that in mind. (My idea of "partying" nowadays is staying up late reading bash.org with coffee, so no worries about that :) From spork at bway.net Wed Sep 27 14:02:21 2006 From: spork at bway.net (Charles Sprickman) Date: Wed, 27 Sep 2006 14:02:21 -0400 (EDT) Subject: [nycbug-talk] Daemonnews? Message-ID: <20060927135948.I76085@sporker.bway.net> Seems to have been off the air for a few days now. I have not seen any mention of it anywhere... Anyone know what happened? IP is up and reachable, but http seems to not be doing anything. Also noticed that the bsdnews.com domain went bye-bye for a bit this weekend. Charles From mikel.king at techally.com Wed Sep 27 14:25:25 2006 From: mikel.king at techally.com (Mikel King) Date: Wed, 27 Sep 2006 14:25:25 -0400 Subject: [nycbug-talk] Daemonnews? In-Reply-To: <20060927135948.I76085@sporker.bway.net> References: <20060927135948.I76085@sporker.bway.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 All, It seems that the server upon which we were hosting has snuffed it. George Sutter and Chris Coleman have restored the backups to another box and we are currently limping along on one of our mirrors; http:// www2.daemonnews.org which as you can see is not fully up yet. I am working on getting our MySql backend mirrored as well as finagling a permanent box or two of our own. Thanks in advance to anyone who'd like to step forward and give us a hand. Cheers, Mikel King On Sep 27, 2006, at 2:02 PM, Charles Sprickman wrote: > Seems to have been off the air for a few days now. I have not seen > any > mention of it anywhere... Anyone know what happened? IP is up and > reachable, but http seems to not be doing anything. Also noticed > that the > bsdnews.com domain went bye-bye for a bit this weekend. > > Charles > > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (Darwin) iD8DBQFFGsIW/MdWADRNwzQRAot1AJ9Tg5QtiweKsDkF6B9KcWzzME05dgCgiQcz 5yLmk4sOTPkKuCoMslTLvUY= =QOLM -----END PGP SIGNATURE----- From george at sddi.net Wed Sep 27 16:29:21 2006 From: george at sddi.net (George R.) Date: Wed, 27 Sep 2006 16:29:21 -0400 Subject: [nycbug-talk] Daemonnews? In-Reply-To: References: <20060927135948.I76085@sporker.bway.net> Message-ID: <451ADF21.40404@sddi.net> Mikel King wrote: > All, > > It seems that the server upon which we were hosting has snuffed it. > George Sutter and Chris Coleman have restored the backups to another > box and we are currently limping along on one of our mirrors; http:// > www2.daemonnews.org which as you can see is not fully up yet. I am > working on getting our MySql backend mirrored as well as finagling a > permanent box or two of our own. > > Thanks in advance to anyone who'd like to step forward and give us a > hand. > And as an FYI. . . YES, we have offered many times to provide hosting/rack space for DN. .. We are always interested in utilizing it for the community. g From dan at langille.org Wed Sep 27 18:52:54 2006 From: dan at langille.org (Dan Langille) Date: Wed, 27 Sep 2006 18:52:54 -0400 Subject: [nycbug-talk] Daemonnews? In-Reply-To: <451ADF21.40404@sddi.net> References: Message-ID: <451AC886.25778.4AA28C2F@dan.langille.org> On 27 Sep 2006 at 16:29, George R. wrote: > Mikel King wrote: > > All, > > > > It seems that the server upon which we were hosting has snuffed it. > > George Sutter and Chris Coleman have restored the backups to another > > box and we are currently limping along on one of our mirrors; http:// > > www2.daemonnews.org which as you can see is not fully up yet. I am > > working on getting our MySql backend mirrored as well as finagling a > > permanent box or two of our own. > > > > Thanks in advance to anyone who'd like to step forward and give us a > > hand. > > > > And as an FYI. . . > > YES, we have offered many times to provide hosting/rack space for DN. .. > > We are always interested in utilizing it for the community. OK, since we're going public, I offerred space too. -- Dan Langille : Software Developer looking for work my resume: http://www.freebsddiary.org/dan_langille.php From pete at nomadlogic.org Thu Sep 28 13:43:02 2006 From: pete at nomadlogic.org (Peter Wright) Date: Thu, 28 Sep 2006 10:43:02 -0700 (PDT) Subject: [nycbug-talk] 64Bit FreeNAS Message-ID: <30731.160.33.20.11.1159465382.squirrel@webmail.nomadlogic.org> Hi, Has anyone had any success building a 64Bit version of FreeNAS. We are trying to do some testing with large memory filers here and are not having much luck with PAE kernel's at this point. Just checking the list to see if anyone has gone this route before I go and build a 64bit kernel/world. thanks! -pete -- ~~oO00Oo~~ Peter Wright pete at nomadlogic.org www.nomadlogic.org/~pete 310.869.9459 From ike at lesmuug.org Thu Sep 28 14:15:23 2006 From: ike at lesmuug.org (Isaac Levy) Date: Thu, 28 Sep 2006 14:15:23 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues Message-ID: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> Hey All, So I'm sad, and wanted to solicit some help to perhaps cheer me up. SITUATION: I've been using File-Backed disks via mdconfig(8) with FreeBSD jail (8) for a very long time. (I tend to call them 'disk images', Apple vocabulary.) Creating a New File-Backed Disk with mdconfig: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/disks- virtual.html#AEN24952 -or- http://tinyurl.com/gqjjv Their usefulness, in my enviornment, is that they are far more flexible to work with in constraining disk size for jails across machines than any other strategy, (namely, hard disk partitions). -- My sadness, and my query has to do with speed- or the painful lack of it. Running a disk image is notably slower than the raw disk, (in my case, fairly snappy RAID5/SATA), but 'usable'. Based on totally unscientific benchmarking, when I've got more than twenty disk images on a given machine, (all running jails), disk I/O becomes far slower than is usable- and each new disk takes the speed down at what seems to be an order of significant magnitude. What I'm saying is, they don't scale. During the FreeBSD 4.x days, (wow what an era), file-backed disks were snappy, with little overhead, no? (vn(4) and vnconfig(8), respectively) -- In the immediate future, I'm taking some jailing systems back to just running jails on the filesystem, (and partitioning, yuck), but in the meantime does anyone have any thoughts on the subject? It simply seems like the file-backing part of the memory disk implementation was slapped on, or perhaps is just cruft left over from the 5.x era? Hrm... Rocket- .ike From ike at lesmuug.org Thu Sep 28 14:33:22 2006 From: ike at lesmuug.org (Isaac Levy) Date: Thu, 28 Sep 2006 14:33:22 -0400 Subject: [nycbug-talk] 64Bit FreeNAS In-Reply-To: <30731.160.33.20.11.1159465382.squirrel@webmail.nomadlogic.org> References: <30731.160.33.20.11.1159465382.squirrel@webmail.nomadlogic.org> Message-ID: Hi Pete, On Sep 28, 2006, at 1:43 PM, Peter Wright wrote: > Hi, Has anyone had any success building a 64Bit version of > FreeNAS. We > are trying to do some testing with large memory filers here and are > not > having much luck with PAE kernel's at this point. Just checking > the list > to see if anyone has gone this route before I go and build a 64bit > kernel/world. > > thanks! > -pete Not sure how much experience you've had with FreeNAS, but because it's heavily based on the m0n0wall project code, (and user conventions), I've tried giving it a whirl- and wanted to mention it when I spoke on m0n0wall/pfSense. I didn't mention it, because on several *very different* x86 machines, I had serious issues getting it to boot cleanly- let alone serve data- so I've shoved it into the 'beta' stack in my brain. I could have made some foolish mistake, but I tried this on different locations, with current disk images several months apart- same cruddy boot issues. I'd LOVE to hear if anyone has had any good experiences with FreeNAS... It seems to be a great drop-in solution for network data storage, just like m0n0wall and pfSense. Best, .ike From pete at nomadlogic.org Thu Sep 28 14:52:20 2006 From: pete at nomadlogic.org (Peter Wright) Date: Thu, 28 Sep 2006 11:52:20 -0700 (PDT) Subject: [nycbug-talk] 64Bit FreeNAS In-Reply-To: References: <30731.160.33.20.11.1159465382.squirrel@webmail.nomadlogic.org> Message-ID: <46478.160.33.20.11.1159469540.squirrel@webmail.nomadlogic.org> > Hi Pete, > > On Sep 28, 2006, at 1:43 PM, Peter Wright wrote: > >> Hi, Has anyone had any success building a 64Bit version of >> FreeNAS. We >> are trying to do some testing with large memory filers here and are >> not >> having much luck with PAE kernel's at this point. Just checking >> the list >> to see if anyone has gone this route before I go and build a 64bit >> kernel/world. >> >> thanks! >> -pete > > Not sure how much experience you've had with FreeNAS, but because > it's heavily based on the m0n0wall project code, (and user > conventions), I've tried giving it a whirl- and wanted to mention it > when I spoke on m0n0wall/pfSense. > > I didn't mention it, because on several *very different* x86 > machines, I had serious issues getting it to boot cleanly- let alone > serve data- so I've shoved it into the 'beta' stack in my brain. > I could have made some foolish mistake, but I tried this on different > locations, with current disk images several months apart- same cruddy > boot issues. > > I'd LOVE to hear if anyone has had any good experiences with > FreeNAS... It seems to be a great drop-in solution for network data > storage, just like m0n0wall and pfSense. > right on, i'm going to give it a whirl. i've read through the docu, and most of the scripts and i should be able to build a 64bit version - but i'll most likely have to dedicate a host to build the kernel/world. we've actually had good performance using freenas for NFSv3 serving. we are in the process of building memfs volumes to be used to cache data...hence we need to be able to access ~16gig's of ram per node. i'll reply to the list with success/problems in a bit. -pete -- ~~oO00Oo~~ Peter Wright pete at nomadlogic.org www.nomadlogic.org/~pete 310.869.9459 From spork at bway.net Thu Sep 28 14:54:33 2006 From: spork at bway.net (Charles Sprickman) Date: Thu, 28 Sep 2006 14:54:33 -0400 (EDT) Subject: [nycbug-talk] 64Bit FreeNAS In-Reply-To: References: <30731.160.33.20.11.1159465382.squirrel@webmail.nomadlogic.org> Message-ID: On Thu, 28 Sep 2006, Isaac Levy wrote: > I'd LOVE to hear if anyone has had any good experiences with > FreeNAS... It seems to be a great drop-in solution for network data > storage, just like m0n0wall and pfSense. While I haven't tried it, I seem to recall a beta pfsense package to make your firewall also be a FreeNAS box. Obviously not the best idea for most environments, but might be handy for home use. Charles > Best, > .ike > > > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > From trish at bsdunix.net Thu Sep 28 15:17:37 2006 From: trish at bsdunix.net (Trish Lynch) Date: Thu, 28 Sep 2006 15:17:37 -0400 (EDT) Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> Message-ID: <20060928151410.D7416@daemon.bsdunix.net> On Thu, 28 Sep 2006, Isaac Levy wrote: > During the FreeBSD 4.x days, (wow what an era), file-backed disks > were snappy, with little overhead, no? (vn(4) and vnconfig(8), > respectively) > > -- > In the immediate future, I'm taking some jailing systems back to just > running jails on the filesystem, (and partitioning, yuck), but in the > meantime does anyone have any thoughts on the subject? It simply > seems like the file-backing part of the memory disk implementation > was slapped on, or perhaps is just cruft left over from the 5.x era? > > Hrm... > Ike, I'm not sure they ever were really "snappy", but I think you've got two things to content with.... not any optimization in the code they currently use now to do file backed storage devices *and* the fact that the size of them has to be much larger than what you used to use back in the 4.x days anyway (I know all my filesystems are definitely much larger than then..... File out who the maintainer of the code is (done by looking in the source for the command), and contact them directly on this issue, they may have some kind of system tuning directly involved with this, either changing block sizes or anything like that in the filesystem creation..... when you get an answer, let us know! -Trish -- Trish Lynch trish at bsdunix.net Key fingerprint = 781D 2B47 AA4B FC88 B919 0CD6 26B2 1D62 6FC1 FF16 From nycbug-list at 2xlp.com Thu Sep 28 15:32:37 2006 From: nycbug-list at 2xlp.com (Jonathan Vanasco) Date: Thu, 28 Sep 2006 15:32:37 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> Message-ID: on a tangent... OSX has a neat tool called iPartition http://www.coriolis-systems.com/iPartition.php something like that, under freebsd, would probably solve your issues. From okan at demirmen.com Thu Sep 28 16:08:27 2006 From: okan at demirmen.com (Okan Demirmen) Date: Thu, 28 Sep 2006 16:08:27 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> Message-ID: <20060928200827.GP24150@clam.khaoz.org> On Thu 2006.09.28 at 14:15 -0400, Isaac Levy wrote: > Hey All, > > So I'm sad, and wanted to solicit some help to perhaps cheer me up. > > SITUATION: > I've been using File-Backed disks via mdconfig(8) with FreeBSD jail > (8) for a very long time. > (I tend to call them 'disk images', Apple vocabulary.) > > Creating a New File-Backed Disk with mdconfig: > http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/disks- > virtual.html#AEN24952 > -or- > http://tinyurl.com/gqjjv > > Their usefulness, in my enviornment, is that they are far more > flexible to work with in constraining disk size for jails across > machines than any other strategy, (namely, hard disk partitions). > > -- > My sadness, and my query has to do with speed- or the painful lack of > it. > Running a disk image is notably slower than the raw disk, (in my > case, fairly snappy RAID5/SATA), but 'usable'. > Based on totally unscientific benchmarking, when I've got more than > twenty disk images on a given machine, (all running jails), disk I/O > becomes far slower than is usable- and each new disk takes the speed > down at what seems to be an order of significant magnitude. > > What I'm saying is, they don't scale. > > During the FreeBSD 4.x days, (wow what an era), file-backed disks > were snappy, with little overhead, no? (vn(4) and vnconfig(8), > respectively) I can't comment on how md(4) is performing, but I did notice you said RAID5/SATA. Have you created a case to eliminate RAID5/SATA? I believe SATA is really mostly for just raw storage and not to be used to anything that requires a reasonable amount I/O. I've run into situations where we ended up tossing cabinets of SATA into just storage and online backup, and replacing with SCSI; no more I/O issues. In another, moved VMWare clusters from SATA to SCSI; again removing performace issues due to I/O. Note that I only have cabinet-size arrays of SATA experience, and not 4-5 SATA disks, so results may differ; as well as with application. I think you might have to do more than "unscientific benchmarking." I'm afraid that may help you narrow down the issue, but I do realize that'll take some time... Cheers, Okan From spork at bway.net Thu Sep 28 16:16:24 2006 From: spork at bway.net (Charles Sprickman) Date: Thu, 28 Sep 2006 16:16:24 -0400 (EDT) Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <20060928200827.GP24150@clam.khaoz.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> <20060928200827.GP24150@clam.khaoz.org> Message-ID: On Thu, 28 Sep 2006, Okan Demirmen wrote: > I can't comment on how md(4) is performing, but I did notice you said > RAID5/SATA. Have you created a case to eliminate RAID5/SATA? I believe > SATA is really mostly for just raw storage and not to be used to > anything that requires a reasonable amount I/O. For what it's worth, I've been punishing a 3Ware 9550-SX-12 for the past few weeks, and it has pretty much made me a convert. Any new boxes we build with hardware raid are getting these cards. They are beating the snot out of our Adaptec ZCR cards, both in performance and management. Things that would require reboots and visits to the BIOS config when using Adaptec were handled quite smoothly. Punishment included yanking a bunch of drives during a long bonnie run, rearranging the drives, yanking power during drive rebuilds, etc. And being able to change stripe sizes without wiping the array is a real plus. Sorry for the marketing, C > Cheers, > Okan > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > From george at galis.org Thu Sep 28 19:22:13 2006 From: george at galis.org (George Georgalis) Date: Thu, 28 Sep 2006 19:22:13 -0400 Subject: [nycbug-talk] what is the threat of the openssl advisory? Message-ID: <20060928232213.GQ13445@run.galis.org> There was an openssl advisory today http://www.openssl.org/news/secadv_20060928.txt http://security.freebsd.org/advisories/FreeBSD-SA-06:23.openssl.asc my primary concern is A buffer overflow was discovered in the SSL_get_shared_ciphers() utility function. An attacker could send a list of ciphers to an application that uses this function and overrun a buffer (CVE-2006-3738). there is no comment on if an exploit is known to exist or how difficult (or easy) it would be to create one based on the patch. http://security.freebsd.org/patches/SA-06:23/ In fact the netbsd openssl looks pretty different than freebsd in the context of applying the patch. Can we determine a level of risk? Are all ssl, openvpn, ssh, https, etc servers needing access restricted to friendly IPs or is the threat just one bit inside "astronomically possible?" -- I cannot tell. // George -- George Georgalis, systems architect, administrator < From ike at lesmuug.org Thu Sep 28 23:13:41 2006 From: ike at lesmuug.org (Isaac Levy) Date: Thu, 28 Sep 2006 23:13:41 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <20060928151410.D7416@daemon.bsdunix.net> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> <20060928151410.D7416@daemon.bsdunix.net> Message-ID: Hi All, Wow everyone had constructive responses- though I don't know why I'm surprised, nycbug talk list always rocks, On Sep 28, 2006, at 3:17 PM, Trish Lynch wrote: > On Thu, 28 Sep 2006, Isaac Levy wrote: > >> During the FreeBSD 4.x days, (wow what an era), file-backed disks >> were snappy, with little overhead, no? (vn(4) and vnconfig(8), >> respectively) >> >> -- >> In the immediate future, I'm taking some jailing systems back to just >> running jails on the filesystem, (and partitioning, yuck), but in the >> meantime does anyone have any thoughts on the subject? It simply >> seems like the file-backing part of the memory disk implementation >> was slapped on, or perhaps is just cruft left over from the 5.x era? >> >> Hrm... >> > > Ike, > > I'm not sure they ever were really "snappy", but I think you've > got two things to content with.... > not any optimization in the code they currently use now to do file > backed storage devices *and* Yes, with GEOM getting all the exiting attention(?)... > the fact that the size of them has to be much larger than what you > used to use back in the 4.x days anyway (I know all my filesystems > are definitely much larger than then..... Ahh- too true. RAID array size back then was around 80-100gb usable space, now the same 1u systems have 850-1400gb. > > File out who the maintainer of the code is (done by looking in the > source for the command), and contact them directly on this issue, > they may have some kind of system tuning directly involved with > this, either changing block sizes or anything like that in the > filesystem creation..... when you get an answer, let us know! > > -Trish Will do- downloading source to grok as I type this... Rocket- .ike From ike at lesmuug.org Thu Sep 28 23:18:27 2006 From: ike at lesmuug.org (Isaac Levy) Date: Thu, 28 Sep 2006 23:18:27 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <20060928200827.GP24150@clam.khaoz.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> <20060928200827.GP24150@clam.khaoz.org> Message-ID: <5992C067-73B9-44E9-AB5C-8CA133BCF41D@lesmuug.org> Hi All, On Sep 28, 2006, at 4:08 PM, Okan Demirmen wrote: > On Thu 2006.09.28 at 14:15 -0400, Isaac Levy wrote: >> Hey All, >> >> So I'm sad, and wanted to solicit some help to perhaps cheer me up. >> >> SITUATION: >> I've been using File-Backed disks via mdconfig(8) with FreeBSD jail >> (8) for a very long time. >> (I tend to call them 'disk images', Apple vocabulary.) >> >> Creating a New File-Backed Disk with mdconfig: >> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/disks- >> virtual.html#AEN24952 >> -or- >> http://tinyurl.com/gqjjv >> >> Their usefulness, in my enviornment, is that they are far more >> flexible to work with in constraining disk size for jails across >> machines than any other strategy, (namely, hard disk partitions). >> >> -- >> My sadness, and my query has to do with speed- or the painful lack of >> it. >> Running a disk image is notably slower than the raw disk, (in my >> case, fairly snappy RAID5/SATA), but 'usable'. >> Based on totally unscientific benchmarking, when I've got more than >> twenty disk images on a given machine, (all running jails), disk I/O >> becomes far slower than is usable- and each new disk takes the speed >> down at what seems to be an order of significant magnitude. >> >> What I'm saying is, they don't scale. >> >> During the FreeBSD 4.x days, (wow what an era), file-backed disks >> were snappy, with little overhead, no? (vn(4) and vnconfig(8), >> respectively) > > I can't comment on how md(4) is performing, but I did notice you said > RAID5/SATA. Have you created a case to eliminate RAID5/SATA? I believe > SATA is really mostly for just raw storage and not to be used to > anything that requires a reasonable amount I/O. I've run into > situations > where we ended up tossing cabinets of SATA into just storage and > online > backup, and replacing with SCSI; no more I/O issues. In another, moved > VMWare clusters from SATA to SCSI; again removing performace issues > due > to I/O. Well, erm- I've found the sata systems to be every bit as snappy at this scale- (4 drives per 1u box). > > Note that I only have cabinet-size arrays of SATA experience, and not > 4-5 SATA disks, so results may differ; as well as with application. Gotcha. At my scale, honestly, I've found no real i/o problems with SATA over SCSI. Technically, I understand SCSI is faster, but the margins are so low at this scale it becomes moot for me- especially over the price and ease of use. > > I think you might have to do more than "unscientific benchmarking." > I'm > afraid that may help you narrow down the issue, but I do realize > that'll > take some time... > > Cheers, > Okan I'm gonna' grok the source, and go back historically on the topic to see if I'm just dreaming about the past or some bs... And should just move on. This is beginning to smell like the kind of scale/ abstraction problem that doesn't go away in computing... Rocket- .ike From ike at lesmuug.org Thu Sep 28 23:19:02 2006 From: ike at lesmuug.org (Isaac Levy) Date: Thu, 28 Sep 2006 23:19:02 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> <20060928200827.GP24150@clam.khaoz.org> Message-ID: <41027382-1B4C-41DF-8325-6002BC961199@lesmuug.org> On Sep 28, 2006, at 4:16 PM, Charles Sprickman wrote: > For what it's worth, I've been punishing a 3Ware 9550-SX-12 for the > past > few weeks, and it has pretty much made me a convert. Any new boxes we > build with hardware raid are getting these cards. They are beating > the > snot out of our Adaptec ZCR cards, both in performance and management. > > Things that would require reboots and visits to the BIOS config > when using > Adaptec were handled quite smoothly. Punishment included yanking a > bunch > of drives during a long bonnie run, rearranging the drives, yanking > power > during drive rebuilds, etc. And being able to change stripe sizes > without > wiping the array is a real plus. > > Sorry for the marketing, > > C Just to check, on FreeBSD Charles? Rocket- .ike From ike at lesmuug.org Thu Sep 28 23:21:11 2006 From: ike at lesmuug.org (Isaac Levy) Date: Thu, 28 Sep 2006 23:21:11 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> Message-ID: <83841A40-11C8-47F2-9589-02F5CABE6D30@lesmuug.org> On Sep 28, 2006, at 3:32 PM, Jonathan Vanasco wrote: > on a tangent... > > OSX has a neat tool called iPartition > http://www.coriolis-systems.com/iPartition.php > > something like that, under freebsd, would probably solve your issues. Well, HECK YEAH, that actually would solve my problems. Does anyone know if there's any historical attempt at flexible partition changes to a disk, and how successful it was? This has to have been a problem people have tackled since the earliest disk drives... Right? Rocket- .ike From spork at bway.net Thu Sep 28 23:25:04 2006 From: spork at bway.net (Charles Sprickman) Date: Thu, 28 Sep 2006 23:25:04 -0400 (EDT) Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <41027382-1B4C-41DF-8325-6002BC961199@lesmuug.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> <20060928200827.GP24150@clam.khaoz.org> <41027382-1B4C-41DF-8325-6002BC961199@lesmuug.org> Message-ID: On Thu, 28 Sep 2006, Isaac Levy wrote: > On Sep 28, 2006, at 4:16 PM, Charles Sprickman wrote: > >> For what it's worth, I've been punishing a 3Ware 9550-SX-12 for the past >> few weeks, and it has pretty much made me a convert. Any new boxes we >> build with hardware raid are getting these cards. They are beating the >> snot out of our Adaptec ZCR cards, both in performance and management. >> >> Things that would require reboots and visits to the BIOS config when using >> Adaptec were handled quite smoothly. Punishment included yanking a bunch >> of drives during a long bonnie run, rearranging the drives, yanking power >> during drive rebuilds, etc. And being able to change stripe sizes without >> wiping the array is a real plus. >> >> Sorry for the marketing, >> >> C > > Just to check, on FreeBSD Charles? Yessir. 6.1 on an Opteron box and then a 4 port version running 6.1 on a 1U P4 box. The 4 port also screams. I started with two drives and when I added two more speed nearly doubled (bonnie/pgbench). The web GUI is so good I actually use it. :) C > Rocket- > .ike > > > From dave at donnerjack.com Thu Sep 28 23:30:25 2006 From: dave at donnerjack.com (David Lawson) Date: Thu, 28 Sep 2006 23:30:25 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <5992C067-73B9-44E9-AB5C-8CA133BCF41D@lesmuug.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> <20060928200827.GP24150@clam.khaoz.org> <5992C067-73B9-44E9-AB5C-8CA133BCF41D@lesmuug.org> Message-ID: > > Well, erm- I've found the sata systems to be every bit as snappy at > this scale- (4 drives per 1u box). There shouldn't be much of a noticeable difference, honestly. SATA bandwidth is 300Mb/s, SCSI is 320Mb/s. I'm inclined to think that any observed difference in performance would be due to controller scaling issues or something similar, rather than the actual throughput on the devices. This changes when you compare SATA and SAS (Serial Attached SCSI), where there's an order of magnitude bandwidth difference between the two. I just noticed that Dell has started selling SAS drives and there appear to be quite a few hitting the market lately, I've got a pretty large server order in for boxes with them, so we'll see how they do. --Dave From spork at bway.net Thu Sep 28 23:31:59 2006 From: spork at bway.net (Charles Sprickman) Date: Thu, 28 Sep 2006 23:31:59 -0400 (EDT) Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <83841A40-11C8-47F2-9589-02F5CABE6D30@lesmuug.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> <83841A40-11C8-47F2-9589-02F5CABE6D30@lesmuug.org> Message-ID: On Thu, 28 Sep 2006, Isaac Levy wrote: > On Sep 28, 2006, at 3:32 PM, Jonathan Vanasco wrote: > >> on a tangent... >> >> OSX has a neat tool called iPartition >> http://www.coriolis-systems.com/iPartition.php >> >> something like that, under freebsd, would probably solve your issues. > > Well, HECK YEAH, that actually would solve my problems. Does anyone > know if there's any historical attempt at flexible partition changes > to a disk, and how successful it was? When I added some drives to one of the boxes mentioned before, I went from 250GB to 500GB. With some bsdlabel fiddling and growfs I successfully grew a partition with data on it. Drawbacks: -you can only grow a partition -you can only grow the last partition (plan ahead!) unless you want to dump/restore data on other partitions -the feedback about the reliability of growfs on -hackers was a little scary -it does not appear that anyone is currently working on it -with UFS2 it does something that kills the dynamic inode allocation stuff > This has to have been a problem people have tackled since the > earliest disk drives... Right? Do some of the Linux tools deal with BSD partitions? I think gpartd(?) is some sort of PartitionMagic clone. Charles > Rocket- > .ike > > > _______________________________________________ > % NYC*BUG talk mailing list > http://lists.nycbug.org/mailman/listinfo/talk > %Be sure to check out our Jobs and NYCBUG-announce lists > %We meet the first Wednesday of the month > From dave at donnerjack.com Thu Sep 28 23:32:47 2006 From: dave at donnerjack.com (David Lawson) Date: Thu, 28 Sep 2006 23:32:47 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <83841A40-11C8-47F2-9589-02F5CABE6D30@lesmuug.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> <83841A40-11C8-47F2-9589-02F5CABE6D30@lesmuug.org> Message-ID: On Sep 28, 2006, at 11:21 PM, Isaac Levy wrote: > On Sep 28, 2006, at 3:32 PM, Jonathan Vanasco wrote: > >> on a tangent... >> >> OSX has a neat tool called iPartition >> http://www.coriolis-systems.com/iPartition.php >> >> something like that, under freebsd, would probably solve your issues. > > Well, HECK YEAH, that actually would solve my problems. Does anyone > know if there's any historical attempt at flexible partition changes > to a disk, and how successful it was? > > This has to have been a problem people have tackled since the > earliest disk drives... Right? Well, yeah. Unfortunately, it's a problem that there aren't a whole lot of good solutions for, AFAIK. LVM under linux addresses some of this issue, but has some serious flaws in its current implementation. I've used gparted to good effect previously, I believe it supports UFS/UFS2, but I don't remember off the top of my head. --Dave From nycbug-list at 2xlp.com Thu Sep 28 23:37:06 2006 From: nycbug-list at 2xlp.com (Jonathan Vanasco) Date: Thu, 28 Sep 2006 23:37:06 -0400 Subject: [nycbug-talk] File Backed Disks- Speed Issues In-Reply-To: <83841A40-11C8-47F2-9589-02F5CABE6D30@lesmuug.org> References: <7891BCF3-6F9E-4B60-8DE3-7C7001DFE48C@lesmuug.org> <83841A40-11C8-47F2-9589-02F5CABE6D30@lesmuug.org> Message-ID: On Sep 28, 2006, at 11:21 PM, Isaac Levy wrote: > Well, HECK YEAH, that actually would solve my problems. Does > anyone know if there's any historical attempt at flexible partition > changes to a disk, and how successful it was? > > This has to have been a problem people have tackled since the > earliest disk drives... Right? I gotta say... if someone could do what iParition does on linux/bsd drives... it would be AMAZING. its sincerely an amazing app. shrink / grow / reorder partitions on live drives with no data loss. i was able to turn a half filled storage drive into a 2 partition disk- 1 with all the storage, the dead space into a bootable mirror of my g5's hard drive... and then reorder it so older macs will boot it ( first few will only boot off the first partition ). it only works on hfs though. suckass. From af.dingo at gmail.com Fri Sep 29 08:50:29 2006 From: af.dingo at gmail.com (Jeff Quast) Date: Fri, 29 Sep 2006 08:50:29 -0400 Subject: [nycbug-talk] what is the threat of the openssl advisory? In-Reply-To: <20060928232213.GQ13445@run.galis.org> References: <20060928232213.GQ13445@run.galis.org> Message-ID: On 9/28/06, George Georgalis wrote: > There was an openssl advisory today > > http://www.openssl.org/news/secadv_20060928.txt > http://security.freebsd.org/advisories/FreeBSD-SA-06:23.openssl.asc > > my primary concern is > > A buffer overflow was discovered in the SSL_get_shared_ciphers() > utility function. An attacker could send a list of ciphers to an > application that uses this function and overrun a buffer > (CVE-2006-3738). > > there is no comment on if an exploit is known to exist or how > difficult (or easy) it would be to create one based on the patch. > http://security.freebsd.org/patches/SA-06:23/ > > In fact the netbsd openssl looks pretty different than freebsd > in the context of applying the patch. Can we determine a level > of risk? Are all ssl, openvpn, ssh, https, etc servers needing > access restricted to friendly IPs or is the threat just one bit > inside "astronomically possible?" -- I cannot tell. > > // George For OpenSSH, to cite http://www.undeadly.org/cgi?action=article&sid=20060928025817&mode=expanded > Re: OpenSSH 4.4 released (mod 10/10) > by djm@ (IP 206.59.235.113) on Thu Sep 28 05:17:36 2006 (GMT) > > It is my understanding that OpenSSH relies on OpenSSL, > > but can we really trust OpenSSL? [etc...] > > OpenSSH doesn't trust OpenSSL for anything more than cryptographic > primitives. In particular, it avoids its default RSA signature verification > code that depends on the OpenSSL ASN.1 code - we use our own > minimal implementation instead (ssh-rsa.c). > > IIRC this has saved us from at least two bugs so far: an ASN.1 bug a > while ago and the new Bleichenbacher attack. > > Thanks Markus Friedl for this code :) of course, you should always review code yourself if this is such a serious issue. System administrators should be proficient in C for this very reason (and why I think recent 'network security' roles coming into corperations are full of smoke) From ike at lesmuug.org Fri Sep 29 10:12:31 2006 From: ike at lesmuug.org (Isaac Levy) Date: Fri, 29 Sep 2006 10:12:31 -0400 Subject: [nycbug-talk] what is the threat of the openssl advisory? In-Reply-To: References: <20060928232213.GQ13445@run.galis.org> Message-ID: <7408BC9C-0E31-4E12-A165-7E0B465A4A86@lesmuug.org> Hey Jeff, On Sep 29, 2006, at 8:50 AM, Jeff Quast wrote: >> Thanks Markus Friedl for this code :) > > of course, you should always review code yourself if this is such a > serious issue. System administrators should be proficient in C for > this very reason (and why I think recent 'network security' roles > coming into corperations are full of smoke) Completely tangent, but I'd argue System Administrators should be proficient in learning, more than being proficient in C. C knowledge is a great base, but practical examples from just the last 2 years of my life have led me to need to hack: C, C++, some Assembly and Forth (a RAID hack), A Java GUI app (swing lib hacking), PHP, ColdFusion, Perl, Javascript (x-site scripting), and some advanced Korn Shell scripts written long ago using *all* the bell and whistle feature of the shell. All of this, of course, had little or no documentation with the code at hand. For those who know me, you know I enjoy the Python programming language, I am not 'Proficient in C', so all the above mentioned stuff was done out of resolving some problem at hand. I don't think I could even speak halfway intelligently on most of the stuff above in a conversation- I've tossed most of that knowledge out of my brain. However, the experiences and the methodology remain- and to me, that's what's important. I'm not tooting my own horn here, but I'm saying I've seen far too many people who knew the C/C++, but couldn't think their way through solving a real problem- which usually just requires creativity. just my .02? Rocket- .ike From george at galis.org Fri Sep 29 12:07:42 2006 From: george at galis.org (George Georgalis) Date: Fri, 29 Sep 2006 12:07:42 -0400 Subject: [nycbug-talk] what is the threat of the openssl advisory? In-Reply-To: References: <20060928232213.GQ13445@run.galis.org> Message-ID: <20060929160742.GC8240@run.galis.org> On Fri, Sep 29, 2006 at 08:50:29AM -0400, Jeff Quast wrote: >On 9/28/06, George Georgalis wrote: >> There was an openssl advisory today >> >> http://www.openssl.org/news/secadv_20060928.txt >> http://security.freebsd.org/advisories/FreeBSD-SA-06:23.openssl.asc >> >> my primary concern is >> >> A buffer overflow was discovered in the SSL_get_shared_ciphers() >> utility function. An attacker could send a list of ciphers to an >> application that uses this function and overrun a buffer >> (CVE-2006-3738). >> >> there is no comment on if an exploit is known to exist or how >> difficult (or easy) it would be to create one based on the patch. >> http://security.freebsd.org/patches/SA-06:23/ >> >> In fact the netbsd openssl looks pretty different than freebsd >> in the context of applying the patch. Can we determine a level >> of risk? Are all ssl, openvpn, ssh, https, etc servers needing >> access restricted to friendly IPs or is the threat just one bit >> inside "astronomically possible?" -- I cannot tell. >> >> // George > >For OpenSSH, to cite >http://www.undeadly.org/cgi?action=article&sid=20060928025817&mode=expanded > >> Re: OpenSSH 4.4 released (mod 10/10) >> by djm@ (IP 206.59.235.113) on Thu Sep 28 05:17:36 2006 (GMT) >> > It is my understanding that OpenSSH relies on OpenSSL, >> > but can we really trust OpenSSL? [etc...] >> >> OpenSSH doesn't trust OpenSSL for anything more than cryptographic >> primitives. In particular, it avoids its default RSA signature verification >> code that depends on the OpenSSL ASN.1 code - we use our own >> minimal implementation instead (ssh-rsa.c). >> >> IIRC this has saved us from at least two bugs so far: an ASN.1 bug a >> while ago and the new Bleichenbacher attack. >> >> Thanks Markus Friedl for this code :) > >of course, you should always review code yourself if this is such a >serious issue. System administrators should be proficient in C for >this very reason (and why I think recent 'network security' roles >coming into corperations are full of smoke) Thanks, I didn't realize openssh didn't depend on openssl for this. (but aparently RedHat thinks it does) Nobody seems concerned apache-ssl and openvpn servers are vulnerable to "SSL_get_shared_ciphers() buffer overflow (CVE-2006-3738)" though. http://www.frsirt.com/english/CVE-2006-3738.php What the function does: http://www.mail-archive.com/openssl-dev at openssl.org/msg17001.html It is not clear to me if this is a server issue at all, or only a client issue (eg firefox or other applications that use ssl, https etc). seems the latter. Anyway the ASN.1/Bleichenbacher issue is not the CVE-2006-3738 issue I'm concerned with. Though it seems a client not a server issue. http://www.frsirt.com/english/CVE-2006-3738.php http://nvd.nist.gov/nvd.cfm?cvename=CVE-2006-3738 http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2006-3738 Agree? // George -- George Georgalis, systems architect, administrator < From md at mailq.de Sat Sep 30 15:46:04 2006 From: md at mailq.de (Mischa Diehm) Date: Sat, 30 Sep 2006 21:46:04 +0200 Subject: [nycbug-talk] Analyzing malicious SSH login attempts In-Reply-To: <45084CBA.80106@pkgsrc.org> References: <20060912095351.4266acc9@wit.genoverly.com> <45084CBA.80106@pkgsrc.org> Message-ID: <20060930194604.GA9855@mailq.de> On Wed, Sep 13, 2006 at 02:23:54PM -0400, Johnny Lam wrote: > Given the way that ssh-agent works (using sockets in /tmp/ssh-XXXXXXX), > the disadvantage is that you have to *really* trust every intermediate > machine through which you do agent forwarding. This is because anyone > with root access on any machine through which you do agent forwarding > can simply use your forwarded credentials because he can access that > socket file. the follwing option in ssh-add is useful in this case: -c Indicates that added identities should be subject to confirmation before being used for authentication. Confirmation is performed by the SSH_ASKPASS program mentioned below. Successful confirma- tion is signaled by a zero exit status from the SSH_ASKPASS pro- gram, rather than text entered into the requester. Mischa