[nycbug-talk] FreeBSD Dual homed

Thu Dec 20 21:26:39 EST 2007

I just figured it was worth a try :-). Someone mentioned they were DSL lines
and I thought the MTU would've been lower on a DSL interface. I've never
used FreeBSD with DSL so it was just a guess. It sounds like you're right
though.

We can't get PMTU on FreeBSD to work over the GRE tunnels between our data
centers. I've ran tcpdumps while transferring files and can see the ICMP
"destination unreachable, need to fragment" packets but FreeBSD doesn't seem
to do anything about them. On the other hand, a linux box on the same subnet
can transfer to the same destination. Actually, the only time FreeBSD does
work over the tunnels is when transferring to a linux box. My theory is that
linux is properly discovering the MTU and FreeBSD is getting the lower MTU
during the tcp handshake. If I'm wrong, I'd like to start a new thread
because I'd love to find a better fix than disabling PMTU.

On Dec 20, 2007 8:35 PM, Trish Lynch <trish at bsdunix.net> wrote:

> What does PMTU discovery have anything to do with the dual homes box - I
> think the one thing to try is enabling source routing.
>
> See - you can only have one default route - so all traffic only passes out
> one interface, when connecting to the other - the one without the route, the
> packets try and go out the *default* route instead of being routed out of
> the interface the connection is on.
>
> Turn on net.inet.ip.sourceroute,  net.inet.ip.accept_sourceroute, and
> net.inet.ip.forwarding and I bet it will work.
>
> (You can also use some other trickery as well, but this is more "correct",
> and yes I'm aware of the implications aregarding spoofed packets, but in
> order to be able to arbitrarily choose the route to use, these need to be
> anabled)
>
> FWIW , I've seen path MTU discovery work just fine on FreeBSD unless
> someone has blocked the ICMP packets needed for it to work. (Type 3, code
> 4). Most of the problems people report with PMTU are people being for lack
> of a better term, *clueless* and filtering ICMP indiscriminately. I'd still
> like to know what MTU has to do with the ability to not route over a certain
> network.
>
> -Trish
> --
> Trish Lynch
> M: 646-401-1405
> H: 201-378-0434
>
> -----Original Message-----
> From: "Brian McGonigle" <brian.mcgonigle at gmail.com>
>
> Date: Thu, 20 Dec 2007 20:19:35
> To:talk at lists.nycbug.org
> Subject: Re: [nycbug-talk] FreeBSD Dual homed
>
>
> Try lowering the MTU or disabling PMTU discovery. I have never seen PMTU
> discovery work on FreeBSD. I always use a lower MTU when going over a WAN.
>
>
> On Dec 20, 2007 4:24 PM, Rodrique Heron < swygue at gmail.com <mailto:
> swygue at gmail.com> > wrote:
>
>
>
> Dan Langille wrote:
> > ... I think I misunderstood you in my original reply.
> >
> > Rodrique Heron wrote:
> >> Hello all-
> >>
> >> Are there any known issues when FreeBSD is dual homed.
> >
> > There may be issues, but I have run dual homed FreeBSD since 1998.
> >
> >
> >> I have two interfaces, each connected to a different subnet. Whenever
> >> both are enabled I can't get any incoming network traffic to the
> server.
> >
> > Can you elaborate upon this? It's not clear what you are trying to do.
> >
> > So one NIC fails to work? No traffic in or out? Both nics?
> >
> > Output of netstat -na would help us understand.
> >
> > > I'm not
> >> routing between the two, therefore no "gateway_enable" in rc.conf. I
> >> don't have any firewalls enabled, I do have my defaultrouter set.
> >
> > I originally said:
> > AFAIK, you cannot route between the two UNLESS you have
> > gateway_enable="YES" in /etc/rc.conf.
> >
> > But what you mean is that you do not wish to route between the two
> > subnets. The FreeBSD box is not a gateway. It is merely dual homed.
> >
> >
> >
> >
> Hope this helps-
>
>
> # sockstat -4
> root sendmail 628 3 tcp4 127.0.0.1:25 <http://127.0.0.1:25>   *:*
> root sshd 609 4 tcp4 *:22 *:*
>
>
>
>
>
> # ifconfig -a
>
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>
>   options=b<RXCSUM,TXCSUM,VLAN_MTU>
>
>   inet
> 150.210.240.36 <http://150.210.240.36>  netmask 0xffffff00 broadcast
> 150.210.240.255 <http://150.210.240.255>
>
>   ether 00:14:22:23:1a:2e
>
>   media: Ethernet autoselect (1000baseTX <full-duplex>)
>
>   status: active
>
> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>
>   options=b<RXCSUM,TXCSUM,VLAN_MTU>
>
>   inet
> 150.210.160.243 <http://150.210.160.243>  netmask 0xffffff00 broadcast
> 150.210.160.255 <http://150.210.160.255>
>
>   ether 00:14:22:23:1a:2f
>
>   media: Ethernet autoselect (1000baseTX <full-duplex>)
>
>   status: active
>
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
>
>   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
>
>   inet6 ::1 prefixlen 128
>
>   inet
> 127.0.0.1 <http://127.0.0.1>  netmask 0xff000000
>
>
>
>
>
> # arp -an
>
> ? (150.210.160.204 <http://150.210.160.204> ) at 00:00:5e:00:01:04 on em1
> [ethernet]
>
> ? (
> 150.210.160.214 <http://150.210.160.214> ) at 00:0b:db:90:73:1f on em1
> [ethernet]
>
> ? (150.210.160.227 <http://150.210.160.227> ) at 00:11:43:ef:ba:36 on em1
> [ethernet]
>
> ? (
> 150.210.160.254 <http://150.210.160.254> ) at 00:00:0c:07:ac:04 on em1
> [ethernet]
>
> ? (150.210.240.32 <http://150.210.240.32> ) at 00:0c:29:62:78:63 on em0
> [ethernet]
>
> ? (
> 150.210.240.39 <http://150.210.240.39> ) at 00:0c:29:f8:e7:2c on em0
> [ethernet]
>
> ? (150.210.240.55 <http://150.210.240.55> ) at 00:1a:64:24:ce:bc on em0
> [ethernet]
>
>
>
>
>
> # netstat -rn -f inet
>
> Routing tables
>
>
>
> Internet:
>
> Destination Gateway Flags Refs Use Netif Expire
>
> default 150.210.160.254 <http://150.210.160.254>   UGS 0 415 em1
>
> 127.0.0.1 <http://127.0.0.1>   127.0.0.1 <http://127.0.0.1>   UH 0 0 lo0
>
> 150.210.160/24 link#2 UC 0 0 em1
>
> 150.210.160.204 <http://150.210.160.204>   00:00:5e:00:01:04 UHLW 1 18 em1
> 552
>
> 150.210.160.214 <http://150.210.160.214>   00:0b:db:90:73:1f UHLW 1 5 em1
> 654
>
> 150.210.160.227 <http://150.210.160.227>   00:11:43:ef:ba:36 UHLW 1 3 em1
> 747
>
> 150.210.160.254 <http://150.210.160.254>   00:00:0c:07:ac:04 UHLW 2 0 em1
> 547
>
> 150.210.240/24 link#1 UC 0 0 em0
>
> 150.210.240.32 <http://150.210.240.32>   00:0c:29:62:78:63 UHLW 1 6 em0
> 547
>
>
> 150.210.240.39 <http://150.210.240.39>   00:0c:29:f8:e7:2c UHLW 1 12 em0
> 547
>
> 150.210.240.55 <http://150.210.240.55>   00:1a:64:24:ce:bc UHLW 1 8 em0
> 743
>
>
>
>
>
>
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org <mailto:talk at lists.nycbug.org>
>
> http://lists.nycbug.org/mailman/listinfo/talk <
> http://lists.nycbug.org/mailman/listinfo/talk>
>
>  _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20071220/32bbcfab/attachment.htm>