I just figured it was worth a try :-). Someone mentioned they were DSL lines and I thought the MTU would've been lower on a DSL interface. I've never used FreeBSD with DSL so it was just a guess. It sounds like you're right though.
<br><br>We can't get PMTU on FreeBSD to work over the GRE tunnels between our data centers. I've ran tcpdumps while transferring files and can see the ICMP "destination unreachable, need to fragment" packets but FreeBSD doesn't seem to do anything about them. On the other hand, a linux box on the same subnet can transfer to the same destination. Actually, the only time FreeBSD does work over the tunnels is when transferring to a linux box. My theory is that linux is properly discovering the MTU and FreeBSD is getting the lower MTU during the tcp handshake. If I'm wrong, I'd like to start a new thread because I'd love to find a better fix than disabling PMTU.
<br><br><div class="gmail_quote">On Dec 20, 2007 8:35 PM, Trish Lynch <<a href="mailto:trish@bsdunix.net">trish@bsdunix.net</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
What does PMTU discovery have anything to do with the dual homes box - I think the one thing to try is enabling source routing.<br><br>See - you can only have one default route - so all traffic only passes out one interface, when connecting to the other - the one without the route, the packets try and go out the *default* route instead of being routed out of the interface the connection is on.
<br><br>Turn on net.inet.ip.sourceroute, net.inet.ip.accept_sourceroute, and net.inet.ip.forwarding and I bet it will work.<br><br>(You can also use some other trickery as well, but this is more "correct", and yes I'm aware of the implications aregarding spoofed packets, but in order to be able to arbitrarily choose the route to use, these need to be anabled)
<br><br>FWIW , I've seen path MTU discovery work just fine on FreeBSD unless someone has blocked the ICMP packets needed for it to work. (Type 3, code 4). Most of the problems people report with PMTU are people being for lack of a better term, *clueless* and filtering ICMP indiscriminately. I'd still like to know what MTU has to do with the ability to not route over a certain network.
<br><br>-Trish<br>--<br>Trish Lynch<br>M: 646-401-1405<br>H: 201-378-0434<br><br>-----Original Message-----<br>From: "Brian McGonigle" <<a href="mailto:brian.mcgonigle@gmail.com">brian.mcgonigle@gmail.com</a>
><br><br>Date: Thu, 20 Dec 2007 20:19:35<br><a href="mailto:To:talk@lists.nycbug.org">To:talk@lists.nycbug.org</a><br>Subject: Re: [nycbug-talk] FreeBSD Dual homed<br><br><br>Try lowering the MTU or disabling PMTU discovery. I have never seen PMTU discovery work on FreeBSD. I always use a lower MTU when going over a WAN.
<br><br><br>On Dec 20, 2007 4:24 PM, Rodrique Heron < <a href="mailto:swygue@gmail.com">swygue@gmail.com</a> <mailto:<a href="mailto:swygue@gmail.com">swygue@gmail.com</a>> > wrote:<br><br><br><br>Dan Langille wrote:
<br>> ... I think I misunderstood you in my original reply.<br>><br>> Rodrique Heron wrote:<br>>> Hello all-<br>>><br>>> Are there any known issues when FreeBSD is dual homed.<br>><br>> There may be issues, but I have run dual homed FreeBSD since 1998.
<br>><br>><br>>> I have two interfaces, each connected to a different subnet. Whenever<br>>> both are enabled I can't get any incoming network traffic to the server.<br>><br>> Can you elaborate upon this? It's not clear what you are trying to do.
<br>><br>> So one NIC fails to work? No traffic in or out? Both nics?<br>><br>> Output of netstat -na would help us understand.<br>><br>> > I'm not<br>>> routing between the two, therefore no "gateway_enable" in
rc.conf. I<br>>> don't have any firewalls enabled, I do have my defaultrouter set.<br>><br>> I originally said:<br>> AFAIK, you cannot route between the two UNLESS you have<br>> gateway_enable="YES" in /etc/rc.conf.
<br>><br>> But what you mean is that you do not wish to route between the two<br>> subnets. The FreeBSD box is not a gateway. It is merely dual homed.<br>><br>><br>><br>><br>Hope this helps-<br><br><br>
# sockstat -4<br>root sendmail 628 3 tcp4 <a href="http://127.0.0.1:25" target="_blank">127.0.0.1:25</a> <<a href="http://127.0.0.1:25" target="_blank">http://127.0.0.1:25</a>> *:*<br>root sshd 609 4 tcp4 *:22 *:*
<br><br><br><br><br><br># ifconfig -a<br><br>em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500<br><br> options=b<RXCSUM,TXCSUM,VLAN_MTU><br><br> inet<br><a href="http://150.210.240.36" target="_blank">
150.210.240.36</a> <<a href="http://150.210.240.36" target="_blank">http://150.210.240.36</a>> netmask 0xffffff00 broadcast <a href="http://150.210.240.255" target="_blank">150.210.240.255</a> <<a href="http://150.210.240.255" target="_blank">
http://150.210.240.255</a>><br><br> ether 00:14:22:23:1a:2e<br><br> media: Ethernet autoselect (1000baseTX <full-duplex>)<br><br> status: active<br><br>em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
<br><br> options=b<RXCSUM,TXCSUM,VLAN_MTU><br><br> inet<br><a href="http://150.210.160.243" target="_blank">150.210.160.243</a> <<a href="http://150.210.160.243" target="_blank">http://150.210.160.243</a>> netmask 0xffffff00 broadcast
<a href="http://150.210.160.255" target="_blank">150.210.160.255</a> <<a href="http://150.210.160.255" target="_blank">http://150.210.160.255</a>><br><br> ether 00:14:22:23:1a:2f<br><br> media: Ethernet autoselect (1000baseTX <full-duplex>)
<br><br> status: active<br><br>lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384<br><br> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3<br><br> inet6 ::1 prefixlen 128<br><br> inet<br><a href="http://127.0.0.1" target="_blank">
127.0.0.1</a> <<a href="http://127.0.0.1" target="_blank">http://127.0.0.1</a>> netmask 0xff000000<br><br><br><br><br><br># arp -an<br><br>? (<a href="http://150.210.160.204" target="_blank">150.210.160.204</a> <
<a href="http://150.210.160.204" target="_blank">http://150.210.160.204</a>> ) at 00:00:5e:00:01:04 on em1 [ethernet]<br><br>? (<br><a href="http://150.210.160.214" target="_blank">150.210.160.214</a> <<a href="http://150.210.160.214" target="_blank">
http://150.210.160.214</a>> ) at 00:0b:db:90:73:1f on em1 [ethernet]<br><br>? (<a href="http://150.210.160.227" target="_blank">150.210.160.227</a> <<a href="http://150.210.160.227" target="_blank">http://150.210.160.227
</a>> ) at 00:11:43:ef:ba:36 on em1 [ethernet]<br><br>? (<br><a href="http://150.210.160.254" target="_blank">150.210.160.254</a> <<a href="http://150.210.160.254" target="_blank">http://150.210.160.254</a>> ) at 00:00:0c:07:ac:04 on em1 [ethernet]
<br><br>? (<a href="http://150.210.240.32" target="_blank">150.210.240.32</a> <<a href="http://150.210.240.32" target="_blank">http://150.210.240.32</a>> ) at 00:0c:29:62:78:63 on em0 [ethernet]<br><br>? (<br><a href="http://150.210.240.39" target="_blank">
150.210.240.39</a> <<a href="http://150.210.240.39" target="_blank">http://150.210.240.39</a>> ) at 00:0c:29:f8:e7:2c on em0 [ethernet]<br><br>? (<a href="http://150.210.240.55" target="_blank">150.210.240.55</a> <
<a href="http://150.210.240.55" target="_blank">http://150.210.240.55</a>> ) at 00:1a:64:24:ce:bc on em0 [ethernet]<br><br><br><br><br><br># netstat -rn -f inet<br><br>Routing tables<br><br><br><br>Internet:<br><br>Destination Gateway Flags Refs Use Netif Expire
<br><br>default <a href="http://150.210.160.254" target="_blank">150.210.160.254</a> <<a href="http://150.210.160.254" target="_blank">http://150.210.160.254</a>> UGS 0 415 em1<br><br><a href="http://127.0.0.1" target="_blank">
127.0.0.1</a> <<a href="http://127.0.0.1" target="_blank">http://127.0.0.1</a>> <a href="http://127.0.0.1" target="_blank">127.0.0.1</a> <<a href="http://127.0.0.1" target="_blank">http://127.0.0.1</a>> UH 0 0 lo0
<br><br>150.210.160/24 link#2 UC 0 0 em1<br><br><a href="http://150.210.160.204" target="_blank">150.210.160.204</a> <<a href="http://150.210.160.204" target="_blank">http://150.210.160.204</a>> 00:00:5e:00:01:04 UHLW 1 18 em1 552
<br><br><a href="http://150.210.160.214" target="_blank">150.210.160.214</a> <<a href="http://150.210.160.214" target="_blank">http://150.210.160.214</a>> 00:0b:db:90:73:1f UHLW 1 5 em1 654<br><br><a href="http://150.210.160.227" target="_blank">
150.210.160.227</a> <<a href="http://150.210.160.227" target="_blank">http://150.210.160.227</a>> 00:11:43:ef:ba:36 UHLW 1 3 em1 747<br><br><a href="http://150.210.160.254" target="_blank">150.210.160.254</a>
<<a href="http://150.210.160.254" target="_blank">http://150.210.160.254</a>> 00:00:0c:07:ac:04 UHLW 2 0 em1 547<br><br>150.210.240/24 link#1 UC 0 0 em0<br><br><a href="http://150.210.240.32" target="_blank">
150.210.240.32</a> <<a href="http://150.210.240.32" target="_blank">http://150.210.240.32</a>> 00:0c:29:62:78:63 UHLW 1 6 em0 547<br><br><br><a href="http://150.210.240.39" target="_blank">150.210.240.39</a>
<<a href="http://150.210.240.39" target="_blank">http://150.210.240.39</a>> 00:0c:29:f8:e7:2c UHLW 1 12 em0 547<br><br><a href="http://150.210.240.55" target="_blank">150.210.240.55</a> <<a href="http://150.210.240.55" target="_blank">
http://150.210.240.55</a>> 00:1a:64:24:ce:bc UHLW 1 8 em0 743<br><br><br><br><br><br><br><br>_______________________________________________<br>talk mailing list<br><a href="mailto:talk@lists.nycbug.org">talk@lists.nycbug.org
</a> <mailto:<a href="mailto:talk@lists.nycbug.org">talk@lists.nycbug.org</a>><br><br><a href="http://lists.nycbug.org/mailman/listinfo/talk" target="_blank">http://lists.nycbug.org/mailman/listinfo/talk</a> <<a href="http://lists.nycbug.org/mailman/listinfo/talk" target="_blank">
http://lists.nycbug.org/mailman/listinfo/talk</a>><br><br> _______________________________________________<br>talk mailing list<br><a href="mailto:talk@lists.nycbug.org">talk@lists.nycbug.org</a><br><a href="http://lists.nycbug.org/mailman/listinfo/talk" target="_blank">
http://lists.nycbug.org/mailman/listinfo/talk</a><br><br></blockquote></div><br>