[talk] Who's built redundant pfsense setups?

[Isaac (.ike) Levy]

On Thu, 4 Dec 2014 10:22:22 -0500
Justin Sherrill <justin at shiningsilence.com> wrote:

> After last year's NYCBSDCon demo, and after having a Cisco ASA crap
> out, I'm trying to put together a redundant pfsense setup that
> handles multiple WANs, NAT, etc.
> 
> Looking at the pfsense Definitive Guide, it details multi-WAN setups
> and also how to get multiple switches in there.  I think/hope I can
> figure that out, but I remember ike or someone saying that when you
> had multiple switches, you needed to make sure they... could share an
> ARP table?

For the same logical link, (e.g. one WAN upstream), the switches do
need to share ARP.  Any switch capable of LAGG/LACP/etc... some sort of
link aggregation, will first and foremost share ARP between the two
physical switches- and become one logical switch.  Even inexpensive
switches are capable of some form of link aggregation.
Adding more than one switch, you'll have to get STP/TRILL or other tech
into the mix, and I could certainly share some recent experiences with
Brocade/10Gb ethernet fun, but at this point it all gets into serious
bucks- and serious planning.

For your model, however, it seems that with multiple upstream WAN's,
you may be better served by single (unmanaged?) switches connected to
the provider uplinks, to allow you to layer CARP failover on top of
Multi-WAN connectivity.

Inside your network, on your LAN, that's perhaps where 2x LAGG'd
switches (acting as one logical switch) would complete redundancy out
to your hosts.

Your hosts, in turn, could connect redundantly via lagg interfaces
(BSD's), or linux bonding.

It's already a lot of complexity there :)

> 
> I don't remember exactly, but I was hoping someone here could make a
> hardware recommendation for what switch to use before my wallet flops
> out.

If you don't have to go larger than 2 directly connected switches, your
wallet won't get plundered too bad.  Add a 3rd switch to your L2 setup,
and the cost/complexity shoots up fast.

I'm merely going to make a stab at sortof not answering your question:

On the Cheap: I dig on the Netgear stuff.  They even make small 8 port
gigabit switches which have LAGG capabilities.  (This is the sort of
gear I and my team prototype on- since it fits in a backpack, and on a
desk).

On the low end of the very high end, just had some excellent experience
with Brocade- mostly because their 10Gbit ethernet is way ahead of
competitors (in price and specs).  Also, Brocades use TRILL over STP
which is great for preventing loops/accidents- but it's all not very
transparent with their fabric 'magic config'.

If your networks are all Gigabit or below, you can really do this whole
thing with whatever you want- HP Procurve (gah) will work just fine,
Cisco gear is easy to come by used (and Michael Lucas's "Cisco
Routers for the Desperate" is a great companion), Juniper gear has a
nice FreeBSD shell exposed below JunOS, etc... 

Hope these anecdotes help, even though I didn't really answer directly
here...

Best,
.ike