[nycbug-talk] breaking up a big cisco

[N.J. Thomas]

We have a (huge) Cisco 6506 switch with a routing module that handles
all our network traffic, bgp, etc.

Roughly, our network looks something like this:

    [Tier1]         [Tier1]
       |               |
       +---+      +----+
           |      |
      +----+------+---+
      |  Cisco  6506  |
      +-+--+-------+--+
        |  |       |
       S1  S2 ...  S50
   (About 50 servers hooked up directly into the Cisco)

The problem with this whole setup is threefold:

    - it is big (about 12 or 16u I think)
    - it is quite expensive (>$50k a couple of years ago)
    - it is a single point of failure

Will be expanding in the near future, adding another Tier1, as well as
anticipating 50-100% growth in the number of servers in about 2-3 years
time. The 6506 being so large and expensive is not a major issue in and
of itself, but because it is a SPOF, its size and cost does affect our
plans.

So the proposal was to split the whole thing up. Roughly, the
architecture would be:

    - arrange servers into 4 cabinets

    - put a switch in each cabinet (Cisco 2960?), to which each server
      would be connected

    - have the carriers go into a switch which feeds into a smaller
      router (Cisco 2821?) which would go out to the switches which the
      servers are connected to

    - have spares available on hand for all switches and routers in case
      something fails

Our proposed network would probably look roughly something like this:

    [Tier1] [Tier1] [Tier1]
       |      |       |
       +-+    +    +--+
         |    |    |
      +--+----+----+--+   +--+----+----+--+
      |  Cisco  2960  |   |  Cisco  2960  |
      +-------+-------+   +-------+-------+ --- servers
              |                   |
      +-------+-------+   +--+----+----+--+
      |  Cisco  2821  |---|  Cisco  2960  | --- servers
      +---------------+   +-------+-------+
                                  |
                          +--+----+----+--+
                          |  Cisco  2960  | --- servers
                          +-------+-------+

Is this a viable solution? How would you modify it to provide some
measure of redundancy?

Another thing I am worried about is the whether or not the 2821 router
can handle 3 carriers. Our traffic needs are fairly low, our Tier 1
carriers are both 5Mbit burstable, and we rarely go above 15Mbit during
peak times. We'd be adding another carrier only for redundancy, not for
the bandwidth. However, because we are an ASN running BGP, I'm not sure
if the 2821 router would be constrained by memory limitations.

thanks,
Thomas

-- 
N.J. Thomas
njt at ayvali.org
Etiamsi occiderit me, in ipso sperabo