Your first thought is probably to use the old fashioned floating static route where you have a weighted route that takes over if the primary route is withdrawn from the routing table.  That only works if the next-hop of that route is no longer available…like when serial interface goes down and the next-hop isn't directly connected any more.  This is Ethernet, though, so there's no way for the firewall doesn't know or doesn't care if a host on the network isn't there any more.  This config has another problem, too.  What about a scenario where the ISP's router is up, but it's interfaces are down?  How about if there are routing issues farther upstream?  You surely don't want to send traffic to a provider's router is the provider is having issues, right?  

If you've ever tried to do something similar on an IOS router, then you've probably done IP SLA.  An ASA has the same functionality, but it's just called SLA monitoring.  You wind up with a config that is a very similar to IP SLA stuff on IOS routers, actually.  I wrote a terrible blog post about it a few years back, and several other bloggers talk about it as well, but the idea is that you have a process, called an SLA monitor on the ASA, that monitors an IP address by pinging it.  You then create a track object that watches the monitor's status.  This track object is applied to a static route, and, if the SLA monitor fails, the route is removed from the routing table.  We've all done something like this with HSRP tracking, so this shouldn't be totally foreign.

Let's take a look at the test network that I've used to simulate the setup at the customer site.

The test is to have INSIDE1 communicate with TARGET.  Each ISP knows where TARGET is through a huge EIGRP AS, but we want to detect any routing problems on ISP1.  If we find a problem, we want to roll over to ISP2 on the BACKUP interface.  What do we monitor, though?  We can monitor the IP of the ISP's router at the data center, but we'd miss any issues upstream.  Let's monitor the IP of the second router on ISP1, which is 10.0.0.2.  In the real world, we'd fine a host somewhere deep on the Intertubes that we think won't go down very often.  In our test, 10.0.0.2 is the closest thing we can find to that.

Let's create a beautiful symphony of ICMP generation.  First, we create the SLA monitor.

sla monitor 1
 type echo protocol ipIcmpEcho 10.0.0.2 interface OUTSIDE
!
sla monitor schedule 1 life forever start-time now

I think you can see that we are creating an ICMP echo process that will ping 10.0.0.2 on the OUTSIDE interface.  The third line is what controls the start and stop of the process; in this case, we start now and don't ever finish thanks to the word forever.  We can't use the SLA monitor directly on our routes, so let's create a track object.

track 100 rtr 1 reachability

Now we have track object 100 that looks to SLA monitor 1 for reachability.  We apply this to the route just like we do on IOS.  We'll go ahead and add the weighted route as well.

route OUTSIDE 0.0.0.0 0.0.0.0 192.0.2.1 1 track 100
route BACKUP 0.0.0.0 0.0.0.0 192.0.2.129 240

Now the default will go through 192.0.2.1 until 10.0.0.2 is unreachable.  If that happens, the route is removed from the routing table, and the weighted default route will take over.  That's all you need.  Of course, I would create another track object for ISP2 so you can at least get a syslog message or SNMP trap if a problem happens over there, but you can probably get away with just the one.

If you've ever done IP SLA on a router, you would call me on the fact that there's some stuff missing.  If you don't force the ICMP packets to ISP1's router, the state of the SLA monitor will keep flopping; you flip to ISP2, the SLA check is healthy again, you flip back, the SLA check dies again…ad nauseum.  That's not the case for the ASA, actually.  Even though the default route has rolled over to the backup, the monitoring process continues to send requests to the old gateway.

Sometime I like it when my gear knows what I'm trying to do; this is one of those times.

Send any stray ICMP packets questions my way.

Audio Commentary

Aaron Conaway

I like to lean my head to the left, hit it with the palm of my right hand, and document what knowledge falls out.

Website - More Posts

The switched port analyzer (SPAN) is a mechanism on Cisco switches that allows you to take traffic on one port and copy it to another.  It’s generally used to get traffic to a sniffer or IDS for analysis, but it’s a great tool to use to sample traffic from a host for troubleshooting.

Let’s use a real-world example.  You’ve told your roommate to quit illegally downloading songs off the Internet, but you suspect he’s still doing it.  Instead of sneaking into his room at night and checking his machine, you can use a SPAN to copy his traffic to another switch interface where you can use tcpdump to record what’s happening.

Let’s say you have a 2950, and the roomie is on F0/1.  You have a Linux box plugged into F0/24 ready to capture the traffic.  Here’s what you do.

monitor session 1 source interface F0/1 both
monitor session 1 destination interface F0/24

This will create a new monitor session (that is, a SPAN session) that copies traffic from port F0/1 in both directions to port F0/24.  Now, when you run tcpdump on your Linux box, you see all the traffic coming in and going out of your roommate’s port.

That’s pretty easy, right?  You can have multiple sources ports by just adding more source lines or using ranges of ports.  You can also just copy received or transmitted traffic from a source.  Check out the contextual help for a little more info.

To see what’s going on, you can do a show monitor or a show monitor session 1 (depending on the IOS version).  You’ll see something like this.

switch#sh monitor
Session 1
---------
Type              : Local Session
Source Ports      :
Both          : Fa0/1
Destination Ports : Fa0/24
Encapsulation : Native
Ingress : Disabled

If you take a look at the destination port when the SPAN is running, you’ll see it’s in a state of up/down (monitoring).  I think you can figure out that this means we’re monitoring some traffic to this port.  Here’s what you’ll see if you look at the port.

switch#sh int f0/24
FastEthernet0/24 is up, line protocol is down (monitoring)
...

There are two big things to keep in mind when doing SPANs.  The first is that monitoring a port can drive CPU utilization way up (depending on the platform and traffic volume), so you may run into problems if you have a bunch of SPANs going at the same time.  Related to this is the fact that, if your switch has to decide between switching and copying traffic, it will stop copying until there’s enough CPU headroom to do that safely, and you’ll lose packets in the meantime.  It’s a switch — not a copier.

The second thing to keep in mind involves those little voices in your head called ethics.  What if you see a VOIP phone call from your boss to the HR department?  How about if you find someone in upper management copying a spreadsheet of people to be fired tomorrow?  How about if you find an engineer’s telnet password to a key system?  These are things that you probably shouldn’t see, so be careful when looking at the packets.  I would suggest you tell someone in your security when you’re going to do a packet capture to make sure someone knows you’re not up to no good.

Send shamrocks questions my way.

Aaron Conaway

I like to lean my head to the left, hit it with the palm of my right hand, and document what knowledge falls out.

Website - More Posts