What impact will a broadcast storm have on the Nortel Switched Firewall Accelerator?

Doc ID    SOLN176876
Version:    2.0
Status:    Published
Published date:    06 May 2012
Created Date:    07 Apr 2005
Author:   
Content Migrator
 

Details

NORT049668

Problem Clarification


Goal:
What impact will a broadcast storm have on the Nortel Switched Firewall Accelerator?


Fact:
Enterprise Data
Switched Firewalls
Nortel Switched Firewall
Alteon Firewall
Nortel Switched Firewall Accelerator
Alteon Switched Firewall Accelerator


Symptom:
VRRP log messages indicating the Accelerators are changing from BACKUP TO MASTER.
NOTICE: Accelerator 00:60:cf:ae:08:00's vrrp status changed to MASTER
Default gateways are reported as down or not in use
Default gateway 1 (192.168.0.1) is down
Acceleration is switching from on to off to on again
CRITICAL: Acceleration Status changed from ACCELERATING to NOT ACCELERATING


Cause:
Broadcast packets are not handled by the SPs. Instead, broadcast packets get forwarded to the MP for processing. A broadcast storm can bring the MP to 100% utilization. When the MP is running at 100% dealing with the Broadcast packets, it will not be able to send VRRP updates, default gateway health checks, accelerator health checks, and others and so on.



Solution


Fix:
The first step is to determine that you are under a Broadcast Storm.
1) Check the port Stats by running /stats/port #/if on the Accelerator. Here is the output

Interface statistics for port 3:
ifHCIn Counters ifHCOut Counters
Octets: 0 0
UcastPkts: 1019 0
BroadcastPkts: 12602991 0
MulticastPkts: 78065 0
Discards: 566 0
Errors: 0 0
ifInUnknownProtos: 0

In the above capture, you will see the BroadcastPkts count is much higher than the UcastPkts or the MulticastPkts. Normal traffic typically has the Ucast as the highest count.

2) You can also check the MP Stats at the time of the attack with the /stats/mp/cpu command. Here is a sample output showing the MP at 100% utilization.

>> MP-specific Statistics# cpu
------------------------------------------------------------------
CPU utilization:
cpuAUtil1Second: 100% cpuBUtil1Second: 100%
cpuAUtil4Seconds: 100% cpuBUtil4Seconds: 100%
cpuAUtil64Seconds: 97% cpuBUtil64Seconds: 98%

3) Determine the source of the Broadcast Storms. Set up a Sniffer on the suspected port and look for broadcast packets. Broadcast packets do not have IP addresses, only source and destination MAC. In a Broadcast packet, the destination MAC is FF:FF:FF:FF:FF:FF

4) When the source is determined, you will need to troubleshoot the offending device to find out why it is sending a Broadcast Storm


Legacy ID

KB90049668

Avaya -- Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy