ERS:8600/8800 VSP-8284XSQ (4.2.0.0) :Shutdown port due to excessive control frames multicast 0, broadcast 26484 packet per second

Doc ID    SOLN208208
Version:    3.0
Status:    Published
Published date:    24 Jun 2016
Created Date:    13 Aug 2012
Author:   
Gaurav Gupta
 

Details

ERS 8600 connected with ERS 4500 switch on port 2/6 port is getting shut down in every hour.
ERS 4500 was running on version 5.3 and was connected further to the edge switches of ES470 stack through which security camera and other printers and server is connected.

VSP 8284XSQ was running version 4.2.0 and was connected to edge - SNMP INFO Port 1/6 is a trunk port

Problem Clarification

2/6 port on ERS 8600 shutting down  every hour ,then to recover it administratively enable and disable was done  .On the other end of 2/6 port  ,ERS 4500 was connected which further connected to the ES470 stack and through that security cameras and users devices are connected.

Traffic flowing on the port is multicast , Broadcast and unicast but in the log messages broadcast packet was seen.

CPU5 [07/30/12 17:33:37] CPU WARNING Shutdown port 2/6 due to excessive control frames multicast 0, broadcast 26484 packet per second

Cause

In the log messages on ERS 8600 when we checked the reason why the port is getting shut it says.

CPU5 [07/30/12 17:33:37] SNMP INFO Link Down(2/6) due to excessive control frames
CPU5 [07/30/12 17:33:37] SNMP INFO Port 2/6 is a trunk port
CPU5 [07/30/12 17:33:37] CPU WARNING Shutdown port 2/6 due to excessive control frames multicast 0, broadcast 26484 packet per second

By default , the  maximum number of  broadcast packets which can be sent on port 2/6 is 10000 pps and in our case  it is  26000 pps which is well above the default threshold. Hence shutting down the port when this threshold is reached.
 

Solution

Taken the PCAP traces by mirroring the port 2/6 with port 3/15 with the help of following commands.
diag mirror-by-port 1 create in-port 2/2 or 2/6 out-port 3/15
diag mirror-by-port 1 mode both

We noticed on port  2/6  the maximum number of broadcasts were  coming from 10.237.0.0 subnet. This was  further isolated to a few servers which were  newly added to the existing network , causing the broadcasts.  Once these servers were isolated from the network everything worked fine.Also as best practice "rate-limit" should be configured on the edge device to prevent excessive Broadcast/Multicast packets from hitting the core switch(in this case, the ERS 8600)

 

Another feature known as Auto-recovery can be used.When a port is shutdown by CP-limit , you can use auto-recovery to automatically bring the port up after a pre configured delay.

Syntax: config ethernet auto-recover-port enable 

             config ethernet auto-recover-delay


Avaya -- Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy