CMS: link down between CM and CMS


Doc ID    SOLN304979
Version:    5.0
Status:    Published
Published date:    06 Oct 2020
Created Date:    16 Feb 2017
Author:   
Ming Jiang
 

Details

CMS all versions.

Problem Clarification

On CM there were some maintenance activity, and after that the CMS primary link drops.

Cause

CM channel 2 is having issues. Thus, socket closed, and not able to respond.

spi.err log shows:
hu Feb 16 12:13:15 [00] PBX  ApplQ sess down note
Thu Feb 16 12:13:16 [00] TCP  CLIENT DISCONNECTED
Thu Feb 16 12:13:16 [00] TCP  OPERATIONAL
Thu Feb 16 12:13:16 [00] PBX  ApplQ sess down note
Thu Feb 16 12:13:17 [00] TCP  read from client socket failed, rcode=-1 errno=104
Thu Feb 16 12:13:17 [00] TCP  OUT OF ORDER

 

 

 

elog shows:
01351 Tue Feb 14 17:21:31 2017  SRC_ERR_NUM=00000 PROCESS=cms_mon        PID=002 547  main.c:00246 SEVERITY=ERROR ACD=00 DUPS=00001 GENERAL error in process interface: death of 'spi' pid=2647 task_id=101 signal=11 core


01101 Tue Feb 14 17:21:32 2017  SRC_ERR_NUM=00022 PROCESS=spi            PID=023649                    shm.c:00415 SEVERITY=ERROR ACD=01 DUPS=00001 IPC error on shared memory: shmdt

01350 Tue Feb 14 17:21:32 2017  SRC_ERR_NUM=00001 PROCESS=link_watch     PID=002 586 AcdStatus.c:00157 SEVERITY=INFO  ACD=01 DUPS=00001 GENERAL error internal to process: Link is down: acd  1

 

When telnet, system shows:
(hostname)=# telnet switch 5001
Trying xx.xx.xx.xx
Connected to switch.
Escape character is '^]'.
Connection closed by foreign host.

And port displayed as CLOSE_WAIT


(hostname)-(Primary)=# netstat -tunap | grep 5001
tcp        0      0 xx.xx.yy.yy:41959         xx.xx.yy.yy:5001          CLOSE_WAIT  25089/spi

Solution

 

 

CM communication channel shows "socket out of service"
status processor-c 2
LinkStatus: Socket out of services:Thus we decided, either processor-channel or interface-channel are bad.

Note: If error= 32 would indicate the wrong CMS IP Address in the Communication Manager.

(hostname)-(Primary)=# netstat -tunap | grep 5003
tcp      264      0 172.29.171.34:46193         172.29.171.26:5003          EST

 

The link comes back after the following changes:

1. Change the processor-channel from 2 to 4
2. Change the interface channel from 5001 to 5003

 


Avaya -- Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy