AES/CM:customer reported issue for device link lost connection with PBX Nice logger "Error:8805 LinkController lost connection with PBX " getting error in Nice logger

Doc ID		SOLN306487
Version:		7.0
Status:		Published
Published date:		26 Jan 2023
Created Date:		16 Mar 2017

Author:

nmokhle

Details

For Reference matrix INC000011626156 customer reported issue for device link lost connection with PBX Nice logger
"Error:8805 LinkController lost connection with PBX " getting error in Nice logger
An unexpected disconnection of a link controller with the PBX is detected causing deregistarion of the Extension.

Problem Clarification

Customer reported issue for device link lost connection with PBX Nice logger
"Error:8805 LinkController lost connection with PBX " getting error in Nice logger
An unexpected disconnection of a link controller with the PBX is detected causing deregistarion of the Extension.

AES version 6.3.3

Cause

Had checked AES logs and got the link lost for third party Nice logger
cd /opt/mvap/logs/sec.log

need to check from Nice side getting drop of link from third party at 12:16:30 system time.

12:16:30.763 SECURITY:FYI:TSAPI:tsviAuthenticateSession:user Cybertech logged in to AVAYA#BOMBKC01ACM01V#CSTA-

S#PNQDIG01AES01-CTI from 10.196.244.44 successfully
12:16:31.280 SECURITY:FYI:TSAPI:ClientHandler::streamClosed():user Cybertech with IP Address 10.196.244.44

disconnected from driver service AVAYA#BOMBKC01ACM01V#CSTA-S#PNQDIG01AES01-CTI
12:17:02.762 SECURITY:FYI:TSAPI:ClientHandler::streamClosed():user Cybertech with IP Address 10.196.244.44

disconnected from driver service AVAYA#BOMBKC01ACM01V#CSTA-S#PNQDIG01AES01-CTI
12:17:15.314 SECURITY:FYI:TSAPI:tsviAuthenticateSession:user Cybertech logged in to AVAYA#BOMBKC01ACM01V#CSTA-

S#PNQDIG01AES01-CTI from 10.196.244.44 successfully
12:17:37.627 SECURITY:FYI:TSAPI:ClientHandler::streamClosed():user Cybertech with IP Address 10.196.244.44

disconnected from driver service AVAYA#BOMBKC01ACM01V#CSTA-S#PNQDIG01AES01-CTI
12:17:50.272 SECURITY:FYI:TSAPI:tsviAuthenticateSession:user Cybertech logged in to AVAYA#BOMBKC01ACM01V#CSTA-

S#PNQDIG01AES01-CTI from 10.196.244.44 successfully

2:cd /var/log/messages.logs
Further I had checked messages logs got no event of disconnections .Also no flap or disconnection from AES end

Mar 3 12:16:27 pnqdig01aes01-cti snmpd[3936]: Connection from UDP: [10.242.188.54]->[10.240.50.163]:-10173
Mar 3 12:16:28 pnqdig01aes01-cti last message repeated 3 times
Mar 3 12:16:28 pnqdig01aes01-cti snmpd[3936]: Connection from UDP: [10.242.188.54]->[10.240.50.163]:-10157
Mar 3 12:16:28 pnqdig01aes01-cti snmpd[3936]: Received SNMP packet(s) from UDP: [10.242.188.54]->

[10.240.50.163]:-10157
Mar 3 12:16:28 pnqdig01aes01-cti snmpd[3936]: Connection from UDP: [10.242.188.54]->[10.240.50.163]:-10157
Mar 3 12:16:29 pnqdig01aes01-cti last message repeated 3 times
Mar 3 12:16:29 pnqdig01aes01-cti snmpd[3936]: Connection from UDP: [10.242.188.54]->[10.240.50.163]:-10142
Mar 3 12:16:29 pnqdig01aes01-cti snmpd[3936]: Received SNMP packet(s) from UDP: [10.242.188.54]->

[10.240.50.163]:-10142
Mar 3 12:16:29 pnqdig01aes01-cti snmpd[3936]: Connection from UDP: [10.242.188.54]->[10.240.50.163]:-10142
Mar 3 12:16:29 pnqdig01aes01-cti last message repeated 3 times
Mar 3 12:16:29 pnqdig01aes01-cti snmpd[3936]: Connection from UDP: [10.242.188.54]->[10.240.50.163]:-11290
Mar 3 12:16:29 pnqdig01aes01-cti snmpd[3936]: Received SNMP packet(s) from UDP: [10.242.188.54]->

[10.240.50.163]:-11290
Mar 3 12:16:30 pnqdig01aes01-cti snmpd[3936]: Connection from UDP: [10.242.188.54]->[10.240.50.163]:-11290
Mar 3 12:16:58 pnqdig01aes01-cti snmpd[3936]: Connection from UDP: [10.242.188.54]->[10.240.50.163]:-14593

Solution

Next plan of action is to check from
1:Nice end for the third pary disconnection
2:Kindly check with network team if any connection flap during the time stamp
3:Need to check from CM end also if any deregistartion had happed during tie stamp .

4:So issue was from network flap during time stamp can see in below logs for deregistration /registration of station from CM logs
The network flapped and the end points are unregistered from main CM server.

5:But if still issue persist and Nice are getting link lost connection with PBX alarm event on Nice logger need to do sequential service restart of AES and Nice logger to have refresh sessions which

should be perform after business hrs since this will have business impact.:

Steps for Sequential restart of AES and Nice service

First :Nice services stop
Second :Stop AES service DMCC and TSAPI service from web page

Third: Start DMCC and TSAPI service on AES webpage

Fourth: Nice Service Start
then check DMCC registration in AES and Nice logger recording and monitor if no alarm is there on Nice logger

Mar 3 12:15:16 bombkc01acm01p logmanager: bg_mapa[20476]: ip-a s S94405 912266703231
Mar 3 12:15:23 bombkc01acm01p logmanager: IPEVT IPT_REG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703231 ip= 10.242.171.193;30347 net_reg= 241 reason=normal
Mar 3 12:15:15 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703231 the 1st ip=10.190.16.187; 1720 the 2nd ip=0.0.0.0; 0 net_reg= 3 reason=switch_request
Mar 3 12:15:16 bombkc01acm01p logmanager: bg_mapa[20476]: ip-a s S94405 912266703231
Mar 3 12:15:23 bombkc01acm01p logmanager: IPEVT IPT_REG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703231 ip= 10.242.171.193;30347 net_reg= 241 reason=normal
Mar 3 12:15:23 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703231 the 1st ip=10.242.171.193;22469 the 2nd ip=0.0.0.0; 0 net_reg= 241 reason=endpoint_request
Mar 3 12:16:31 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

911171109509 ip= 10.242.188.54;32121 net_reg= 242 reason=0
Mar 3 12:16:31 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912271804452 ip= 10.242.188.54;30524 net_reg= 242 reason=0
Mar 3 12:16:32 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912271804014 ip= 10.242.188.54;32382 net_reg= 242 reason=0
Mar 3 12:16:32 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266706277 ip= 10.242.188.54;33834 net_reg= 242 reason=0
Mar 3 12:16:32 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703745 ip= 10.242.188.54;30109 net_reg= 242 reason=0
Mar 3 12:16:32 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703797 ip= 10.242.188.54;30206 net_reg= 242 reason=0

Mar 3 12:16:33 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

911171109612 ip= 10.242.188.54;30493 net_reg= 242 reason=0
Mar 3 12:16:33 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

914471309612 ip= 10.242.188.54;31287 net_reg= 242 reason=0
Mar 3 12:16:33 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

911207141104 ip= 10.242.188.54;32565 net_reg= 242 reason=0
Mar 3 12:16:33 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703005 ip= 10.242.188.54;33712 net_reg= 242 reason=0
Mar 3 12:17:01 bombkc01acm01p crond[5404]: (root) CMD (/usr/sbin/logrotate /etc/opt/ecs/kerlog.conf)
Mar 3 12:18:30 bombkc01acm01p logmanager: IPEVT IPT_REG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

911171109509 ip= 10.242.188.54;32277 net_reg= 242 reason=normal
Mar 3 12:18:30 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

911171109509 the 1st ip=10.242.188.54;21018 the 2nd ip=0.0.0.0; 0 net_reg= 242 reason=endpoint_request
Mar 3 12:18:32 bombkc01acm01p logmanager: IPEVT IPT_REG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912271804452 ip= 10.242.188.54;32595 net_reg= 242 reason=normal
Mar 3 12:18:32 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912271804452 the 1st ip=10.242.188.54;23661 the 2nd ip=0.0.0.0; 0 net_reg= 242 reason=endpoint_request
Mar 3 12:18:34 bombkc01acm01p logmanager: IPEVT IPT_REG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912271804014 ip= 10.242.188.54;33288 net_reg= 242 reason=normal
Mar 3 12:18:34 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912271804014 the 1st ip=10.242.188.54;21959 the 2nd ip=0.0.0.0; 0 net_reg= 242 reason=endpoint_request
Mar 3 12:18:36 bombkc01acm01p logmanager: IPEVT IPT_REG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266706277 ip= 10.242.188.54;34901 net_reg= 242 reason=normal
Mar 3 12:16:32 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

917971015025 ip= 10.242.188.54;30444 net_reg= 242 reason=0
Mar 3 12:16:32 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

918071935525 ip= 10.242.188.54;34477 net_reg= 242 reason=0
Mar 3 12:16:32 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912071249005 ip= 10.242.188.54;30175 net_reg= 242 reason=0
Mar 3 12:16:33 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

911247138245 ip= 10.242.188.54;31141 net_reg= 242 reason=0

Mar 3 12:16:33 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

914471309612 ip= 10.242.188.54;31287 net_reg= 242 reason=0
Mar 3 12:16:33 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

911207141104 ip= 10.242.188.54;32565 net_reg= 242 reason=0
Mar 3 12:16:33 bombkc01acm01p logmanager: IPEVT IPT_UNREG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

911171109509 ip= 10.242.188.54;32277 net_reg= 242 reason=normal
Mar 3 12:18:30 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912271804452 ip= 10.242.188.54;32595 net_reg= 242 reason=normal
Mar 3 12:18:32 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912271804014 ip= 10.242.188.54;33288 net_reg= 242 reason=normal
Mar 3 12:18:34 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266706277 ip= 10.242.188.54;34901 net_reg= 242 reason=normal
Mar 3 12:18:36 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266706277 the 1st ip=10.242.188.54;24964 the 2nd ip=0.0.0.0; 0 net_reg= 242 reason=endpoint_request
Mar 3 12:18:38 bombkc01acm01p logmanager: IPEVT IPT_REG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703745 ip= 10.242.188.54;32924 net_reg= 242 reason=normal
Mar 3 12:18:38 bombkc01acm01p logmanager: IPEVT IPT_TCP_UP board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703745 the 1st ip=10.242.188.54;23086 the 2nd ip=0.0.0.0; 0 net_reg= 242 reason=endpoint_request
Mar 3 12:18:40 bombkc01acm01p logmanager: IPEVT IPT_REG board=PROCR ip=10.242.171.129 net_reg= 1 ext=

912266703797 ip= 10.242.188.54;31171 net_reg= 242 reason=normal

Attachment Description

1) Clarify the different between:

· From “TSAPI Link Details” page, what is the meaning of the field “Associations”?

· From “DMCC Service Summary – Session Summary” page, what is the meaning of “# of Associate Devices”?

2) The process when the logger start up, should it follow the below sequence to communicate with AES:

· Logger start a DMCC Session & open a TSAPI Stream with AE Services.

· Then build a DMCC device list in AE Services (DMCC/TSAPI).

· Logger start placing monitors on stations / VDN's / Hunt Group in Communication Manager (CM)

Does it mean that the logger need to send a second request to AES to start the station monitoring, after it setup the DMCC device list for registration. This is a 2-steps process. Please correct me if I’m wrong.

Additional Relevant Phrases

AES: NICE link dropped AES lost connection

Avaya -- Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy