1st error is related to 3rd error. VSP reported ‘cannot allocate memory’ then cause tomcat died and reboot to restore the service.VSP memory issue is a known issue. No fix plan for this. Generally, it will affect the applications of tomcat. For example,
a. CM/AES will report ‘LIC-ERROR’
b. CM service crond restart
c. Heartbeat failure between CM and VSP. That won’t cause CM interchange as of CM6.3.11.
VSP-logs:
2017-11-10 07:29:07,619 ERROR [com.avaya.vsp.webconsole.virtapi.libvirt.DefaultConnectionFactory] ---- pool ----: DefaultConnectionFactory createConnect - Exception log cannot fork child process: Cannot allocate memory
org.libvirt.LibvirtException: cannot fork child process: Cannot allocate memory
at org.libvirt.ErrorHandler.processError(Unknown Source)
at org.libvirt.Connect.processError(Unknown Source)
at org.libvirt.Connect.<init>(Unknown Source)
vsp-alarm
<139>Nov 10 07:30:44 kccsps2f vsp.alarm: +09:00 2017 344 1 com.avaya.vsp | 2 com.avaya.vsp.syslog O_AVPC240000 CDom Webconsole tomcat died.
vsp-history
<158>Nov 10 07:30:44 kccsps2f logger: +09:00 2017 371 1 com.avaya.vsp | 0 CDom Webconsole tomcat died.
<158>Nov 10 07:30:44 kccsps2f ./run: +09:00 2017 378 1 com.avaya.vsp | 0 Starting Webconsole tomcat.
Normally there is watchdog function between CM and CDOM, CM use the scripts /opt/vsp/vsp_trigger to curl the CDOM webpage, if CM could curl which means the connection between CM and CDOM is working well. From CM messages and ecs logs, CM cannot curl Cdom due to tomcat reboot. So CM initiate “service crond restart”.
Nov 10 07:30:42 kccsps3f /opt/vsp/vsp_trigger: cannot send watchdog trigger to https://cdom.vsp:9443/webconsole/vm/heartbeat; curl returned:
Nov 10 07:30:42 kccsps3f /opt/vsp/vsp_trigger: cannot send watchdog trigger to https://cdom.vsp:9443/webconsole/vm/heartbeat; curl returned:
Nov 10 07:30:42 kccsps3f /opt/vsp/vsp_trigger: cannot send watchdog trigger to https://cdom.vsp:9443/webconsole/vm/heartbeat; curl returned:
Nov 10 07:30:42 kccsps3f /opt/vsp/vsp_trigger: cannot send watchdog trigger to https://cdom.vsp:9443/webconsole/vm/heartbeat; curl returned:
Nov 10 07:31:05 kccsps3f nwc_refresh: entered
Nov 10 07:31:10 kccsps3f nwc_refresh: entered
Nov 10 07:31:10 kccsps3f vsp_init: reload..
Nov 10 07:31:10 kccsps3f nwc_get_changes: authentication file is identical with lacfile: do not install
Nov 10 07:31:10 kccsps3f nwc_get_changes: get CM specific params..
Nov 10 07:31:10 kccsps3f nwc_get_changes: get common params..
Nov 10 07:31:11 kccsps3f nwc_do_generic: set SSH key
Nov 10 07:31:11 kccsps3f nwc_do_generic: set timezone
Nov 10 07:31:11 kccsps3f crond[4148]: (CRON) STARTUP (V5.0)
Nov 10 07:31:11 kccsps3f crond[4148]: (CRON) @reboot jobs will be run at computer's startup. ()
Nov 10 07:31:11 kccsps3f watchd: service crond was successfully restarted.
Nov 10 07:31:11 kccsps3f watchd: maintananceMng(life-cycle): service crond is UP
2nd error is related to services_vm. That is an error when creating certificate for services_vm.
Services_vm is a default template on VSP, but customer can select don’t install this one.
For process reboot, it still will check it. It is normal.
