The issue was caused by the time change on the PBX side, we could find quite some error messages like below in the CMS elog file, need to check the time and ntp service for this ACD from the CM side.
Two scenarios here:
Scenario 1: Around the time Apr 06 09:59:59 CMS harchive process just start for the interval 9:45 – 10:00, then the ACD switch time changed back for several seconds e.g. to 09:59:56, so the CMS time also changed(as CMS application time is synched to the Master ACD time) and this cause the harchive job to stop and restart, it will release the original data in the memory for this interval(9:45 – 09:59:59), then it will restarts harchive again to recollect the data for only (09:59:56 – 09:59:59), it is only 3 seconds data, that is why some entries will show 0 record in this CMS split/skill interval report.
Scenario 2: Around the time let’s say Apr 06 11:00:08 CMS harchive process just finished the data archive for the interval 10:45 -11:00, then the switch time changed back for several seconds e.g. to 10:59:56, so the CMS time also changed(as CMS application time is synched to the Master ACD time) and this caused the harchive job to restart the archive job for 10:45 -11:00 interval again, but since the original data in the memory for this interval has been released already.
So the harchive job will only archive the data for (10:59:56 - 10:59:59) three seconds’ data, that is why the system sometimes will generate the duplicated entry for this interval
and also the second duplicated entry will all show 0 records in this CMS report.
Need to check this Master ACD CM time, check the NTP service on this ACD from the CM side, make sure the NTP source and network are stable, CM time is not adjusting frequently as well.
01401 Sun Apr 06 02:23:32 2014 SRC_ERR_NUM=-00002 PROCESS=spi PID=00
1420 pbxclock.c:00348 SEVERITY=INFO ACD=01 DUPS=00004 SPI pbx err
or: Switch time seconds went backward
01401 Sun Apr 06 04:33:26 2014 SRC_ERR_NUM=-00002 PROCESS=spi PID=00
1420 pbxclock.c:00348 SEVERITY=INFO ACD=01 DUPS=00005 SPI pbx err
or: Switch time seconds went backward
01401 Sun Apr 06 09:59:59 2014 SRC_ERR_NUM=-00001 PROCESS=spi PID=00
1420 pbxclock.c:00255 SEVERITY=INFO ACD=01 DUPS=00007 SPI pbx err
or: Switch time changed