[TRAFODION-1805] monitor would give a wrong info for sqps when kill monitor and up node for some times

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: foundation
• Environment: SUSE VM. daily build: 20160122_0900
• Assignee: Unassigned
• Reporter: Joshua Liu
• Created: 03/Feb/16 05:15
• Updated: 03/Feb/16 05:15

Description

Kill monitor in one node and use sqshell -a, up node to up it for some times
After up the node(here it’s suseha-4,nid 2), sqcheck would show
trafodion@suseha-2:/shares/Joshua> sqcheck
Checking if processes are up.
Checking attempt: 2; user specified max: 2. Execution time in seconds: 4.

The SQ environment is not up at all, or partially up and not operational. Check the logs.

Process Configured Actual Down
------- ---------- ------ ----
DTM 5 5
RMS 10 8 2
DcsMaster 1 1
DcsServer 5 5
mxosrvr 40 32 8
from ps -ef, we can see the mxsscp processes were up:
trafodion@suseha-2:~> pdsh $MY_NODES ps -ef | grep mxsscp
suseha-4: 501 41940 41664 0 08:01 ? 00:00:08 mxsscp SQMON1.1 00002 00002 041940 $ZSC002 192.168.0.46:33430 00004 00002 00006 SPARE
suseha-2: 501 13919 13776 0 14:24 pts/1 00:00:00 grep mxsscp
suseha-2: 501 31436 31132 0 07:54 ? 00:00:09 mxsscp SQMON1.1 00000 00000 031436 $ZSC000 192.168.0.42:41942 00004 00000 00006 SPARE
suseha-5: 501 52635 51607 0 07:49 ? 00:00:08 mxsscp SQMON1.1 00003 00003 052635 $ZSC003 192.168.0.45:33233 00004 00003 00005 SPARE
suseha-6: 501 19601 18637 0 07:49 ? 00:00:10 mxsscp SQMON1.1 00004 00004 019601 $ZSC004 192.168.0.10:48671 00004 00004 00005 SPARE
suseha-3: 501 49054 48798 0 07:58 ? 00:00:11 mxsscp SQMON1.1 00001 00001 049054 $ZSC001 192.168.0.44:51353 00004 00001 00006 SPARE
but we could see different sqps info in node suseha-2 and suseha-4
trafodion@suseha-2:/shares/Joshua> sqps | grep mxsscp
[$Z001I4P] 000,00031436 001 GEN ES-A- $ZSC000 $PSD000 mxsscp
[$Z001I4P] 001,00049054 001 GEN ES-A- $ZSC001 $PSD001 mxsscp
[$Z001I4P] 003,00052635 001 GEN ES-A- $ZSC003 NONE mxsscp
[$Z001I4P] 004,00019601 001 GEN ES-A- $ZSC004 NONE mxsscp
trafodion@suseha-2:/shares/Joshua> ssh suseha-4
Last login: Tue Feb 2 17:03:47 2016 from suseha-2.novalocal
trafodion@suseha-4:~> sqps | grep mxsscp
[$Z0217Y6] 002,00041940 001 GEN ES-A- $ZSC002 $PSD002 mxsscp
[$Z0217Y6] 003,00052635 001 GEN ES-A- $ZSC003 NONE mxsscp
[$Z0217Y6] 004,00019601 001 GEN ES-A- $ZSC004 NONE mxsscp