Solution Manager disp+work not starting
Hi Frnds,
I have a problem with Solman disp+work processes.
Solman is on windows 2003 platform and database is DB2 9.1
while starting from the sap mgmt console, the disp+work process starts for a while and then it stops working giving below error:
*** DP_FATAL_ERROR => DpWPCheck: no more work processes
*** DISPATCHER EMERGENCY SHUTDOWN ***
increase tracelevel of WPs
Pls refer to the attached file for details.
Would be greatful to any help.
---------------------------------------------------
trc file: "dev_disp", trc level: 1, release: "700"
---------------------------------------------------
sysno 00
sid SMN
systemid 562 (PC with Windows NT)
relno 7000
patchlevel 0
patchno 185
intno 20050900
make: multithreaded, Unicode, 64 bit, optimized
pid 3652
Fri May 15 16:12:11 2009
kernel runs with dp version 241000(ext=110000) (@(#) DPLIB-INT-VERSION-241000-UC)
length of sys_adm_ext is 576 bytes
*** SWITCH TRC-HIDE on ***
***LOG Q00=> DpSapEnvInit, DPStart (00 3652) [dpxxdisp.c 1285]
shared lib "dw_xml.dll" version 185 successfully loaded
shared lib "dw_xtc.dll" version 185 successfully loaded
shared lib "dw_stl.dll" version 185 successfully loaded
shared lib "dw_gui.dll" version 185 successfully loaded
shared lib "dw_mdm.dll" version 185 successfully loaded
rdisp/softcancel_sequence : -> 0,5,-1
use internal message server connection to port 3900
Fri May 15 16:12:23 2009
*** WARNING => DpNetCheck: NiHostToAddr(www.doesnotexist0046.qqq.nxst) took 12 seconds
Fri May 15 16:12:40 2009
*** WARNING => DpNetCheck: NiAddrToHost(1.0.0.0) took 17 seconds
***LOG GZZ=> 2 possible network problems detected - check tracefile and adjust the DNS settings [dpxxtool2.c 5418]
MtxInit: 30000 0 0
DpSysAdmExtInit: ABAP is active
DpSysAdmExtInit: VMC (JAVA VM in WP) is not active
DpIPCInit2: start server >solman_SMN_00 <
DpShMCreate: sizeof(wp_adm) 29792 (1752)
DpShMCreate: sizeof(tm_adm) 5912704 (29416)
DpShMCreate: sizeof(wp_ca_adm) 24064 (80)
DpShMCreate: sizeof(appc_ca_adm) 8000 (80)
DpCommTableSize: max/headSize/ftSize/tableSize=500/16/552064/552080
DpShMCreate: sizeof(comm_adm) 552080 (1088)
DpSlockTableSize: max/headSize/ftSize/fiSize/tableSize=0/0/0/0/0
DpShMCreate: sizeof(slock_adm) 0 (104)
DpFileTableSize: max/headSize/ftSize/tableSize=0/0/0/0
DpShMCreate: sizeof(file_adm) 0 (72)
DpShMCreate: sizeof(vmc_adm) 0 (1864)
DpShMCreate: sizeof(wall_adm) (41664/36752/64/192)
DpShMCreate: sizeof(gw_adm) 48
DpShMCreate: SHM_DP_ADM_KEY (addr: 000000000F0B0050, size: 6614016)
DpShMCreate: allocated sys_adm at 000000000F0B0050
DpShMCreate: allocated wp_adm at 000000000F0B21F0
DpShMCreate: allocated tm_adm_list at 000000000F0B9650
DpShMCreate: allocated tm_adm at 000000000F0B96B0
DpShMCreate: allocated wp_ca_adm at 000000000F65CF30
DpShMCreate: allocated appc_ca_adm at 000000000F662D30
DpShMCreate: allocated comm_adm at 000000000F664C70
DpShMCreate: system runs without slock table
DpShMCreate: system runs without file table
DpShMCreate: allocated vmc_adm_list at 000000000F6EB900
DpShMCreate: allocated gw_adm at 000000000F6EB980
DpShMCreate: system runs without vmc_adm
DpShMCreate: allocated ca_info at 000000000F6EB9B0
DpShMCreate: allocated wall_adm at 000000000F6EB9C0
MBUF state OFF
DpCommInitTable: init table for 500 entries
rdisp/queue_size_check_value : -> off
ThTaskStatus: rdisp/reset_online_during_debug 0
EmInit: MmSetImplementation( 2 ).
MM global diagnostic options set: 0
client 0 initializing ....
InitFreeList
block size is 4096 kByte.
Info: em/initial_size_MB( 6143MB) not multiple of em/blocksize_KB( 4096KB)
Info: em/initial_size_MB rounded up to 6144MB
Using implementation view
Using memory model view.
Memory Reset disabled as NT default
1535 blocks reserved for free list.
ES initialized.
mm.dump: set maximum dump mem to 96 MB
J2EE server info
start = TRUE
state = STARTED
pid = 4060
argv[0] = E:usrsapSMNDVEBMGS00exejcontrol.EXE
argv[1] = E:usrsapSMNDVEBMGS00exejcontrol.EXE
argv[2] = pf=E:usrsapSMNSYSprofileSMN_DVEBMGS00_solman
argv[3] = -DSAPSTART=1
argv[4] = -DCONNECT_PORT=65000
argv[5] = -DSAPSYSTEM=00
argv[6] = -DSAPSYSTEMNAME=SMN
argv[7] = -DSAPMYNAME=solman_SMN_00
argv[8] = -DSAPPROFILE=E:usrsapSMNSYSprofileSMN_DVEBMGS00_solman
argv[9] = -DFRFC_FALLBACK=ON
argv[10] = -DFRFC_FALLBACK_HOST=localhost
start_lazy = 0
start_control = SAP J2EE startup framework
DpJ2eeStart: j2ee state = STARTED
rdisp/http_min_wait_dia_wp : 1 -> 1
***LOG CPS=> DpLoopInit, ICU ( 3.0 3.0 4.0.1) [dpxxdisp.c 1692]
***LOG Q0K=> DpMsAttach, mscon ( solman) [dpxxdisp.c 12429]
DpStartStopMsg: send start message (myname is >solman_SMN_00 <)
DpStartStopMsg: start msg sent
Fri May 15 16:12:41 2009
CCMS: AlInitGlobals : alert/use_sema_lock = TRUE.
CCMS: Initalizing shared memory of size 60000000 for monitoring segment.
CCMS: Checking Downtime Configuration of Monitoring Segment.
CCMS: start to initalize 3.X shared alert area (first segment).
DpJ2eeLogin: j2ee state = CONNECTED
DpMsgAdmin: Set release to 7000, patchlevel 0
MBUF state PREPARED
MBUF component UP
DpMBufHwIdSet: set Hardware-ID
***LOG Q1C=> DpMBufHwIdSet [dpxxmbuf.c 1050]
DpMsgAdmin: Set patchno for this platform to 185
Release check o.K.
Fri May 15 16:12:46 2009
***LOG Q0I=> NiIRead: recv (10054: WSAECONNRESET: Connection reset by peer) [nixxi.cpp 4424]
*** ERROR => NiIRead: SiRecv failed for hdl 4 / sock 292
(SI_ECONN_BROKEN/10054; I4; ST; 127.0.0.1:1077) [nixxi.cpp 4424]
DpJ2eeMsgProcess: j2ee state = CONNECTED (NIECONN_BROKEN)
DpIJ2eeShutdown: send SIGINT to SAP J2EE startup framework (pid=4060)
*** ERROR => DpProcKill: kill failed [dpntdisp.c 371]
DpIJ2eeShutdown: j2ee state = SHUTDOWN
Fri May 15 16:13:20 2009
my types changed after wp death/restart 0xbf --> 0xbe
my types changed after wp death/restart 0xbe --> 0xbc
my types changed after wp death/restart 0xbc --> 0xb8
my types changed after wp death/restart 0xb8 --> 0xb0
my types changed after wp death/restart 0xb0 --> 0xa0
my types changed after wp death/restart 0xa0 --> 0x80
*** DP_FATAL_ERROR => DpWPCheck: no more work processes
*** DISPATCHER EMERGENCY SHUTDOWN ***
increase tracelevel of WPs
NiWait: sleep (10000ms) ...
NiISelect: timeout 10000ms
NiISelect: maximum fd=437
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Fri May 15 16:13:30 2009
NiISelect: TIMEOUT occured (10000ms)
dump system status
Workprocess Table (long) Fri May 15 10:43:30 2009
========================
No Ty. Pid Status Cause Start Err Sem CPU Time Program Cl User Action Table
-------------------------------------------------------------------------------------------------------------------------------
0 DIA 4068 Ended no 1 0 0
1 DIA 4076 Ended no 1 0 0
2 DIA 4084 Ended no 1 0 0
3 DIA 4092 Ended no 1 0 0
4 DIA 2144 Ended no 1 0 0
5 DIA 2068 Ended no 1 0 0
6 DIA 1564 Ended no 1 0 0
7 DIA 2168 Ended no 1 0 0
8 DIA 2304 Ended no 1 0 0
9 DIA 2292 Ended no 1 0 0
10 UPD 2308 Ended no 1 0 0
11 ENQ 2280 Ended no 1 0 0
12 BTC 684 Ended no 1 0 0
13 BTC 680 Ended no 1 0 0
14 BTC 1172 Ended no 1 0 0
15 SPO 2608 Ended no 1 0 0
16 UP2 2828 Ended no 1 0 0
Dispatcher Queue Statistics Fri May 15 10:43:30 2009
===========================
+------+--------+--------+--------+--------+--------+
| Typ | now | high | max | writes | reads |
+------+--------+--------+--------+--------+--------+
| NOWP | 0 | 2 | 2000 | 6 | 6 |
+------+--------+--------+--------+--------+--------+
| DIA | 4 | 4 | 2000 | 4 | 0 |
+------+--------+--------+--------+--------+--------+
| UPD | 0 | 0 | 2000 | 0 | 0 |
+------+--------+--------+--------+--------+--------+
| ENQ | 0 | 0 | 2000 | 0 | 0 |
+------+--------+--------+--------+--------+--------+
| BTC | 0 | 0 | 2000 | 0 | 0 |
+------+--------+--------+--------+--------+--------+
| SPO | 0 | 0 | 2000 | 0 | 0 |
+------+--------+--------+--------+--------+--------+
| UP2 | 0 | 0 | 2000 | 0 | 0 |
+------+--------+--------+--------+--------+--------+
max_rq_id 11
wake_evt_udp_now 0
wake events total 7, udp 6 ( 85%), shm 1 ( 14%)
since last update total 7, udp 6 ( 85%), shm 1 ( 14%)
Dump of tm_adm structure: Fri May 15 10:43:30 2009
=========================
Term uid man user term lastop mod wp ta a/i (modes)
Workprocess Comm. Area Blocks Fri May 15 10:43:30 2009
=============================
Slots: 300, Used: 1, Max: 0
+------+--------------+----------+-------------+
| id | owner | pid | eyecatcher |
+------+--------------+----------+-------------+
| 0 | DISPATCHER | -1 | *WPCAAD000* |
NiWait: sleep (5000ms) ...
NiISelect: timeout 5000ms
NiISelect: maximum fd=437
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Fri May 15 16:13:35 2009
NiISelect: TIMEOUT occured (5000ms)
DpHalt: shutdown server >solman_SMN_00 < (normal)
DpJ2eeDisableRestart
DpModState: buffer in state MBUF_PREPARED
NiBufSend starting
NiIWrite: hdl 3 sent data (wrt=110,pac=1,MESG_IO)
MsINiWrite: sent 110 bytes
MsIModState: change state to SHUTDOWN
DpModState: change server state from STARTING to SHUTDOWN
Switch off Shared memory profiling
ShmProtect( 57, 3 )
ShmProtect(SHM_PROFILE, SHM_PROT_RW
ShmProtect( 57, 1 )
ShmProtect(SHM_PROFILE, SHM_PROT_RD
DpWakeUpWps: wake up all wp's
Stop work processes
Stop gateway
killing process (4044) (SOFT_KILL)
Stop icman
killing process (4052) (SOFT_KILL)
Terminate gui connections
wait for end of work processes
wait for end of gateway
[DpProcDied] Process lives (PID:4044 HANDLE:404)
waiting for termination of gateway ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=437
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Fri May 15 16:13:36 2009
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process died (PID:4044 HANDLE:404)
wait for end of icman
[DpProcDied] Process lives (PID:4052 HANDLE:416)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=437
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Fri May 15 16:13:37 2009
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives (PID:4052 HANDLE:416)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=437
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Fri May 15 16:13:38 2009
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives (PID:4052 HANDLE:416)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=437
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Fri May 15 16:13:39 2009
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives (PID:4052 HANDLE:416)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=437
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Fri May 15 16:13:40 2009
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives (PID:4052 HANDLE:416)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=437
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Fri May 15 16:13:41 2009
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives (PID:4052 HANDLE:416)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=437
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Fri May 15 16:13:42 2009
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process died (PID:4052 HANDLE:416)
[DpProcDied] Process died (PID:4060 HANDLE:388)
DpStartStopMsg: send stop message (myname is >solman_SMN_00 <)
AdGetSelfIdentRecord: > <
AdCvtRecToExt: opcode 60 (AD_SELFIDENT), ser 0, ex 0, errno 0
AdCvtRecToExt: opcode 4 (AD_STARTSTOP), ser 0, ex 0, errno 0
DpConvertRequest: net size = 189 bytes
NiBufSend starting
NiIWrite: hdl 3 sent data (wrt=562,pac=1,MESG_IO)
MsINiWrite: sent 562 bytes
send msg (len 110+452) to name -, type 4, key -
DpStartStopMsg: stop msg sent
NiIRead: hdl 3 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 3
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 3 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 3
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 3 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 3
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 3 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 3
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 3 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 3
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 3 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 3
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 3 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 3
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 3 recv would block (errno=EAGAIN)
NiIRead: read for hdl 3 timed out (0ms)
DpHalt: no more messages from the message server
DpHalt: send keepalive to synchronize with the message server
NiBufSend starting
NiIWrite: hdl 3 sent data (wrt=114,pac=1,MESG_IO)
MsINiWrite: sent 114 bytes
send msg (len 110+4) to name MSG_SERVER, type 0, key -
MsSndName: MS_NOOP ok
Send 4 bytes to MSG_SERVER
NiIRead: hdl 3 recv would block (errno=EAGAIN)
NiIPeek: peek successful for hdl 3 (r)
NiIRead: hdl 3 received data (rcd=114,pac=1,MESG_IO)
NiBufIIn: NIBUF len=114
NiBufIIn: packet complete for hdl 3
NiBufReceive starting
MsINiRead: received 114 bytes
MSG received, len 110+4, flag 3, from MSG_SERVER , typ 0, key -
Received 4 bytes from MSG_SERVER
Received opcode MS_NOOP from msg_server, reply MSOP_OK
MsOpReceive: ok
MsSendKeepalive : keepalive sent to message server
NiIRead: hdl 3 recv would block (errno=EAGAIN)
Fri May 15 16:13:43 2009
NiIPeek: peek for hdl 3 timed out (r; 1000ms)
NiIRead: read for hdl 3 timed out (1000ms)
DpHalt: no more messages from the message server
DpHalt: sync with message server o.k.
detach from message server
***LOG Q0M=> DpMsDetach, ms_detach () [dpxxdisp.c 12775]
NiBufSend starting
NiIWrite: hdl 3 sent data (wrt=110,pac=1,MESG_IO)
MsINiWrite: sent 110 bytes
MsIDetach: send logout to msg_server
MsIDetach: call exit function
DpMsShutdownHook called
NiBufISelUpdate: new MODE -- (r-) for hdl 3 in set0
SiSelNSet: set events of sock 316 to: ---
NiBufISelRemove: remove hdl 3 from set0
SiSelNRemove: removed sock 316 (pos=3)
SiSelNRemove: removed sock 316
NiSelIRemove: removed hdl 3
MBUF state OFF
AdGetSelfIdentRecord: > <
AdCvtRecToExt: opcode 60 (AD_SELFIDENT), ser 0, ex 0, errno 0
AdCvtRecToExt: opcode 40 (AD_MSBUF), ser 0, ex 0, errno 0
AdCvtRecToExt: opcode 40 (AD_MSBUF), ser 0, ex 0, errno 0
blks_in_queue/wp_ca_blk_no/wp_max_no = 1/300/17
LOCK WP ca_blk 1
make DISP owner of wp_ca_blk 1
DpRqPutIntoQueue: put request into queue (reqtype 1, prio LOW, rq_id 14)
MBUF component DOWN
NiICloseHandle: shutdown and close hdl 3 / sock 316
NiBufIClose: clear extension for hdl 3
MsIDetach: detach MS-system
cleanup EM
EsCleanup ....
EmCleanup() -> 0
Es2Cleanup: Cleanup ES2
***LOG Q05=> DpHalt, DPStop ( 3652) [dpxxdisp.c 10973]
Good Bye .....