Memory pipes (MPI)

The ICM require two memory pipes (MPI) for each HTTP connection.

The MPI configuration in the ICM is limited by parameter

mpi/max_pipes. The default value for this paramter is 4000.

If you has 4000 for that parameter the operation system should provide 8000 operating system semaphores because eatch MPI require two semaphores.

To increase the parameter mpi/max_pipes you should take care the operation system configuration also. Follow the note 715400 to set the operation system as you need.

715400 Max. number of memory pipes (MPI) restricted to 4,000

ERROR MESSAGE:

ERROR => no more free event-flags. [mpixx_mt.c 4555]

ERROR => MpiEvtOpen(0) failed: 11 [mpix_mt.c 1169]

ERROR => IcmConnInitBuffer: MpiCreate2 failed (rc=11)

means that no more semaphores can be created because the ICM reach the

Operation System limits.

HTTPWatch traces

1. Install the free version of HTTPWatch 6.x (http://www.httpwatch.com)

on your Internet Explorer.

2. Start the browser, start the HTTPWatch, click on 'Record' button,

and please reproduce the issue.

3. On finish 'Save' the contents of HTTPWatch as a .hwl file

(Please note: Exporting as CSV or XML is not so helpful, so please use

the default save option).

4. Save .hwl file to this message.

Logon-Check Availability

Function module responsible: SCSM_RUN_RFC_LOGON_CHECK

All ABAP instance with the 'Logon-Check' is set in the CCMSPING availability configuration

will be monitored by the agent.

The SCSM_RUN_RFC_LOGON_CHECK will do this by creating an ARFC session

for each instance and it will wait for an answers of each of these ARFC.

If you have the note 929577, the sessions time out will be by default 2 seconds. If the note is not implementdthe timeout is determined by RFCPING_TIMEOUT.

Trace file's rotation and system restart

You have lost the trace file to perform an anlysis on the system behavior after an error/crash/hung situation.

Every system restart, the trace files are overwritten.
When you start the instance, all files from the services (message
server, dispatcher, work process, ICM and others) are truncated and only
the actual log status information are writen.

But, before truncate the actual trace files, the system create a backup
of this trace file with the .old extension.

Although exist the mechanism to create a backup of the trace file, if
you restart the system more than one time, the .old trace file will be
overwritten too.

A good pratice is make a copy of the work directory before the start up.

Parameter changes without restart

If you need to change some memory parameters but you cannot restart the instance, have solution.
You can change the parameters from RZ11 and run the remport RSMEMORY.

610965 Parameter in RZ11 cannot be switched

java start up

To start up the java stack in a dualstack system or only java.

jcontrol -DSAPSTART=1 pf=/usr/sap/xxx/SYS/profile/xxx_DVEBMGS00_dostname

This is the same than as sapcontrol.

Error in monitoring information

If you are not able to see the statistics information from ST03/ST03N, this can be a error in the timezone configuration.
Please note that the time can been seen at diffenret levels:

1. Operating System Time (or) System time
2. Time zone at Client level whcih is client dependent
3. User time zone whcih can be set using tocde SU3

Make sure if the timezone between the clients and operation system are the same.

926290 SWNCCOLL: SAP workload NW collector collects no data
741734 Incorrect times due to the time zone settings
481835 Analyzing the time zone settings
198411 Current data and information about time zones

monitoring updates SM13

To monitoring the failed updates in SM13 you should go to
RZ20 ->
SAP CCMS Technical Expert Monitors ->
All Contexts on Local Application Server ->
Update
UpdateServices
Old Update Record

Select this MTE and click in Proprerties.
In the performance Attribute have the thresholds that will be compared
with the last 24 hours, the the number of the update is around the
threshoulds, the new alert will be created.

Operating system

AIX
1)
instfix -ik

2) to get the memory consuption for a specific process.
You should get the INUSE output and multiply for 4096 to get the value in bytes:
svmon -P (to analyse a specific process)

svmon -G (to analyse the total memory)
svmon -Pnsm (to analyse the top process)
ps auxw  | sort -r +3  (process sort by memory consuption)

svmon -Pt15 | perl -e 'while(<>){print if($.==2||$&&&!$s++);$.=0 if(/^-+$/)}'

SOLARIS

1) to analyse the memory usage for each process:

/usr/bin/ps -eo 'zone user pid ppid vsz rss args'

WINDOWS
1) to find a specific string in a OUTPUT:
netstat -an | findstr LISTEN

2) to list process from prompt line

tasklist

3) services
sc qc


HP-UX

memory statistics:
ps -el | sort -rnk10 
will list all processes on the system, sorted by the SZ column.

NIPING tests

#Name resolution
niping -v -H

# With option -L 6048000 the test will run for 7 days.

niping -c -H -B 10000 -D 100 -L 6048000

#niping as server

niping -s -I 0 -T NIPING_SERVER_TRACE

#niping as client

niping -c -H -X3 -B 10 -D 30 -T NIPING_CLIENT_TRACE

500235

Looking for a RFC

You can find all RFC in SM59, but no always is easy to find a specific RFC if you do not have the name.

The table RFCDES have all RFC connection with hostname or IP Address.
You can find in this table with a easy way.

Windows services for start SAP

User should be SAP*SID*_*NR*

To create a new service:

1) Log into the NT machine as adm.
2) Run 'sapstartsrv.exe'.
3) Select "Uninstall Service + Unregister COM Interface"
4) Specify your SID and System Number (NR) and click ok.
5) Select "Install Service + Register COM Interface + Start Service
6) Specify your SID and System Number (NR).
7) Specify your startprofile (use the 'browse' button).
8) Specify your user as (domain\adm or workgroup\SIDadm).
9) Specify the passsword for adm.
10) Click OK.

SNMPTRAP for SAP system

You should use the package SNMPLIB delivery by SAP.

Download
Market Place
Download ->
Support Packages and Patches
Entry by Application Group ->
Additional Components ->
SAP Kernel ->
SAP KERNEL 32/64bit ->
-> Database Independent

To test the package:
You should check the dependencies with LDD.
Call trapsend from OS side.

1. Run command: rectraps rectraps.cfg

2. In another session run command: trapsend -v 2c -m MIB:SAP_R3_MIB -M
DIR: -c public -t 5 -r 5
UDP:localhost:162 '' SAP-MIB::r3maTrap SAP-MIB::r3maAlertMsgString s
'test'

Notes:
585110 RZ20: Sending alerts as SNMP traps

Make last versions

STARTSAP and STOPSAP scripts
#809477 startsap and stopsap <= SAP WebAs 640, 700/701 and 710/711

CPS (REDWOOD)
#1257610 How to download SAP CPS for NetWeaver 7.0 and 7.01

monitoring from OS

If you wold like to monitoring the SAP system from command line or automatize a script to do this or colect information from OS side to update a external tool or report you can use the DPMON or sapcontrol.

DPMON is an interactive tool provide from SAP.

sapcontrol is a tool that you can colect information from instance and from remote instances.


More info:
675778 Analysis after system downtime
42074 Using the R/3 dispatcher monitor 'dpmon'

ST02

- the general information is collected by RSTUNE50 (AFAIK)
- the historical information is collected by RSTUNE80 (AFAIR)

SAPTUNE_GET_SUMMARY_STATISTIC
SAPTUNE_ROLL_PAGE_SIZES

lgtst and msclients

lgsts is a tool provide from SAP to test the message server services as logoun groups:
The simple syntax is:

lgtst -H -S sapms

This will provide all groups configured in the message server and application servers avaliable.
You can increase the test with the options below:
-G name of a logon group;
-C loop counter;
-Y delay of n milliseconds between each call;

lgtst -H -S -G GROUP

This will test a specific logon group called GROUP.

More info:
#64015 Description of test program lgtst

More options:
lgtst pf= name= -H -S sapms


For java instance you should use the msclients

msclients -mshost hostname -msserv port -d J2E

ICM TIME OUT

to set the ICM time out you should increase the parameters TIMEOUT and
PROCTIMEOUT in the icm/server_port_ parameter.

Take a look in note:
#824554 - ICM and SAP Web Dispatcher Timeout Parameter

This online documentation can help:
http://help.sap.
com/saphelp_nw70ehp1/helpdata/en/48/88b52977323cb8e10000000a42189d/frame
set.htm


TAGS
IcmConnTimeOut

The ping test (OS01 - Specific IP address) use the program RSHOST14 to perform the test

triks

Transactions, Reports and Function Module:

- To ping a remote IP addres = OS01. Report RSHOST14

- Perform commands in the OS level = report RSBDCOS0

- See the work process trace file = SM50 and w

- Check the last trace file changed = ST11

- Change the application server = SM51 and twice in the appl

- Send message for an user logged = Function Module TH_POPUP

R3:

- Ctrl & + = open a new session

- Ctrl Shift F8 = go to command bar

-

SAPMSSY8

AutoAbap dispatcher program SAPMSSY8 should be started every 300 seconds.
Parameters: rdisp/autoabaptime
SAP not recomend to change this value.

availability monitoring

Local autoreaction depends on the method dispatcher (sapmssy8), therefore you have a delay up to 300 seconds.

The only workaround for this (based on ccmsping availability) would be to use two systems:

the first one is responsible for the availabilty monitoring, the second one would monitor the first system for the Availability MTEs. In this case it would be possible to use a central autoreaction method for the Availability MTEs: the alert occurs in the first system, will be forwarded to the second system by sapccm4x which triggers the central autoreaction in the second system.

semaphore

semaphore 43 in SPOOL workprocess

The spool uses semaphore 43 on every application server to synchronize
accesses on shared use data structures in the shared memory. If a work
process is blocked during a spool action for which it set SEM43, other
processes gradually encounter these semaphores and wait.

#661487 Spool: Analyzing problems with semaphore 43

Web Dispatcher / ICM

NOTES
552286 Troubleshooting for the SAP Web Dispatcher
634262 Note on the preclarification of SAP Web dispatcher messages


DONWLOAD
http://service.sap.com/swdc
-> Download
-> Support Packages and Patches
-> Entry by Application Group
-> Additional Components
-> SAP Kernel
-> SAP KERNEL **-BIT
-> SAP KERNEL **-BIT
-> SAP KERNEL **-BIT
->
-> #Database independent
-> sapwebdisp.SAR

SERVICE WINDOWS

Try create a service with SC tool:
sc create SAPWebDisp binpath= " service pf=" type= own start= auto

ADMIN MANAGEMENT

You can create a separated port to connect in the web dispatcher admin
You should create a net port to webdispatcher listen:
icm/server_port_1 = PROT=HTTP,PORT=8999

and redirect the admin to this port:
icm/HTTP/admin_0 = PREFIX=/sap/admin,DOCROOT=./admin,PORT=8999

ICMAUTH

How to create/delete/edit users in the icmauth authentication for webdispatcher?
wdispmon -a


KNOW ERRORS/ISSUES

**************************************
*** ERROR => PlugInHandleNetData: server: premature EOS (0/620) in request [http_plg_mt. 1801]
**************************************
-----------------------------------------------------------------------------
The PlugInHandleNetData is a function wich analyses the input stream
from the network. This error you get means that some error occured
during receiving the data stream from the server above.
There is generally nothing to worry about, this can be caused by a
temporary network error or some synchronization fault.
-----------------------------------------------------------------------------
This message "PlugInHandleNetData" appears in the icm trace when you are
executing stateful applications and the user session (in SM04) is
already deleted. This session is deleted after the timeout in
rdisp/plugin_auto_logout. After this time the user session in SM04 will
be deleted. This parameter is only used for stateful sessions.
-----------------------------------------------------------------------------


**************************************
*** ERROR => htmlEncode: called with empty string
**************************************
"This is a bug of web dispatcher and was solved on patch level 11
So, please apply the kernel patch level 115 and check whether it
helps you."

851852 - ICM Patch Collection (7.00)

**************************************
*** ERROR => PlugInHandleServData: header not terminated
**************************************

The response from the server does not contain a terminated HTTP header.

sapsrtarsrv

The sapstartsrv is a tool to organize the satart an stop process.

936273 sapstartsrv for all platforms
995116 Backward porting of sapstartsrv for earlier release

This is a service for the administration of a SAP instance. Ideally, the service start and stop the SAP in the correct sequence.

897933 Start and stop sequence for SAP system

To increase the sapstartsrv trace set the parameter:

service/trace = 1

AIX memory management

AIX memory management
ES/TABLE = SHM_SEGS
ES/SHM_SEGS_VERSION = 2

1088458 AIX: Performance improvement for ES/SHM_SEGS_VERSION = 2
191801 AIX 64-bit with a lot of Extended Memory
445533 Lots of extended memory on AIX (64-bit)
789477 Large extended memory on AIX (64-bit) as of Kernel 6.20

AIX signal 33

OS limits

sappfpar check pf="INSTANCE PROFILE"

323816 User limits on AIX

581319 Signal 33 on AIX

1563816 Signal 33 on AIX

8755 Signal 33 received by operating system

1121904 SAP on AIX: Recommendations for paging space

trace files

Work process:
#84850 Increase trace level of work processes
#112 Trace and error information in the "dev_" files

Web Dispatcher
#634262 Note on the preclarification of SAP Web dispatcher messages
in point 8
or set parameter 3 in rdisp/TRACE.

ICM

You can increase the trace level by adding the following parameter to your instance profile:

rdisp/TRACE = 2

and restart the system after the change is made.

Similarly, you can increase the trace level through the SMICM transaction.

Open SMICM, then "goto->trace level->set" and enter 2.

network notes

63342 List: NI error codes

26086 TCP/IP Connection to partner broken
430896 Error messages: Q0U MsSClientHandle, inv_name
413330 Network timeouts

DNS
124562 Hostname resolution problems (DNS timeouts)
129997 Hostname and IP address lookup
611361 Hostnames of SAP servers

NIPING
500235 Network Diagnosis with NIPING
799428 Downloading SAProuter and/or niping

WINDOWS
155147 WinNT: Connection reset by peer

gateway notes

384971 - System parameters for a high interface load

Email autoreaction in system monitoring (RZ20)

Check the note #176492 and ensure than all steps are right in the configuration.

After that, you can trigger an auto-reaction method manualy.
Goto the monitor (in RZ20), Extras - >Active mantainance function
Select the item and go to Edit -> Nodes -> Start Methods -> Start AutoReaction

And check if the emails are display in SCOT or SOST transaction.


FOLOW STEPS

1. Central auto-reaction method is assigned to an MTE-class in the
monitor segment of the satellite system for which and agent is
installed via the CEN system, transaction RZ21. After this check
via the transaction RZ20 on the satellite system if the method
assignment is correctly done (select from the menu "Views>Status
auto-reaction").
2. create a new alert for one of the monitoring nodes on the satellite
system for which a Central Auto-reaction is assigned (Alert status
will be ACTION_REQUIRED
3. the agent will send the alert information to the CEN system and
store this information in the table CSMCENTOOL and the agent will
trigger the event SAP_CSM_TRIGGER_CENSYS_DISPATCH
4. the SAP_CSM_TRIGGER_CENSYS_DISPATCH event will start the background
job SAP_CCMS_CENSYS_DISPATCHER
5. the background job SAP_CCMS_CENSYS_DISPATCHER will execute the
Central Auto reaction method on the CEN system and when this is
done successfully it will change the status of the alert to ACTIVE
and it will delete the entry in the table CSMCENTOOL.
6.) Check the Open Alerts in the transaction RZ20 on the CEN system:
RZ20>SAP CCMS Technical Expert Monitors>CCMS Selfmonitoring>Tooldisp
atching (central system)>Messages << select this node with the
cursor and click on 'Display
Details'



----------------------------------------
local autoreaction depends on the method dispatcher (sapmssy8), therefore you have a delay up to 300 seconds.

central autoreaction starts an *event triggered* batch job which is started immediately.

Cronacle trace files

-- RFC Agents .lis and .log files --

In Redood Explorer, you look under Monitor - Jobs - SAP agents
You need to click on the RFC agent job. Then at the bottom of the screen, you see Output and Log with path to the lis and log file for this agent job.
You can open it directly from there or retrieve it from the OS level via the path given in the RWE display.


-- Schedulers TraceFiles are located in --

Redwood explorer -> Configuration > environment > schedulers > >
parameters > scheduler_dump_dest)


-- Job log file --

In the job .lis and .log

Memory management

Extend Memory - Heap Memory - Roll Area


Parameters
EM
em/initial_size_MB
ztta/roll_extension
ztta/roll_extension_dia
ztta/roll_extension_nondia

HM (swap - PRIV mode)
abap/heap_area_dia
abap/heap_area_nondia
abap/heap_area_total

RA
ztta/roll_first
ztta/roll_area