slow startup systems in the managementconsole 1

[Guest_imported]

Datastage server 7.5ver is running on Windows 2003 server.
System config:
OS: 2003 sever(service pack 1)
CPU: 2 (3.6GHZ each)
Ram: 2gb
Physical Memory : 146GB(freespace 136GB)

with the above configuration I was trying to connect to the box remotely from my desktop machine.
I setup in the eventserver admin such a way that I selected both options like "automatic startup" and "seperate system process". Everything looks good.

there are more watches exist.

Issue 1:
I was able to start server in the services and it is starting very quickly.But when I am opeing management console connection is file but system up time is very high. I tried with single MSL but it is taking the same time.

 

[BocaBurger]

Is this 7.5.0 or 7.5.1?

English not clear:
when I am opeing management console connection is file but system up time is very high. I tried with single MSL but it is taking the same time.

The system up time is how long the system has been running. If it is high, this is a good thing.

Check your history tab. If there are a large nuber of errors or warnings, it can slow down the loading of the Management Console and eventually prevent it from opening.

If you use a database adapter, you need to get the DB PTF from IBM.

How many watches do you have in your systems? What is your maxthreads setting in the .ini?



BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[eyetry]

DB PTF ?

 

[Guest_imported]

Sorry for the confusion Boca.
Here the ans for your questions
Version 7.5(119)
MaxThreads=20
Watches =120 (it could be more)
All maps not using database (its just a spreadsheet "Microsoft Excel")
I am using "FTP adapters" ,"R3/ALE","FILE".

Suggest me changes need to be done at .ini file level.

One more Issue:
In the IFD there are 4 different mapcomponent all components have its own watches but reffering to the same map source(mmc).
(SOURCE TRIGGERED)Map1(Target is to SAP using cardlevel adapter)
(SOURCE TRIGGERED)Map2(Target is to SAP using cardlevel adapter)
(FILE NOT TRIGGERING FOR 3RD MAP ) Map3

The Map3 component is not triggering the source file.

This is happening for all the systems.

===
Attached the .ini file


[Execution]

ResourceCfgFile=resource.mrc
rectLeft=220
rectTop=220
rectRight=524
rectBottom=476

[Lci]

; LCI display mode -
LciMode=1 ; 0 means use Standard IO, 1 means use Curses


[M4File]
; to enable cluster support journaling for file triggers,
; uncomment and enter the shared drive and directory for the
; active node where journal files should be created:
;Journaling=

; used to enable logging for the file listener -- it should
; be a valid complete path (directory and filename)
;Logging=

; number of files to process at a time per directory (0 meaning all)
; 25 is the default - decrease this value as more directory watches
; are added for trigger uniformity
;FileListenCount=25

; time in milliseconds between directory processing - 50 is the default
; increase this value if the CPU level is high when Event Server is idle
;ListenSleepTime=50

; time in milliseconds between repetitive directory scans - 50 is the default
; operates at a lower level than ListenSleepTime
; increase the vaule if the CPU level is high when Event Server is idle
; This is a Unix Only configurable parameter
;DirScanWaitTime=50

[Launcher]

; Debug Controls
; command line values are in parenthesis...
; 0 means off, 1 means on
; (debug lines are equivalent to -dxxxx)
;
Debug1=0 ; map start and map end
Debug2=0 ; not used in this version
Debug3=0 ; not used in this version
Debug4=0 ; general messages
Debug5=0 ; initialization/failure messages
Debug6=0 ; general messages

ResourceCfgFile=resource.mrc

;NT only, use to disable cpu and memory counters
;LoadPDH=0

; Miscellaneous Controls
;
; maximum concurrent maps (-mx). This value needs to be set carefully: if too low
; then CPU may not be used to full potential. If too high, then excessive memory
; use or excessive context switching can reduce overall performance. 20 is a
; reasonable value for an average server. The higher the number of CPUs on the
; server, the greater the number of maps that can be executed concurrently.
MaxThreads=20

PauseTimeout=5000 ; not used in this version

; file time granularity in seconds
TriggerTime=1

; launcher log types - e = error, w = warning, s = startup c = cfg summary
;LauncherLog=ewsc

;use the following to pause the listener threads if too many init pending occur
InitPendingHigh=0

;use the following to resume the listener threads if paused from InitPendingHigh
InitPendingLow=0

;use the following to set the max threads per watch.
WatchMaxThreads=0

;use the following with WatchMaxThreads to disable the MaxThreads option above.
DisableMaxThreads=0

;use the following for the time (in milliseconds) init pending threads should
;remain idle between mapping attempts
InitPendingIdleMS=1

;use the following to define the max number of RUN maps to be cached in the memory
;default value is 300, minimum value is 10
;set this value to -1 to disable RUN map caching
;RunMapCacheMaxNum=300

[Resource Manager]

TableSize=10 ; size in megabytes for the resource manager tables. Default is 10
CentralMgmt=0 ; centralized (global) resource management vs. local. Default is 0 (local)

;Debug0=1 ; Debug includes simple error codes
Debug1=0 ; Debug includes RM table states
Debug2=0 ; Debug connection sharing
Debug3=0 ; Debug includes some low-level codes
Debug4=0 ; Debug includes RM dispatch codes
Debug5=0 ; Debug to memory, then write at shutdown time
Debug5Cnt=50000 ; Number of lines to keep in memory
Debug6=0 ; Debug active (basic stuff)
Debug7=0 ; Debug sharing manager
Debug8=0 ; Debug queue manager
;DebugName= ; Define the complete filename of the debug file
;DebugAppend=1 ; Append the debug log (default is to overwrite)
;Debug9=1 ; Debug final system state at shutdown time

;Disable=0 ; Disable resource management. Be careful with this option !

;MaxResourcePending=80 ; Maximum Resource Pendings to allow

;StackSize = 400 ; Stack size in Kbytes per thread (this setting is HP-UX 11.XX specific).

; Transaction manager to use for global transactions. Possible values are:
;
; MQS,[queue manager],[trace file]
; - MQSeries transaction manager.
; [queue manager] is the name of the queue manager to attach to.
; If omitted the default queue manager is assumed.
;
; MSDTC,[machine name],[trace file]
; - Microsoft Distributed Transaction Controller
; <machine name> is the name of the machine on which the master
; MSDTC service is running that controls resources
; and slave MSDTCs. If omitted the local machine is assumed.
;TransManager=MQS

;When data passed to/from an adapter exceeds this size (in Mbytes), data will be
;paged to a temporary file to limit the memory consumption of the process

;StreamMaxMemLimit=500

;When data passed to/from an adapter exceeds the size specified by
;StreamMaxMemLimit, a temporary file will be created in the following directory
;for paged data. Ensure this path points to a directory with sufficient filespace.

;StreamMaxMemDirectory=/tmp

[Connections Manager]

;IgnoreGPFs=0 ; Ignore system exceptions (such as GPFs) on connection threads
; if ignored, a map failure occurs instead (which may not be any better)

;ShareConnectionThreads=0 ; 0=no sharing, 1=sharing for GTX only, 2=sharing for same adapter type, 3=share across for all adapters

;Idle=60 ; Global default for IdleXXX parameters.
;SLim=4 ; Global default for SLimXXX parameters.
;HLim=0 ; Global default for HLimXXX parameters.
;Min=0 ; Global default for MinXXX parameters.
;Keep=0 ; Global default for TimeXXX parameters.

;HSleep=2 ; Cycle time for checking HLim (in seconds)
;HLimTimeout=0 ; Global Maximum time to wait in HLim state (in seconds)

;PollWaitTimeMin=2000 ; Minimal polling interval in milliseconds
;PollWaitTimeMax=10000 ; Maximal polling interval in milliseconds

; This list of entries is current as of Nov 2002. Additional entries
; can be manually added as required; consult your DataStage TX Command
; Server Guide and follow the format used here.

; IdleXXX is the number of seconds to allow a connection to be idle
; before automatically disconnecting. 0 to disable this feature.

; SLimXXX is a soft (advisory) limit. This number can be exceeded at
; peak load, but this tries to limit the number of idle connections.
; Limits are only checked when opening a connection, so it is possible
; to have > number of idle connections. XXX is adapter type, using the
; same notation used on a command line. Ex: MQS is MQSeries, etc.

; HLimXXX is a hard (mandatory) limit. This number will not be exceeded.
; Connections will be placed in a pending state rather than exceed this
; number.

; MinXXX is a minimum number of connections to keep. When a limit (SLim
; or HLim) is exceeded, idle connections will be removed until no more
; than Min remain (if possible).

; KeepXXX is how long (in seconds) an idle connection is expected to
; remain valid. Connections idle longer than this will be tested prior
; to being used (if the adapter supports it). If the test fails or is
; not supported, the connection will not be reused. A value of 0 disables
; this feature. A value of -1 will force a test every time, but this is
; not recommended.

;IdleDB=0
;SLimDB=0
;HLimDB=0
;KeepDB=0
;MinDB=0

;IdleMQS=0
;SLimMQS=0
;HLimMQS=0
;KeepMQS=0
;MinMQS=0

;IdleMQSC=0
;SLimMQSC=0
;HLimMQSC=0
;KeepMQSC=0
;MinMQSC=0

;IdleMSMQ=0
;SLimMSMQ=0
;HLimMSMQ=0
;KeepMSMQ=0
;MinMSMQ=0

;IdleRV=0
;SLimRV=0
;HLimRV=0
;KeepRV=0
;MinRV=0

;IdleRVTX=0
;SLimRVTX=0
;HLimRVTX=0
;KeepRVTX=0
;MinRVTX=0

;IdleAQ=0
;SLimAQ=0
;HLimAQ=0
;KeepAQ=0
;MinAQ=0

;IdleTUX=0
;SLimTUX=0
;HLimTUX=0
;KeepTUX=0
;MinTUX=0

;IdleTUXMT=0
;SLimTUXMT=0
;HLimTUXMT=0
;KeepTUXMT=0
;MinTUXMT=0

;IdleROMA=0
;SLimROMA=0
;HLimROMA=0
;KeepROMA=0
;MinROMA=0

;IdleFSMGR=0
;SLimFSMGR=0
;HLimFSMGR=0
;KeepFSMGR=0
;MinFSMGR=0

;IdleALNC=0
;SLimALNC=0
;HLimALNC=0
;KeepALNC=0
;MinALNC=0

IdleALE=30
;SLimALE=0
;HLimALE=0
;KeepALE=0
;MinALE=0

;IdleBDC=0
;SLimBDC=0
;HLimBDC=0
;KeepBDC=0
;MinBDC=0

;IdleBAPI=0
;SLimBAPI=0
;HLimBAPI=0
;KeepBAPI=0
;MinBAPI=0

;IdleBWSB=0
;SLimBWSB=0
;HLimBWSB=0
;KeepBWSB=0
;MinBWSB=0

IdleEMAIL=30
;SLimEMAIL=0
;HLimEMAIL=0
;MinEMAIL=0

IdleFTP=30
;SLimFTP=0
;HLimFTP=0
;MinFTP=0

;IdleHTTP=0
;SLimHTTP=0
;HLimHTTP=0
;KeepHTTP=0
;MinHTTP=0

;IdleJMS=0
;SLimJMS=0
;HLimJMS=0
;KeepJMS=0
;MinJMS=0

;IdleSOCKET=0
;SLimSOCKET=0
;HLimSOCKET=0
;KeepSOCKET=0
;MinSOCKET=0

;IdleJCA=0
;SLimJCA=0
;HLimJCA=0
;KeepJCA=0
;MinJCA=0

;IdleCORBA=0
;SLimCORBA=0
;HLimCORBA=0
;KeepCORBA=0
;MinCORBA=0

;IdleCICS=0
;SLimCICS=0
;HLimCICS=0
;MinCICS=0

;IdleJNDI=0
;SLimJNDI=0
;HLimJNDI=0
;MinJNDI=0

;IdleSNMP=0
;SLimSNMP=0
;HLimSNMP=0
;MinSNMP=0

;IdlePSCI=0
;SLimPSCI=0
;HLimPSCI=0
;KeepPSCI=0
;MinPSCI=0

;IdleAXION=0
;SLimAXION=0
;HLimAXION=0
;MinAXION=0

;===============================================================================
; The following three sections are used by the Java Native Interface Layer
;-------------------------------------------------------------------------------
; 1. The "JNI Layer Trace" section contains the following entries:
; a) The "switch" entry turns the JNI layer tracing on and off
; b) The "file" entry specifies the full path for the JNI layer trace file
; 2. The "External Jar Files" section lists the JAR files to be included in
; the CLASSPATH variable which will be passed to the Java Virtual Machine.
; Each entry in this section specifies the full path of a single JAR file.
; The entries are named "jar1", "jar2", ..., "jarN" (N <= 128)
; 3. The "JVM Options" section specifies additional options which will be passed
; to the Java Virtual Machine
; Each entry in this section specifies a single JVM option.
; The entries are named "option1", "option2", ..., "optionN" (N <= 128)
;===============================================================================
;[JNI Layer Trace]
;switch=off
;file=

;[External Jar Files]
;jar1=
;jar2=
;...

;[JVM Options]
;option1=
;option2=
;...
;===============================================================================


;===============================================================================
;Using this option, the SNMP agent will set the values of the dstxSystemName
;and dstxMslFileName MIB variables to "myTestSystem". The purpose of this
;configuration option is to allow users to replace default string values of these
;variables that may be very long and difficult to remember with the custom
;short string value.
;-------------------------------------------------------------------------------
;[SNMP]
;SystemName=myTestSystem
;===============================================================================

 

[BocaBurger]

DB PTF = Patch for database adapters. ODBC included.

Some suggestions:
Maxthreads = 12,
watchmaxthreads = 5
FileListenCount=20
ListenSleepTime=150
InitPendingIdleMS=175
StreamMaxMemLimit=15 ; remember to set path to directory.
Logging= ; Give the path to the logs directory
LauncherLog=esc ; This could show you the problem in combination with the previous entry

PollWaitTimeMin=20

IdleDB=60
;SLimDB=0
HLimDB=19

IdleALE=60
;SLimALE=0
;HLimALE=0 ; Set this based on info from your BASIS team.
KeepALE=30





BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[Guest_imported]

Thanks Boca for the suggestions you gave, I just put in the ".ini" file and it works such a way that the system uptime reduced from 10min to 3min. This is really a good sign.

Issue:
I have a time event settup some maps like for every 30min it will look into the folder for files . Because files were not there in the directory the managementconsole in the tab "history/maperrors" showing the following error.

CardError= -550
maperror= 12

it is happening for every 30 min.

Suggest me how to solve this.

Thanks

 

[BocaBurger]

If you are looking for files, why not just set a file trigger for the files?
If a map has a file source, time trigger and the file is not there, then it is an error.

You might want to play with time and event cooredination, but a file trigger is the easiest solution.


BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[Guest_imported]

Some of them are FTP (-url ftp://xxx*.xml;type=D) one with FTP has the problem.

 

[BocaBurger]

Make the source data optional. Then the map won't fail. This is a problem when you expect data to be available and it isn't. Make the source optional is not good for validation.

Best solution would be for the source to notify you that data is present (via e-mail, socket, or other triggerable source, so you can run a map to do the FTP get.
Etither that or set up an FTP server so they can send you the data, and you can trigger from that. Almost any fucntioning server or workstation could have Linux installed (use a hardened distribution) and set up FTP or SFTP, and place in your DMZ.





BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[Guest_imported]

The following Error I got it in the compundsytem.txt.I am just curious to know that is it problem in start time here.

I changed the priority setting of first map to "highest" even still getting the same.

I changed the IdleFTP = 60 from 30 in dstx.ini. No change observed. pls help me out in this issue.

System: xxxxxxx
Component: ftp_get
Map: C:\xxx\xxxxxxxxxx.mmc
Start Time: Tue Jan 24 10:01:07 2006
Instance: 1
Run Time: 4.3375 seconds
Return Code: 12 - Source not available

Input 1: FTP: -url ftp://xxx/folder1/inbound/*.xml;type=D
Output 1: File: c:\xx\xxxx.txt

Audit File: <none>

==============================================================================

*** ERROR:

System: yyyyyyyy
Component: ftp_get1
Map: C:\yyyyy\yyyyyyyyyy.mmc
Start Time: Tue Jan 24 10:01:08 2006
Instance: 1
Run Time: 8.1589 seconds
Return Code: 12 - Source not available

Input 1: FTP: -url ftp://yyy/folder2/inbound/*.xml;type=D
Output 1: File: c:\yy\yyyy.txt

Audit File: <none>

 

[janhes]

Look in the map logs to find out which source card is not available but it looks like *.xml.

 

[janhes]

Are you triggering your map on the *.xml file? With ftp the file is created as soon as the ftp starts so your map will start running before all the data has arrived. You will also get a double trigger, once when the file is first created and once when the file transfer is complete. If your map is deleting the input, then the file may not be available to satisfy the second trigger resulting in the Source Not Available message.

 

[Guest_imported]

There is only one card. I have GPFs creating in the install directory but did't cause from crashing the ES .Help me out to solve this issue.

 

[Guest_imported]

Map is not deleting the source file janhes.Thanks

 

[janhes]

You could try changing TriggerTime=1 to a larger value depending on the size of the input files. This is how long the map waits before trigering after it receives a trigger request. So for ftp you could try 10. May need to be bigger if files are larger.

 

[BocaBurger]

10 is much too high. 2 or 3 is the practical maximum. Also set the map delay on this map to match the trigger time. If this doesn't work, here are some possible solutions:

1. Have the source FTP the file with a .tmp extension, and then rename when complete.
2. Have the source send a 1 byte trigger file after the data file is sent, make sure the file names are the same *.trig and *.XML, where the * resolves to the same file name.
Set the trigger on the .trig file.
3. Increase the speed of the network and/or hardware so the FTP doesn't take as long.



BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[janhes]

I usually use TriggerTime=5 but I occasionally get a problem with error 12 even though I mainly trigger on an empty dummy file.

 

[BocaBurger]

Looked at the error again.
Since the source is FTP, there is no trigger on the map. It would be time triggered. Type=D, this means get a list of the files in the directory. No files in the directory, you get a source not available.

There is no good way to fix this from DS TX alone. You would need to keep a file in the directory that would not be processed by the map. Your RUN map call should have an IF statement so that if the file name is dummy.xml=none, else RUN(map) etc. dummy.xml would remain in the source directory.
You would pass the file names to the run map, but not send the dummy.xml file name.




BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!