Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Problem with Informix checkpints. 1

Status
Not open for further replies.

AndrisS

IS-IT--Management
Oct 12, 2006
20
LV
Good day,

We have problem with full checkpoints on Informix DB.
Sometimes checkpoints are more then 6 sec.

Please help me to reslove problem.
Please see below oconfig file and output from onstat -F command.

oconfig

#**************************************************************************
#
# INFORMIX SOFTWARE, INC.
#
# Title: onconfig.std
# Description: Informix Dynamic Server Configuration Parameters
#
#**************************************************************************

# Root Dbspace Configuration

ROOTNAME rootdb # Root dbspace name
ROOTPATH /dev/rrootdb # Path for device containing root dbspace
ROOTOFFSET 0 # Offset of root dbspace into device (Kbytes)
ROOTSIZE 131072 # Size of root dbspace (Kbytes)

# Disk Mirroring Configuration Parameters

MIRROR 0 # Mirroring flag (Yes = 1, No = 0)
MIRRORPATH # Path for device containing mirrored root
MIRROROFFSET 0 # Offset into mirrored device (Kbytes)

# Physical Log Configuration

PHYSDBS plog # Location (dbspace) of physical log
PHYSFILE 16000 # Physical log file size (Kbytes)

# Logical Log Configuration

LOGFILES 785 # Number of logical log files
LOGSIZE 1024 # Logical log size (Kbytes)

# Diagnostics

MSGPATH /ctxtools/informix/online.log # System message log file path
CONSOLE /dev/null # System console message path
ALARMPROGRAM /ctxtools/informix/etc/log_full.sh # Alarm program path
TBLSPACE_STATS 1 # Maintain tblspace statistics

# System Archive Tape Device

TAPEDEV /ctxtools/informix/devices/tape # Tape device path
#TAPEBLK 16 # Tape block size (Kbytes)
TAPEBLK 32768
TAPESIZE 20971520 # Maximum amount of data to put on tape (Kbytes)

# Log Archive Tape Device

LTAPEDEV /ctxtools/informix/devices/ltape # Log tape device path
LTAPEBLK 16 # Log tape block size (Kbytes)
LTAPESIZE 104857600 # Max amount of data to put on log tape (Kbytes)

# Optical

STAGEBLOB # Informix Dynamic Server staging area

# System Configuration

SERVERNUM 0 # Unique id corresponding to a OnLine instance
DBSERVERNAME cortex_online # Name of default database server
DBSERVERALIASES cortex_local # List of alternate dbservernames
NETTYPE soctcp,8,128,NET # Configure poll thread(s) for nettype
DEADLOCK_TIMEOUT 60 # Max time to wait of lock in distributed env.
RESIDENT 0 # Forced residency flag (Yes = 1, No = 0)

MULTIPROCESSOR 1 # 0 for single-processor, 1 for multi-processor
NUMCPUVPS 4 # Number of user (cpu) vps
SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps to one

NOAGE 1 # Process aging
AFF_SPROC 0 # Affinity start processor
AFF_NPROCS 0 # Affinity number of processors

# Shared Memory Parameters

LOCKS 1000000 # Maximum number of locks
#BUFFERS 131072 # Maximum number of shared buffers
BUFFERS 65536
NUMAIOVPS # Number of IO vps
PHYSBUFF 64 # Physical log buffer size (Kbytes)
LOGBUFF 32 # Logical log buffer size (Kbytes)
CLEANERS 128 # Number of buffer cleaner processes
SHMBASE 0x30000000 # Shared memory base address
#SHMVIRTSIZE 131072 # initial virtual shared memory segment size
SHMVIRTSIZE 1048676
#SHMADD 65536 # Size of new shared memory segments (Kbytes)
SHMADD 262144
SHMTOTAL 0 # Total shared memory (Kbytes). 0=>unlimited
CKPTINTVL 30 # Check point interval (in sec)
LRUS 128 # Number of LRU queues
LRU_MAX_DIRTY 1 # LRU percent dirty begin cleaning limit
LRU_MIN_DIRTY 0 # LRU percent dirty end cleaning limit
TXTIMEOUT 0xa # Transaction timeout (in sec)
STACKSIZE 32


NOFUZZYCKPT 1
# Dynamic Logging
# DYNAMIC_LOGS:
# 2 : server automatically add a new logical log when necessary. (ON)
# 1 : notify DBA to add new logical logs when necessary. (ON)
# 0 : cannot add logical log on the fly. (OFF)
#
# When dynamic logging is on, we can have higher values for LTXHWM/LTXEHWM,
# because the server can add new logical logs during long transaction rollback.
# However, to limit the number of new logical logs being added, LTXHWM/LTXEHWM
# can be set to smaller values.
#
# If dynamic logging is off, LTXHWM/LTXEHWM NEED to be set to smaller values
# to avoid long transaction rollback hanging the server due to lack of logical
# log space, i.e. 50/60 or lower.

DYNAMIC_LOGS 0
LTXHWM 50
LTXEHWM 60

# System Page Size
# BUFFSIZE - OnLine no longer supports this configuration parameter.
# To determine the page size used by OnLine on your platform
# see the last line of output from the command, 'onstat -b'.


# Recovery Variables
# OFF_RECVRY_THREADS:
# Number of parallel worker threads during fast recovery or an offline restore.
# ON_RECVRY_THREADS:
# Number of parallel worker threads during an online restore.

OFF_RECVRY_THREADS 10 # Default number of offline worker threads
ON_RECVRY_THREADS 10 # Default number of online worker threads

# Data Replication Variables
DRINTERVAL 0 # DR max time between DR buffer flushes (in sec)
DRTIMEOUT 15 # DR network timeout (in sec)
DRLOSTFOUND /ctxtools/informix/etc/dr.lostfound # DR lost+found file path

# CDR Variables
CDR_EVALTHREADS 1,2 # evaluator threads (per-cpu-vp,additional)
CDR_DSLOCKWAIT 5 # DS lockwait timeout (seconds)
CDR_QUEUEMEM 4096 # Maximum amount of memory for any CDR queue (Kbytes)
CDR_NIFCOMPRESS 0 # Link level compression (-1 never, 0 none, 9 max)
CDR_SERIAL 0,0 # Serial Column Sequence
CDR_DBSPACE # dbspace for syscdr database
CDR_QHDR_DBSPACE # CDR queue dbspace (default same as catalog)
CDR_QDATA_SBSPACE # CDR queue smart blob space
CDR_QDATA_SBFLAGS 0 # Log/no-log (default no log)


# Backup/Restore variables
BAR_ACT_LOG /ctxtools/informix/bar_act.log
# ON-Bar Log file - not in /tmp please
BAR_DEBUG_LOG /ctxtools/informix/bar_dbug.log
# ON-Bar Debug Log - not in /tmp please
BAR_MAX_BACKUP 0
BAR_RETRY 1
BAR_NB_XPORT_COUNT 10
BAR_XFER_BUF_SIZE 31
RESTARTABLE_RESTORE on
BAR_PROGRESS_FREQ 0

# Informix Storage Manager variables
ISM_DATA_POOL ISMData
ISM_LOG_POOL ISMLogs

# Read Ahead Variables
RA_PAGES # Number of pages to attempt to read ahead
RA_THRESHOLD # Number of pages left before next group

# DBSPACETEMP:
# OnLine equivalent of DBTEMP for SE. This is the list of dbspaces
# that the OnLine SQL Engine will use to create temp tables etc.
# If specified it must be a colon separated list of dbspaces that exist
# when the OnLine system is brought online. If not specified, or if
# all dbspaces specified are invalid, various ad hoc queries will create
# temporary files in /tmp instead.

DBSPACETEMP tempdb # Default temp dbspaces

# DUMP*:
# The following parameters control the type of diagnostics information which
# is preserved when an unanticipated error condition (assertion failure) occurs
# during OnLine operations.
# For DUMPSHMEM, DUMPGCORE and DUMPCORE 1 means Yes, 0 means No.

DUMPDIR /tmp # Preserve diagnostics in this directory
DUMPSHMEM 1 # Dump a copy of shared memory
DUMPGCORE 0 # Dump a core image using 'gcore'
DUMPCORE 0 # Dump a core image (Warning:this aborts OnLine)
DUMPCNT 1 # Number of shared memory or gcore dumps for
# a single user's session

FILLFACTOR 90 # Fill factor for building indexes

# method for OnLine to use when determining current time
USEOSTIME 1 # 0: use internal time(fast), 1: get time from OS(slow)

# Parallel Database Queries (pdq)
MAX_PDQPRIORITY 20 # Maximum allowed pdqpriority
DS_MAX_QUERIES 1000 # Maximum number of decision support queries
DS_TOTAL_MEMORY # Decision support memory (Kbytes)
DS_MAX_SCANS 1048576 # Maximum number of decision support scans
DATASKIP off # List of dbspaces to skip

# OPTCOMPIND
# 0 => Nested loop joins will be preferred (where
# possible) over sortmerge joins and hash joins.
# 1 => If the transaction isolation mode is not
# "repeatable read", optimizer behaves as in (2)
# below. Otherwise it behaves as in (0) above.
# 2 => Use costs regardless of the transaction isolation
# mode. Nested loop joins are not necessarily
# preferred. Optimizer bases its decision purely
# on costs.
OPTCOMPIND 0 # To hint the optimizer

DIRECTIVES 1 # Optimizer DIRECTIVES ON (1/Default) or OFF (0)

ONDBSPACEDOWN 2 # Dbspace down option: 0 = CONTINUE, 1 = ABORT, 2 = WAIT
OPCACHEMAX 0 # Maximum optical cache size (Kbytes)

# HETERO_COMMIT (Gateway participation in distributed transactions)
# 1 => Heterogeneous Commit is enabled
# 0 (or any other value) => Heterogeneous Commit is disabled
HETERO_COMMIT 1

SBSPACENAME # Default smartblob space name - this is where blobs
# go if no sbspace is specified when the smartblob is
# created. It is also used by some datablades as
# the location to put their smartblobs.
SYSSBSPACENAME # Default smartblob space for use by the Informix
# Server. This is used primarily for Informix Server
# system statistics collection.

BLOCKTIMEOUT 3600 # Default timeout for system block
SYSALARMPROGRAM /ctxtools/informix/etc/evidence.sh # System Alarm program path

# Optimization goal: -1 = ALL_ROWS(Default), 0 = FIRST_ROWS
OPT_GOAL -1

ALLOW_NEWLINE 0 # embedded newlines(Yes = 1, No = 0 or anything but 1)

#
# The following are default settings for enabling Java in the database.
# Replace all occurrences of /usr/informix with the value of $INFORMIXDIR.

#VPCLASS jvp,num=1 # Number of JVPs to start with

JVPJAVAHOME /ctxtools/informix/extend/krakatoa/jre
# JRE installation root directory
JVPHOME /ctxtools/informix/extend/krakatoa
# Krakatoa installation directory

JVPPROPFILE /ctxtools/informix/extend/krakatoa/.jvpprops # JVP property file
JVPLOGFILE /ctxtools/informix/jvp.log # JVP log file.

JDKVERSION 1.3 # JDK version supported by this server

# The path to the JRE libraries relative to JVPJAVAHOME
JVPJAVALIB /bin

# The JRE libraries to use for the Java VM

JVPJAVAVM hpi:jvm:java:net:zip:jpeg

# use JVPARGS to change Java VM configuration
#To display jni call
#JVPARGS -verbose:jni

# Classpath to use upon Java VM start-up (use _g version for debugging)

#JVPCLASSPATH /usr/informix/extend/krakatoa/krakatoa_g.jar:/usr/informix/extend/krakatoa/jdbc_g.jar
JVPCLASSPATH /usr/informix/extend/krakatoa/krakatoa.jar:/usr/informix/extend/krakatoa/jdbc.jar


onstat -F

Fg Writes LRU Writes Chunk Writes
0 37681152 29571184

address flusher state data
5013b618 0 I 0 = 0X0
5013bc18 1 I 0 = 0X0
5013c218 2 I 0 = 0X0
5013c818 3 I 0 = 0X0
5013ce18 4 I 0 = 0X0
5013d418 5 I 0 = 0X0
5013da18 6 I 0 = 0X0
5013e018 7 I 0 = 0X0
5013e618 8 I 0 = 0X0
5013ec18 9 I 0 = 0X0
5013f218 10 I 0 = 0X0
5013f818 11 I 0 = 0X0
states: Exit Idle Chunk Lru
 
the config-file looks a bit like anybody pulled all known switches at the same time, without knowing what they are doing... :)

to make any detailed comments on your config please provide:

- number of physical cpus (inlude info on hyperthreading)
- physical ram
- is your server a dedicated database server (i.e. is allowed to use the complete memory)
- how many chunks are in your database?
- what does the tablespace/chunks/disks-config look like? how many and how are they spread on the disks?
- when are checkpoints > 6 sec? during heavy workload or more often?
- how many users are connected to the database at the same time?
 
Yes, you right about config file.
Thank for reply.

And here is info what you need:

- 2 CPU
- 4G of phisical RAM
- yes, it is dedicated

- Fg Writes LRU Writes Chunk Writes
0 37681152 29571184

- it RAID disks, where are 15 tablespaces
- the chescpoints some times 6 sec some times 50 sec, there is no overload.
- in the same time connected 600 users.
 
Is the output from onstat -F truncated for the purpose of posting or is that the entire output?

If it is then this onconfig file might not the file IDS is using because, according to the onstat-F output, you have 11 LRU's and the onconfig states you have 128????

Run onstat -R and look at the bottom of the output. This will verify total # of LRU pairs, total # of BUFFERS, and and LRU_MIN_DIRTY & LRU_MAX_DIRTY numbers.
 
forgot to add that I've experimented with the OPTCOMPIND setting (seen many recommendations to set it to 0 for an OLTP system) and I found my best results with leaving it at the default of 2.

A few processes ran faster, most ran the same, but many ran up to 10 - 20 times LONGER with OPTCOMPIND set to 2, even with statistics updated & current.

No real advice on this one 'cause it's site specific but it is probably something to consider when the immediate problem is resolved.
 
The total output of onstat -F command is:
Fg Writes LRU Writes Chunk Writes
0 37681152 29571184

address flusher state data
5013b618 0 I 0 = 0X0
5013bc18 1 I 0 = 0X0
5013c218 2 I 0 = 0X0
5013c818 3 I 0 = 0X0
5013ce18 4 I 0 = 0X0
5013d418 5 I 0 = 0X0
5013da18 6 I 0 = 0X0
5013e018 7 I 0 = 0X0
5013e618 8 I 0 = 0X0
5013ec18 9 I 0 = 0X0
5013f218 10 I 0 = 0X0
5013f818 11 I 0 = 0X0
5013fe18 12 I 0 = 0X0
50140418 13 I 0 = 0X0
50140a18 14 I 0 = 0X0
50141018 15 I 0 = 0X0
50141618 16 I 0 = 0X0
50141c18 17 I 0 = 0X0
50142218 18 I 0 = 0X0
50142818 19 I 0 = 0X0
50142e18 20 I 0 = 0X0
50143418 21 I 0 = 0X0
50143a18 22 I 0 = 0X0
50144018 23 I 0 = 0X0
50144618 24 I 0 = 0X0
50144c18 25 I 0 = 0X0
50145218 26 I 0 = 0X0
50145818 27 I 0 = 0X0
50145e18 28 I 0 = 0X0
50146418 29 I 0 = 0X0
50146a18 30 I 0 = 0X0
50147018 31 I 0 = 0X0
50147618 32 I 0 = 0X0
50147c18 33 I 0 = 0X0
50148218 34 I 0 = 0X0
50148818 35 I 0 = 0X0
50148e18 36 I 0 = 0X0
50149418 37 I 0 = 0X0
50149a18 38 I 0 = 0X0
5014a018 39 I 0 = 0X0
5014a618 40 I 0 = 0X0
5014ac18 41 I 0 = 0X0
5014b218 42 I 0 = 0X0
5014b818 43 I 0 = 0X0
5014be18 44 I 0 = 0X0
5014c418 45 I 0 = 0X0
5014ca18 46 I 0 = 0X0
5014d018 47 I 0 = 0X0
5014d618 48 I 0 = 0X0
5014dc18 49 I 0 = 0X0
5014e218 50 I 0 = 0X0
5014e818 51 I 0 = 0X0
5014ee18 52 I 0 = 0X0
5014f418 53 I 0 = 0X0
5014fa18 54 I 0 = 0X0
50150018 55 I 0 = 0X0
50150618 56 I 0 = 0X0
50150c18 57 I 0 = 0X0
50151218 58 I 0 = 0X0
50151818 59 I 0 = 0X0
50151e18 60 I 0 = 0X0
50152418 61 I 0 = 0X0
50152a18 62 I 0 = 0X0
50153018 63 I 0 = 0X0
50153618 64 I 0 = 0X0
50153c18 65 I 0 = 0X0
50154218 66 I 0 = 0X0
50154818 67 I 0 = 0X0
50154e18 68 I 0 = 0X0
50155418 69 I 0 = 0X0
50155a18 70 I 0 = 0X0
50156018 71 I 0 = 0X0
50156618 72 I 0 = 0X0
50156c18 73 I 0 = 0X0
50157218 74 I 0 = 0X0
50157818 75 I 0 = 0X0
50157e18 76 I 0 = 0X0
50158418 77 I 0 = 0X0
50158a18 78 I 0 = 0X0
50159018 79 I 0 = 0X0
50159618 80 I 0 = 0X0
50159c18 81 I 0 = 0X0
5015a218 82 I 0 = 0X0
5015a818 83 I 0 = 0X0
5015ae18 84 I 0 = 0X0
5015b418 85 I 0 = 0X0
5015ba18 86 I 0 = 0X0
5015c018 87 I 0 = 0X0
5015c618 88 I 0 = 0X0
5015cc18 89 I 0 = 0X0
5015d218 90 I 0 = 0X0
5015d818 91 I 0 = 0X0
5015de18 92 I 0 = 0X0
5015e418 93 I 0 = 0X0
5015ea18 94 I 0 = 0X0
5015f018 95 I 0 = 0X0
5015f618 96 I 0 = 0X0
5015fc18 97 I 0 = 0X0
50160218 98 I 0 = 0X0
50160818 99 I 0 = 0X0
50160e18 100 I 0 = 0X0
50161418 101 I 0 = 0X0
50161a18 102 I 0 = 0X0
50162018 103 I 0 = 0X0
50162618 104 I 0 = 0X0
50162c18 105 I 0 = 0X0
50163218 106 I 0 = 0X0
50163818 107 I 0 = 0X0
50163e18 108 I 0 = 0X0
50164418 109 I 0 = 0X0
50164a18 110 I 0 = 0X0
50165018 111 I 0 = 0X0
50165618 112 I 0 = 0X0
50165c18 113 I 0 = 0X0
50166218 114 I 0 = 0X0
50166818 115 I 0 = 0X0
50166e18 116 I 0 = 0X0
50167418 117 I 0 = 0X0
50167a18 118 I 0 = 0X0
50168018 119 I 0 = 0X0
50168618 120 I 0 = 0X0
50168c18 121 I 0 = 0X0
50169218 122 I 0 = 0X0
50169818 123 I 0 = 0X0
50169e18 124 I 0 = 0X0
5016a418 125 I 0 = 0X0
5016aa18 126 I 0 = 0X0
50fe0018 127 I 0 = 0X0
states: Exit Idle Chunk Lru



And the output of onstat -R command:

128 buffer LRU queue pairs priority levels
# f/m pair total % of length LOW MED_LOW MED_HIGH HIGH
0 f 512 99.2% 508 0 415 93 0
1 m 0.8% 4 0 4 0 0
2 f 510 99.0% 505 0 419 86 0
3 m 1.0% 5 0 5 0 0
4 f 512 99.0% 507 0 409 97 1
5 m 1.0% 5 0 5 0 0
6 f 511 99.4% 508 0 415 93 0
7 m 0.6% 3 0 3 0 0
8 f 512 99.4% 509 0 429 79 1
9 m 0.6% 3 0 3 0 0
10 f 509 99.2% 505 0 423 81 1
11 m 0.8% 4 0 4 0 0
12 f 512 99.4% 509 0 424 83 2
13 m 0.6% 3 0 3 0 0
14 f 512 99.8% 511 0 407 103 1
15 m 0.2% 1 0 1 0 0
16 f 510 99.6% 508 0 427 79 2
17 m 0.4% 2 0 2 0 0
18 f 510 99.4% 507 0 425 81 1
19 m 0.6% 3 0 3 0 0
20 f 512 99.0% 507 0 430 77 0
21 m 1.0% 5 0 5 0 0
22 f 512 99.4% 509 0 416 91 2
23 m 0.6% 3 0 3 0 0
24 f 511 99.0% 506 0 412 93 1
25 m 1.0% 5 0 5 0 0
26 f 510 99.8% 509 0 413 96 0
27 m 0.2% 1 0 1 0 0
28 f 510 99.2% 506 0 427 78 1
29 m 0.8% 4 0 4 0 0
30 f 511 99.6% 509 0 418 88 3
31 m 0.4% 2 0 2 0 0
32 f 510 99.0% 505 0 417 87 1
33 m 1.0% 5 0 5 0 0
34 f 510 99.2% 506 0 423 83 0
35 m 0.8% 4 0 3 1 0
36 f 511 99.6% 509 0 416 92 1
37 m 0.4% 2 0 2 0 0
38 f 510 99.6% 508 0 418 88 2
39 m 0.4% 2 0 2 0 0
40 f 511 99.4% 508 0 431 77 0
41 m 0.6% 3 0 3 0 0
42 f 510 99.0% 505 0 415 89 1
43 m 1.0% 5 0 5 0 0
44 f 509 99.6% 507 0 421 86 0
45 m 0.4% 2 0 2 0 0
46 f 512 99.6% 510 0 415 95 0
47 m 0.4% 2 0 2 0 0
48 f 510 99.4% 507 0 419 86 2
49 m 0.6% 3 0 3 0 0
50 f 510 99.4% 507 0 412 94 1
51 m 0.6% 3 0 3 0 0
52 f 512 99.0% 507 0 417 88 2
53 m 1.0% 5 0 4 1 0
54 f 512 99.8% 511 0 436 75 0
55 m 0.2% 1 0 1 0 0
56 f 511 99.6% 509 0 423 82 4
57 m 0.4% 2 0 2 0 0
58 f 511 99.8% 510 0 417 92 1
59 m 0.2% 1 0 1 0 0
60 f 510 100.0% 510 0 410 99 1
61 m 0.0% 0 0 0 0 0
62 f 508 99.2% 504 1 421 80 2
63 m 0.8% 4 0 4 0 0
64 f 512 99.6% 510 0 429 81 0
65 m 0.4% 2 0 2 0 0
66 f 511 99.0% 506 0 433 73 0
67 m 1.0% 5 0 5 0 0
68 f 511 99.2% 507 1 427 77 2
69 m 0.8% 4 0 4 0 0
70 f 511 99.8% 510 1 425 83 1
71 m 0.2% 1 0 1 0 0
72 f 512 99.4% 509 1 410 96 2
73 m 0.6% 3 0 3 0 0
74 f 511 99.0% 506 1 416 88 1
75 m 1.0% 5 0 5 0 0
76 f 510 99.0% 505 0 424 79 2
77 m 1.0% 5 0 5 0 0
78 f 511 99.0% 506 0 420 83 3
79 m 1.0% 5 0 5 0 0
80 f 510 99.4% 507 0 420 85 2
81 m 0.6% 3 0 3 0 0
82 f 510 99.4% 507 0 426 78 3
83 m 0.6% 3 0 3 0 0
84 f 512 99.2% 508 0 425 82 1
85 m 0.8% 4 0 4 0 0
86 f 511 99.0% 506 0 427 79 0
87 m 1.0% 5 0 5 0 0
88 f 511 99.0% 506 0 419 85 2
89 m 1.0% 5 0 5 0 0
90 f 509 99.4% 506 0 406 100 0
91 m 0.6% 3 0 3 0 0
92 f 512 99.8% 511 0 439 70 2
93 m 0.2% 1 0 1 0 0
94 f 510 99.2% 506 0 405 99 2
95 m 0.8% 4 0 4 0 0
96 f 511 99.0% 506 0 429 76 1
97 m 1.0% 5 0 5 0 0
98 f 511 99.8% 510 0 416 94 0
99 m 0.2% 1 0 1 0 0
100 f 512 99.6% 510 0 420 89 1
101 m 0.4% 2 0 2 0 0
102 f 510 99.6% 508 0 437 70 1
103 m 0.4% 2 0 2 0 0
104 f 511 99.0% 506 0 416 88 2
105 m 1.0% 5 0 5 0 0
106 f 510 99.4% 507 0 421 86 0
107 m 0.6% 3 0 3 0 0
108 f 512 99.4% 509 0 418 90 1
109 m 0.6% 3 0 3 0 0
110 f 509 99.4% 506 0 418 88 0
111 m 0.6% 3 0 3 0 0
112 f 513 99.8% 512 0 416 95 1
113 m 0.2% 1 0 1 0 0
114 f 510 99.4% 507 0 416 89 2
115 m 0.6% 3 0 3 0 0
116 f 512 99.0% 507 0 407 100 0
117 m 1.0% 5 0 5 0 0
118 f 511 99.0% 506 0 419 85 2
119 m 1.0% 5 0 5 0 0
120 f 511 99.8% 510 0 407 103 0
121 m 0.2% 1 0 1 0 0
122 f 512 99.4% 509 0 410 97 2
123 m 0.6% 3 0 3 0 0
124 f 511 99.2% 507 1 440 64 2
125 m 0.8% 4 0 4 0 0
126 f 510 100.0% 510 1 416 91 2
127 m 0.0% 0 0 0 0 0
128 f 513 99.2% 509 0 416 92 1
129 m 0.8% 4 0 4 0 0
130 f 511 99.6% 509 0 432 76 1
131 m 0.4% 2 0 2 0 0
132 f 512 99.6% 510 0 418 91 1
133 m 0.4% 2 0 2 0 0
134 f 512 99.6% 510 0 429 81 0
135 m 0.4% 2 0 2 0 0
136 f 511 99.6% 509 0 438 70 1
137 m 0.4% 2 0 2 0 0
138 f 511 99.6% 509 1 422 85 1
139 m 0.4% 2 0 2 0 0
140 f 511 99.0% 506 0 427 78 1
141 m 1.0% 5 0 5 0 0
142 f 510 100.0% 510 0 414 94 2
143 m 0.0% 0 0 0 0 0
144 f 508 99.4% 505 0 429 75 1
145 m 0.6% 3 0 3 0 0
146 f 512 99.0% 507 0 426 81 0
147 m 1.0% 5 0 5 0 0
148 f 510 99.4% 507 0 431 76 0
149 m 0.6% 3 0 3 0 0
150 F 512 100.0% 512 0 430 82 0
151 m 0.0% 0 0 0 0 0
152 f 511 99.4% 508 0 426 80 2
153 m 0.6% 3 0 3 0 0
154 f 512 99.8% 511 0 433 76 2
155 m 0.2% 1 0 1 0 0
156 f 512 99.2% 508 0 410 95 3
157 m 0.8% 4 0 4 0 0
158 f 511 99.6% 509 0 423 84 2
159 m 0.4% 2 0 2 0 0
160 f 510 99.4% 507 0 410 95 2
161 m 0.6% 3 0 3 0 0
162 f 511 99.0% 506 0 412 94 0
163 m 1.0% 5 0 5 0 0
164 f 512 99.2% 508 0 422 82 4
165 m 0.8% 4 0 4 0 0
166 f 512 99.6% 510 0 426 84 0
167 m 0.4% 2 0 2 0 0
168 f 514 99.2% 510 0 419 85 6
169 m 0.8% 4 0 4 0 0
170 f 510 98.8% 504 0 424 79 1
171 m 1.2% 6 0 6 0 0
172 f 507 100.0% 507 0 423 84 0
173 m 0.0% 0 0 0 0 0
174 f 513 99.0% 508 0 415 93 0
175 m 1.0% 5 0 5 0 0
176 f 511 100.0% 511 0 437 74 0
177 m 0.0% 0 0 0 0 0
178 f 508 99.6% 506 0 430 73 3
179 m 0.4% 2 0 2 0 0
180 f 511 99.4% 508 0 433 72 3
181 m 0.6% 3 0 3 0 0
182 f 512 99.0% 507 0 416 90 1
183 m 1.0% 5 0 5 0 0
184 f 512 99.4% 509 0 428 80 1
185 m 0.6% 3 0 3 0 0
186 f 511 99.6% 509 0 423 84 2
187 m 0.4% 2 0 2 0 0
188 f 510 99.0% 505 0 410 93 2
189 m 1.0% 5 0 5 0 0
190 f 512 99.4% 509 0 427 82 0
191 m 0.6% 3 0 3 0 0
192 f 511 99.0% 506 0 430 74 2
193 m 1.0% 5 0 5 0 0
194 f 512 99.2% 508 0 425 81 2
195 m 0.8% 4 0 4 0 0
196 f 509 99.2% 505 0 421 81 3
197 m 0.8% 4 0 4 0 0
198 f 511 100.0% 511 0 424 87 0
199 m 0.0% 0 0 0 0 0
200 f 513 99.2% 509 0 418 91 0
201 m 0.8% 4 0 4 0 0
202 f 510 99.0% 505 0 415 90 0
203 m 1.0% 5 0 5 0 0
204 f 511 99.6% 509 0 435 73 1
205 m 0.4% 2 0 2 0 0
206 f 512 99.8% 511 0 423 88 0
207 m 0.2% 1 0 1 0 0
208 f 512 99.4% 509 0 426 82 1
209 m 0.6% 3 0 3 0 0
210 f 511 99.6% 509 0 406 102 1
211 m 0.4% 2 0 2 0 0
212 f 512 100.0% 512 0 426 85 1
213 m 0.0% 0 0 0 0 0
214 f 512 99.6% 510 0 422 87 1
215 m 0.4% 2 0 2 0 0
216 f 512 99.4% 509 0 436 73 0
217 m 0.6% 3 0 3 0 0
218 f 512 99.0% 507 0 421 86 0
219 m 1.0% 5 0 5 0 0
220 f 512 99.4% 509 0 423 86 0
221 m 0.6% 3 0 3 0 0
222 f 510 99.4% 507 0 429 77 1
223 m 0.6% 3 0 3 0 0
224 f 512 99.6% 510 0 429 81 0
225 m 0.4% 2 0 2 0 0
226 f 511 99.4% 508 0 414 92 2
227 m 0.6% 3 0 3 0 0
228 f 512 100.0% 512 0 434 77 1
229 m 0.0% 0 0 0 0 0
230 f 510 99.4% 507 0 431 76 0
231 m 0.6% 3 0 3 0 0
232 f 512 99.4% 509 0 439 69 1
233 m 0.6% 3 0 2 1 0
234 f 511 99.0% 506 0 406 99 1
235 m 1.0% 5 0 5 0 0
236 f 512 99.0% 507 0 430 77 0
237 m 1.0% 5 0 4 1 0
238 f 512 99.2% 508 0 428 78 2
239 m 0.8% 4 0 4 0 0
240 f 511 99.2% 507 0 412 93 2
241 m 0.8% 4 0 4 0 0
242 f 505 99.8% 504 0 404 99 1
243 m 0.2% 1 0 1 0 0
244 f 512 99.0% 507 0 415 90 2
245 m 1.0% 5 0 5 0 0
246 f 508 99.4% 505 0 423 81 1
247 m 0.6% 3 0 3 0 0
248 f 512 99.4% 509 0 412 97 0
249 m 0.6% 3 0 2 1 0
250 f 511 99.8% 510 0 431 79 0
251 m 0.2% 1 0 1 0 0
252 f 511 99.6% 509 0 405 104 0
253 m 0.4% 2 0 2 0 0
254 f 511 99.4% 508 0 423 85 0
255 m 0.6% 3 0 3 0 0
386 dirty, 65403 queued, 65536 total, 65536 hash buckets, 4096 buffer size
start clean at 1% (of pair total) dirty, or 5 buffs dirty, stop at 0%
0 priority downgrades, 0 priority upgrades
 
thanks for the information, this brings us some light.

could you please post the output of

onstat

you can leave out the userhtreads, important is the profile-information.

Do you use cooked files (database is located in files) or raw devices?

could you pleas repost the last lines of onstat -R? your last posting was truncated and the relevant info is missing.

ok. lets go:

for a start i would suggest the following changes:

1. raise the buffers significantly, lets say 262144 for a beginning
2. CKPTINTVL 300 (that is the default)
3. comment out the NOFUZZYCKPT
4. NUMAIOVPS 16
5. CLEANERS 15
6. LRUS 15
7. RESIDENT 1
8. LRU_MAX_DIRTY 20
9. LRU_MIN_DIRTY 10

Thats a lot of changes at a single point of time, so be prepared to reset these, if you realise a problem. Normally you change only few settings and see what happens, then the next, but i think in your case there are some necessary changes, so we will slow down with changes after that :)
 
another point .. i never saw any installation with USEOSTIME set to 1. is there a reason for that? if not, set it to 0.
 
here is continue from onstat -R

111 m 0.6% 3 0 3 0 0
112 f 513 99.8% 512 0 416 95 1
113 m 0.2% 1 0 1 0 0
114 f 510 99.4% 507 0 416 89 2
115 m 0.6% 3 0 3 0 0
116 f 512 99.0% 507 0 407 100 0
117 m 1.0% 5 0 5 0 0
118 f 511 99.0% 506 0 419 85 2
119 m 1.0% 5 0 5 0 0
120 f 511 99.8% 510 0 407 103 0
121 m 0.2% 1 0 1 0 0
122 f 512 99.4% 509 0 410 97 2
123 m 0.6% 3 0 3 0 0
124 f 511 99.2% 507 1 440 64 2
125 m 0.8% 4 0 4 0 0
126 f 510 100.0% 510 1 416 91 2
127 m 0.0% 0 0 0 0 0
128 f 513 99.2% 509 0 416 92 1
129 m 0.8% 4 0 4 0 0
130 f 511 99.6% 509 0 432 76 1
131 m 0.4% 2 0 2 0 0
132 f 512 99.6% 510 0 418 91 1
133 m 0.4% 2 0 2 0 0
134 f 512 99.6% 510 0 429 81 0
135 m 0.4% 2 0 2 0 0
136 f 511 99.6% 509 0 438 70 1
137 m 0.4% 2 0 2 0 0
138 f 511 99.6% 509 1 422 85 1
139 m 0.4% 2 0 2 0 0
140 f 511 99.0% 506 0 427 78 1
141 m 1.0% 5 0 5 0 0
142 f 510 100.0% 510 0 414 94 2
143 m 0.0% 0 0 0 0 0
144 f 508 99.4% 505 0 429 75 1
145 m 0.6% 3 0 3 0 0
146 f 512 99.0% 507 0 426 81 0
147 m 1.0% 5 0 5 0 0
148 f 510 99.4% 507 0 431 76 0
149 m 0.6% 3 0 3 0 0
150 F 512 100.0% 512 0 430 82 0
151 m 0.0% 0 0 0 0 0
152 f 511 99.4% 508 0 426 80 2
153 m 0.6% 3 0 3 0 0
154 f 512 99.8% 511 0 433 76 2
155 m 0.2% 1 0 1 0 0
156 f 512 99.2% 508 0 410 95 3
157 m 0.8% 4 0 4 0 0
158 f 511 99.6% 509 0 423 84 2
159 m 0.4% 2 0 2 0 0
160 f 510 99.4% 507 0 410 95 2
161 m 0.6% 3 0 3 0 0
162 f 511 99.0% 506 0 412 94 0
163 m 1.0% 5 0 5 0 0
164 f 512 99.2% 508 0 422 82 4
165 m 0.8% 4 0 4 0 0
166 f 512 99.6% 510 0 426 84 0
167 m 0.4% 2 0 2 0 0
168 f 514 99.2% 510 0 419 85 6
169 m 0.8% 4 0 4 0 0
170 f 510 98.8% 504 0 424 79 1
171 m 1.2% 6 0 6 0 0
172 f 507 100.0% 507 0 423 84 0
173 m 0.0% 0 0 0 0 0
174 f 513 99.0% 508 0 415 93 0
175 m 1.0% 5 0 5 0 0
176 f 511 100.0% 511 0 437 74 0
177 m 0.0% 0 0 0 0 0
178 f 508 99.6% 506 0 430 73 3
179 m 0.4% 2 0 2 0 0
180 f 511 99.4% 508 0 433 72 3
181 m 0.6% 3 0 3 0 0
182 f 512 99.0% 507 0 416 90 1
183 m 1.0% 5 0 5 0 0
184 f 512 99.4% 509 0 428 80 1
185 m 0.6% 3 0 3 0 0
186 f 511 99.6% 509 0 423 84 2
187 m 0.4% 2 0 2 0 0
188 f 510 99.0% 505 0 410 93 2
189 m 1.0% 5 0 5 0 0
190 f 512 99.4% 509 0 427 82 0
191 m 0.6% 3 0 3 0 0
192 f 511 99.0% 506 0 430 74 2
193 m 1.0% 5 0 5 0 0
194 f 512 99.2% 508 0 425 81 2
195 m 0.8% 4 0 4 0 0
196 f 509 99.2% 505 0 421 81 3
197 m 0.8% 4 0 4 0 0
198 f 511 100.0% 511 0 424 87 0
199 m 0.0% 0 0 0 0 0
200 f 513 99.2% 509 0 418 91 0
201 m 0.8% 4 0 4 0 0
202 f 510 99.0% 505 0 415 90 0
203 m 1.0% 5 0 5 0 0
204 f 511 99.6% 509 0 435 73 1
205 m 0.4% 2 0 2 0 0
206 f 512 99.8% 511 0 423 88 0
207 m 0.2% 1 0 1 0 0
208 f 512 99.4% 509 0 426 82 1
209 m 0.6% 3 0 3 0 0
210 f 511 99.6% 509 0 406 102 1
211 m 0.4% 2 0 2 0 0
212 f 512 100.0% 512 0 426 85 1
213 m 0.0% 0 0 0 0 0
214 f 512 99.6% 510 0 422 87 1
215 m 0.4% 2 0 2 0 0
216 f 512 99.4% 509 0 436 73 0
217 m 0.6% 3 0 3 0 0
218 f 512 99.0% 507 0 421 86 0
219 m 1.0% 5 0 5 0 0
220 f 512 99.4% 509 0 423 86 0
221 m 0.6% 3 0 3 0 0
222 f 510 99.4% 507 0 429 77 1
223 m 0.6% 3 0 3 0 0
224 f 512 99.6% 510 0 429 81 0
225 m 0.4% 2 0 2 0 0
226 f 511 99.4% 508 0 414 92 2
227 m 0.6% 3 0 3 0 0
228 f 512 100.0% 512 0 434 77 1
229 m 0.0% 0 0 0 0 0
230 f 510 99.4% 507 0 431 76 0
231 m 0.6% 3 0 3 0 0
232 f 512 99.4% 509 0 439 69 1
233 m 0.6% 3 0 2 1 0
234 f 511 99.0% 506 0 406 99 1
235 m 1.0% 5 0 5 0 0
236 f 512 99.0% 507 0 430 77 0
237 m 1.0% 5 0 4 1 0
238 f 512 99.2% 508 0 428 78 2
239 m 0.8% 4 0 4 0 0
240 f 511 99.2% 507 0 412 93 2
241 m 0.8% 4 0 4 0 0
242 f 505 99.8% 504 0 404 99 1
243 m 0.2% 1 0 1 0 0
244 f 512 99.0% 507 0 415 90 2
245 m 1.0% 5 0 5 0 0
246 f 508 99.4% 505 0 423 81 1
247 m 0.6% 3 0 3 0 0
248 f 512 99.4% 509 0 412 97 0
249 m 0.6% 3 0 2 1 0
250 f 511 99.8% 510 0 431 79 0
251 m 0.2% 1 0 1 0 0
252 f 511 99.6% 509 0 405 104 0
253 m 0.4% 2 0 2 0 0
254 f 511 99.4% 508 0 423 85 0
255 m 0.6% 3 0 3 0 0
386 dirty, 65403 queued, 65536 total, 65536 hash buckets, 4096 buffer size
start clean at 1% (of pair total) dirty, or 5 buffs dirty, stop at 0%
0 priority downgrades, 0 priority upgrades
 
The databse is on raw devices.

Here is the output of onstat command:

Userthreads
address flags sessid user tty wait tout locks nreads nwrites
5013b018 ---P--D 1 informix - 0 0 0 4659 37840
5013b618 ---P--F 0 informix - 0 0 0 0 101303
5013bc18 ---P--F 0 informix - 0 0 0 0 68221
5013c218 ---P--F 0 informix - 0 0 0 0 132448
5013c818 ---P--F 0 informix - 0 0 0 0 207009
5013ce18 ---P--F 0 informix - 0 0 0 0 67678
5013d418 ---P--F 0 informix - 0 0 0 0 109964
5013da18 ---P--F 0 informix - 0 0 0 0 107440
5013e018 ---P--F 0 informix - 0 0 0 0 158404
5013e618 ---P--F 0 informix - 0 0 0 0 80440
5013ec18 ---P--F 0 informix - 0 0 0 0 68970
5013f218 ---P--F 0 informix - 0 0 0 0 26244
5013f818 ---P--F 0 informix - 0 0 0 0 19350
5013fe18 ---P--F 0 informix - 0 0 0 0 2427
50140418 ---P--F 0 informix - 0 0 0 0 20352
50140a18 ---P--F 0 informix - 0 0 0 0 1794
50141018 ---P--F 0 informix - 0 0 0 0 172
50141618 ---P--F 0 informix - 0 0 0 0 608
50141c18 ---P--F 0 informix - 0 0 0 0 21118
50142218 ---P--F 0 informix - 0 0 0 0 114964
50142818 ---P--F 0 informix - 0 0 0 0 11043
50142e18 ---P--F 0 informix - 0 0 0 0 5876
50143418 ---P--F 0 informix - 0 0 0 0 5815
50143a18 ---P--F 0 informix - 0 0 0 0 20638
50144018 ---P--F 0 informix - 0 0 0 0 13330
50144618 ---P--F 0 informix - 0 0 0 0 35957
50144c18 ---P--F 0 informix - 0 0 0 0 37798
50145218 ---P--F 0 informix - 0 0 0 0 47397
50145818 ---P--F 0 informix - 0 0 0 0 9694
50145e18 ---P--F 0 informix - 0 0 0 0 62352
50146418 ---P--F 0 informix - 0 0 0 0 3448
50146a18 ---P--F 0 informix - 0 0 0 0 56433
50147018 ---P--F 0 informix - 0 0 0 0 47375
50147618 ---P--F 0 informix - 0 0 0 0 9252
50147c18 ---P--F 0 informix - 0 0 0 0 55371
50148218 ---P--F 0 informix - 0 0 0 0 8013
50148818 ---P--F 0 informix - 0 0 0 0 53278
50148e18 ---P--F 0 informix - 0 0 0 0 4571
50149418 ---P--F 0 informix - 0 0 0 0 67232
50149a18 ---P--F 0 informix - 0 0 0 0 12578
5014a018 ---P--F 0 informix - 0 0 0 0 2739
5014a618 ---P--F 0 informix - 0 0 0 0 57790
5014ac18 ---P--F 0 informix - 0 0 0 0 565
5014b218 ---P--F 0 informix - 0 0 0 0 17563
5014b818 ---P--F 0 informix - 0 0 0 0 4063
5014be18 ---P--F 0 informix - 0 0 0 0 6186
5014c418 ---P--F 0 informix - 0 0 0 0 37480
5014ca18 ---P--F 0 informix - 0 0 0 0 5099
5014d018 ---P--F 0 informix - 0 0 0 0 69252
5014d618 ---P--F 0 informix - 0 0 0 0 37501
5014dc18 ---P--F 0 informix - 0 0 0 0 27235
5014e218 ---P--F 0 informix - 0 0 0 0 76101
5014e818 ---P--F 0 informix - 0 0 0 0 47902
5014ee18 ---P--F 0 informix - 0 0 0 0 28537
5014f418 ---P--F 0 informix - 0 0 0 0 16711
5014fa18 ---P--F 0 informix - 0 0 0 0 6299
50150018 ---P--F 0 informix - 0 0 0 0 5546
50150618 ---P--F 0 informix - 0 0 0 0 2711
50150c18 ---P--F 0 informix - 0 0 0 0 19133
50151218 ---P--F 0 informix - 0 0 0 0 3543
50151818 ---P--F 0 informix - 0 0 0 0 33588
50151e18 ---P--F 0 informix - 0 0 0 0 72865
50152418 ---P--F 0 informix - 0 0 0 0 4627
50152a18 ---P--F 0 informix - 0 0 0 0 6289
50153018 ---P--F 0 informix - 0 0 0 0 4757
50153618 ---P--F 0 informix - 0 0 0 0 871
50153c18 ---P--F 0 informix - 0 0 0 0 588
 
50154218 ---P--F 0 informix - 0 0 0 0 8398
50154818 ---P--F 0 informix - 0 0 0 0 2274
50154e18 ---P--F 0 informix - 0 0 0 0 6653
50155418 ---P--F 0 informix - 0 0 0 0 2180
50155a18 ---P--F 0 informix - 0 0 0 0 315
50156018 ---P--F 0 informix - 0 0 0 0 14615
50156618 ---P--F 0 informix - 0 0 0 0 1223
50156c18 ---P--F 0 informix - 0 0 0 0 47416
50157218 ---P--F 0 informix - 0 0 0 0 2544
50157818 ---P--F 0 informix - 0 0 0 0 1523
50157e18 ---P--F 0 informix - 0 0 0 0 131258
50158418 ---P--F 0 informix - 0 0 0 0 5383
50158a18 ---P--F 0 informix - 0 0 0 0 3985
50159018 ---P--F 0 informix - 0 0 0 0 10274
50159618 ---P--F 0 informix - 0 0 0 0 4913
50159c18 ---P--F 0 informix - 0 0 0 0 48092
5015a218 ---P--F 0 informix - 0 0 0 0 4174
5015a818 ---P--F 0 informix - 0 0 0 0 129673
5015ae18 ---P--F 0 informix - 0 0 0 0 15319
5015b418 ---P--F 0 informix - 0 0 0 0 34433
5015ba18 ---P--F 0 informix - 0 0 0 0 5386
5015c018 ---P--F 0 informix - 0 0 0 0 45679
5015c618 ---P--F 0 informix - 0 0 0 0 172495
5015cc18 ---P--F 0 informix - 0 0 0 0 56148
5015d218 ---P--F 0 informix - 0 0 0 0 4845
5015d818 ---P--F 0 informix - 0 0 0 0 2048
5015de18 ---P--F 0 informix - 0 0 0 0 80434
5015e418 ---P--F 0 informix - 0 0 0 0 11179
5015ea18 ---P--F 0 informix - 0 0 0 0 61774
5015f018 ---P--F 0 informix - 0 0 0 0 1745
5015f618 ---P--F 0 informix - 0 0 0 0 22768
5015fc18 ---P--F 0 informix - 0 0 0 0 49399
50160218 ---P--F 0 informix - 0 0 0 0 12122
50160818 ---P--F 0 informix - 0 0 0 0 358
50160e18 ---P--F 0 informix - 0 0 0 0 101620
50161418 ---P--F 0 informix - 0 0 0 0 23085
50161a18 ---P--F 0 informix - 0 0 0 0 16415
50162018 ---P--F 0 informix - 0 0 0 0 9841
50162618 ---P--F 0 informix - 0 0 0 0 95724
50162c18 ---P--F 0 informix - 0 0 0 0 24831
50163218 ---P--F 0 informix - 0 0 0 0 2483
50163818 ---P--F 0 informix - 0 0 0 0 60784
50163e18 ---P--F 0 informix - 0 0 0 0 7214
50164418 ---P--F 0 informix - 0 0 0 0 20352
50164a18 ---P--F 0 informix - 0 0 0 0 25033
50165018 ---P--F 0 informix - 0 0 0 0 77659
50165618 ---P--F 0 informix - 0 0 0 0 8542
50165c18 ---P--F 0 informix - 0 0 0 0 93207
50166218 ---P--F 0 informix - 0 0 0 0 59214
50166818 ---P--F 0 informix - 0 0 0 0 1662
50166e18 ---P--F 0 informix - 0 0 0 0 77255
50167418 ---P--F 0 informix - 0 0 0 0 62792
50167a18 ---P--F 0 informix - 0 0 0 0 9912
50168018 ---P--F 0 informix - 0 0 0 0 5770
50168618 ---P--F 0 informix - 0 0 0 0 6085
50168c18 ---P--F 0 informix - 0 0 0 0 27977
50169218 ---P--F 0 informix - 0 0 0 0 10173
50169818 ---P--F 0 informix - 0 0 0 0 588
50169e18 ---P--F 0 informix - 0 0 0 0 459
5016a418 ---P--F 0 informix - 0 0 0 0 8297
5016aa18 ---P--F 0 informix - 0 0 0 0 101
50fe0018 ---P--F 0 informix - 0 0 0 0 22198
50fe0618 ---P--- 14 informix - 0 0 0 0 22623
50fe0c18 ---P--B 15 informix - 0 0 0 4101 27222
50fe1218 Y--P--- 409115 nauda - 55b51578 0 1 521 34
50fe1e18 ---P--D 18 informix - 0 0 0 0 0
 
50fe3618 Y--P--- 409008 nauda - 57cca978 0 1 159 85
50fe3c18 Y--P--- 405425 sami XC20 51a55fc8 0 1 18460 0
50fe4218 Y--P--- 409128 nauda - 574c2248 0 1 7948 17
50fe4e18 Y--P--- 134354 cortex - 513dd478 0 1 0 0
50fe5418 Y--P--- 134359 cortex - 51b1f168 0 1 0 0
50fe5a18 Y--P--- 134360 cortex - 51e2f3b0 0 1 55 52
50fe6018 Y--P--- 134361 cortex - 513dd5d0 0 1 997 1420
50fe6618 Y--P--- 134362 cortex - 51a72d08 0 1 72 56
50fe6c18 Y--P--- 134363 cortex - 51da55e8 0 1 209 54334
50fe7218 Y--P--- 134364 cortex - 5422fbb0 0 1 188 176
50fe7818 Y--P--- 134365 cortex - 513dd620 0 1 0 0
50fe7e18 Y--P--- 134366 cortex - 51a72d58 0 1 0 0
50fe8418 Y--P--- 134367 cortex - 51da5740 0 1 2 0
50fe8a18 Y--P--- 134368 cortex - 513dd670 0 1 577 0
50fe9018 Y--P--- 134369 cortex - 51b1fbd0 0 1 0 0
50fe9618 Y--P--- 134370 cortex - 51e2f4f8 0 1 0 0
50fe9c18 Y--P--- 134371 cortex - 56ee79b8 0 1 0 0
50fea218 Y--P--- 134372 cortex - 51d0d268 0 1 155 0
50fea818 Y--P--- 134373 cortex - 5173e458 0 1 0 0
50feae18 Y--P--- 134374 cortex - 53273ed0 0 1 0 0
50feb418 Y--P--- 134375 cortex - 51284448 0 1 887 104
50feba18 Y--P--- 134376 cortex - 519ebcf8 0 1 519 2993
50fec018 Y--P--- 134377 cortex - 51d0d930 0 1 1593 3371
50fec618 Y--P--- 134378 cortex - 5574ebd0 0 1 11756 25520
50fecc18 Y--P--- 134379 cortex - 512a2b48 0 1 144395 271104
50fed818 Y--P--- 409233 cortex - 57aed2e8 0 1 0 0
50fede18 Y--P--- 409109 nauda - 563e2c00 0 2 20 37
50fee418 Y--P--- 409114 nauda - 57b5f1f8 0 2 9 3
50feea18 Y--P--- 134358 cortex - 51d0dce8 0 2 0 0
50fef018 Y--P--- 134385 cortex - 51d0dd38 0 2 35 0
50fef618 Y--P--- 134386 cortex - 53212ee8 0 2 684 255
50fefc18 Y--P--- 409078 nauda - 578a6478 0 1 17 34
50ff0218 Y--P--- 134388 cortex - 531f3d60 0 1 0 0
50ff0818 Y--P--- 134389 cortex - 512a2ea0 0 1 0 0
50ff0e18 Y--P--- 134390 cortex - 51a0c3a8 0 1 0 0
50ff1418 Y--P--- 134391 cortex - 51d0dd88 0 1 147 0
50ff1a18 Y--P--- 134384 cortex - 512a2ef0 0 1 0 0
50ff2018 Y--P--- 134393 cortex - 51b4c788 0 1 0 0
50ff2618 Y--P--- 134394 cortex - 51ea5fc8 0 1 484 18319
50ff2c18 Y--P--- 134395 cortex - 57252220 0 1 383 18474
50ff3218 Y--P--- 134396 cortex - 512c08d8 0 1 415 18340
50ff3818 Y--P--- 134397 cortex - 51b4ce70 0 1 524 18811
50ff3e18 Y--P--- 134398 cortex - 554fc210 0 1 469 18564
50ff4418 Y--P--- 134399 cortex - 512c0b38 0 1 0 8
50ff4a18 Y--P--- 134400 cortex - 51ab8420 0 1 0 1926
50ff5018 Y--P--- 134401 cortex - 512c0c90 0 1 0 0
50ff5618 Y--P--- 134402 cortex - 51b65228 0 1 166 5481
50ff5c18 Y--P--- 134403 cortex - 51f09218 0 1 192 5406
50ff6218 Y--P--- 134404 cortex - 512c0ce0 0 1 179 5393
50ff6818 Y--P--- 134405 cortex - 51b65278 0 1 131 5458
50ff6e18 Y--P--- 134406 cortex - 5173e068 0 1 0 0
50ff7418 Y--P--- 134407 cortex - 512c0d30 0 1 0 0
50ff7a18 Y--P--- 134408 cortex - 51ab8e88 0 1 0 0
50ff8018 Y--P--- 134409 cortex - 51dec530 0 1 1282 81
50ff8618 Y--P--- 134410 cortex - 53ffbbd8 0 1 0 0
50ff8c18 Y--P--- 134411 cortex - 512f0738 0 1 0 0
50ff9218 Y--P--- 134412 cortex - 51ab8ed8 0 1 0 0
50ff9818 Y--P--- 259029 cortex - 5559a520 0 1 904 0
50ff9e18 Y--P--- 134414 cortex - 52ec3fc8 0 1 23655 24572
50ffa418 Y--P--- 134416 cortex - 51d5b1d0 0 1 38 0
50ffaa18 Y--P--- 134417 cortex - 53fb0e78 0 1 26110 22749
50ffb018 Y--P--- 134418 cortex - 512f0af0 0 1 0 0
50ffb618 Y--P--- 134415 cortex - 51a2fb70 0 1 20018 11937
50ffbc18 Y--P--- 134419 cortex - 52ec3c88 0 1 2 5
50ffc218 Y--P--- 134420 cortex - 51b65d68 0 1 4 3
50ffc818 Y--P--- 134421 cortex - 51248580 0 1 0 0
50ffce18 Y--P--- 134422 cortex - 51a2fc10 0 1 1581 14533
50ffd418 Y--P--- 134423 cortex - 51d5b588 0 1 0 3
50ffda18 Y--P--- 134424 cortex - 512f0b90 0 1 0 35
50ffe018 Y--P--- 134426 cortex - 51e08070 0 1 210 18
50ffe618 Y--P--- 134425 cortex - 51adda68 0 1 31016 32103
50fff218 Y--P--- 408979 nauda - 57adf0b8 0 1 10227 17
50fffe18 Y--P--- 409103 nauda - 573e20b8 0 2 42631 22
51000418 Y--P--- 407220 cortex - 539f2890 0 1 11482 5669
51000a18 Y--P--- 409156 nauda - 57ba52e8 0 1 20 17
51001018 Y--P--- 409065 nauda - 57be1ba8 0 1 349 17
51001618 Y--P--- 409095 nauda - 5745b518 0 1 11 34
51002218 Y--P--- 405048 mami XI83 50723ed0 0 1 0 0
51002e18 Y--P--- 409135 nauda - 54172a70 0 1 7 0
51003a18 Y--P--- 409208 nauda - 59dfa250 0 2 26 20
51004018 Y--P--- 409107 limo XI86 5730a6f8 0 1 2 0
51004c18 Y--P--- 408971 nauda - 57fb76f8 0 2 15 41
51005218 Y--P--- 409177 nauda - 57af8018 0 2 6107 2
51005818 Y--P--- 405273 risp XY11 5574fed0 0 1 15118 0
51005e18 Y--P--- 405638 olie LPB2 53a077d8 0 1 4362 0
51007618 Y--P--- 217552 cortex - 53615c40 0 1 312 25
51008e18 Y--P--- 134428 cortex - 513bf3c8 0 1 1593 21081
51009418 Y--P--- 406025 olvo DPE3 52f99a30 0 1 29 0
51009a18 Y--P--- 259032 cortex - 531f3fc8 0 1 0 0
5100a618 Y--P--- 405120 aner XI58 51d78818 0 1 17 0
5100ac18 Y--P--- 409182 cortex - 57be12e8 0 1 0 0
5100b218 Y--P--- 407188 vlbo XC44 58453318 0 1 27606 0
5100b818 Y--P--- 405182 aldz XN10 53903178 0 1 15112 0
5100be18 Y--P--- 409116 cortex - 59dfa7a8 0 1 204 0
5100c418 Y--P--- 407318 sami XC20 59dfacf8 0 1 4611 0
5100ca18 Y--P--- 300268 nauda - 5432bb08 0 2 0 0
5100d018 Y--P--- 408733 xbank - 563e2660 0 1 46 0
5100e218 Y--P--- 408963 nauda - 563e25c0 0 1 3345 51
53d75018 Y--P--- 134427 cortex - 519eb2a0 0 1 14 0
53d76e18 Y--P--- 409229 cortex - 563e2610 0 1 1 0
53d77418 Y--P--- 405322 alkr XI80 52f46a30 0 1 23 0
53d77a18 Y--P--- 409188 nauda - 5788c248 0 1 6 0
53d78018 Y--P--- 409086 nauda - 577fd478 0 1 2305 0
53d79e18 Y--P--- 409134 nauda - 56bb1160 0 2 41 22
53d7aa18 Y--P--- 405311 jugr REA8 5150c7c0 0 1 13917 0
53d7b618 Y--P--- 407230 cortex - 52f5abd0 0 1 11353 5814
53d7bc18 Y--P--- 409199 nauda - 53ec5b10 0 2 5 1
53d7c818 Y--P--- 409141 nauda - 57194838 0 1 86 17
53d7ce18 Y--P--- 409219 nauda - 577a1e78 0 1 7 17
53d7d418 Y--P--- 409119 cortex - 57f1acf8 0 1 159 0
53d7da18 Y--P--- 409068 nauda - 535e73d8 0 2 11 37
53d7e618 Y--P--- 134392 cortex - 51e086a8 0 1 711 0
53d7ec18 Y--P--- 409048 cortex - 5a65d0c8 0 1 21 19
53d7f218 Y--P--- 409162 nauda - 578a64c8 0 2 11 2
53d7f818 Y--P--- 408842 xbank - 5799c838 0 1 48 0
53d7fe18 Y--P--- 409176 nauda - 53b29160 0 2 24486 1
53d80418 Y--P--- 409209 nauda - 54f8c608 0 2 7 2
53d80a18 Y--P--- 409113 nauda - 54f8c838 0 1 1 0
53d81018 Y--P--- 409012 nauda - 572917b0 0 2 9 20
53d81618 Y--P--- 405116 pakr EOD-GROU 51e2f120 0 1 212 0
53d81c18 Y--P--- 409096 nauda - 51ea5250 0 2 42610 6
53d82e18 Y--P--- 409187 nauda - 568f2c00 0 2 7 5
53d84018 Y--P--- 409046 cortex - 52914868 0 1 38 0
53d84618 Y--P--- 408895 cortex - 573fd6f8 0 1 1 1
53d85218 Y--P--- 409193 nauda - 548dcca8 0 2 7 2
53d85818 Y--P--- 408845 nauda - 57a8b658 0 1 1111 17
53d87018 Y--P--- 406238 inaj XI70 53db6a80 0 1 156056 1
53d87618 Y--P--- 409203 nauda - 53171b08 0 1 1235 0
53d87c18 Y--P--- 409045 nauda - 5432b9c8 0 1 1336 34
53d88218 Y--P--- 409178 nauda - 55b51668 0 1 8 17
53d88e18 Y--P--- 409106 nauda - 5788cb08 0 1 33 0
53d8a018 Y--P--- 409053 nauda - 54126a38 0 1 6210 34
53d8a618 Y--P--- 408734 xbank - 59dfa160 0 1 0 0
53d8ac18 Y--P--- 409057 nauda - 53171c48 0 1 420 34
53d8b218 Y--P--- 397444 gemu XI1C 5888c998 0 1 54 0
53d8b818 Y--P--- 409175 nauda - 54f8ce50 0 1 2 0
53d8be18 Y--P--- 409197 nauda - 59e84160 0 1 1 0
53d8d018 Y--P--- 400175 techdept XI1F 53fa0928 0 1 60 0
53d8d618 Y--P--- 353189 cortex - 5198f918 0 2 59220 28458
53d8ee18 Y--P--- 409202 nauda - 57b477e8 0 1 15 17
53d8f418 Y--P--- 409221 nauda - 57aed428 0 2 14 3
53d90618 Y--P--- 408997 nauda - 57de9018 0 2 21 25
53d91218 Y--P--- 409098 nauda - 5718fa68 0 2 9 19
53d91e18 Y--P--- 406220 vagl VPA6 54b1bd20 0 1 7610 0
53d92418 Y--P--- 409099 nauda - 57ea4bf8 0 1 14 17
53d92a18 Y--P--- 409136 cortex - 54e4c838 0 1 0 0
 
53d93018 Y--P--- 408956 xbank - 535e7978 0 1 0 0
53d94218 Y--P--- 409225 nauda - 568f2f48 0 1 68 0
53d94818 Y--P--- 409146 cortex - 577418d8 0 1 8 0
53d95418 Y--P--- 409165 nauda - 57aaaa68 0 1 3 0
53d95a18 Y--P--- 407217 cortex - 543ba610 0 1 11595 5415
53d96018 ---P--M 375021 informix 1 0 0 0 0 0
53d96c18 Y--P--- 405179 bask VMV5 51284fc8 0 1 8540 0
53d97218 Y--P--- 407223 cortex - 5373c650 0 1 11215 5509
53d98418 Y--P--- 408987 nauda - 57f1c978 0 1 19 51
288 active, 384 total, 330 maximum concurrent

Profile
dskreads pagreads bufreads %cached dskwrits pagwrits bufwrits %cached
11461126 17511130 515693227 97.78 3908113 8321507 22292721 82.47

isamtot open start read write rewrite delete commit rollbk
407500317 4995169 28258938 246601074 4853651 1621826 869580 496864 22

gp_read gp_write gp_rewrt gp_del gp_alloc gp_free gp_curs
0 0 0 0 0 0 0

ovlock ovuserthread ovbuff usercpu syscpu numckpts flushes
0 0 0 4849.57 1130.66 2917 7849

bufwaits lokwaits lockreqs deadlks dltouts ckpwaits compress seqscans
1732448 386 1047483626 0 0 6830 166462 382788

ixda-RA idx-RA da-RA RA-pgsused lchwaits
814045 113610 7255818 8181535 622729


I gave you full information! :)
 
ok, so you have way too many LRUS.

As a dumb rule you normally set CLEANERS and LRUS to the number of dbspaces. So i suggest to start with the above mentioned changes.
 
For this bank seste we have to use full checkpoints,not fuzzy.
 
concerning the onstat-output:

- there is a lot of bufwaits, this should improve increasing the buffers
- same goes for read-cache-hit is 97.78%, which is ok
- same goes for write-cache-hit is 82.47%, this could be better. we will see ...

another point is the seqscans... how long did your server collect these statistics? do you clean it regularly by onstat -z or is it reset on database reboot? do you reboot the database on a regular basis, e.g. for backup?

seqscans is the number of selects against database-tables, which must scan a whole table without use of an index. so this could be a hint to non optimal query-code.

you can find out by issuing the following command:

database sysmaster;
set isolation to dirty read;

Code:
SELECT  dbsname[1,18],
        tabname[1,18],
        seqscans,
        nrows,
        npused
FROM    sysptprof, sysptnhdr
WHERE   sysptprof.partnum = sysptnhdr.partnum
AND     tabname[1,3] != "sys"
AND     dbsname[1,3] != "sys"
AND     seqscans > 0
AND     nrows > 1000
ORDER BY 1 asc, seqscans desc, seqscans desc;
 
The output from select is:

dbsname tabname seqscans nrows npused
karte order_det 422 778617 33568
karte form 36 1071888 8451
karte xml_data 15 114281 75446
karte user 13 2023 83
karte ligum_det 2 178987 25325
lsb hostfeecode 4329 4216 19
lsb numdescr 1391 4427 54
lsb crddet 655 542005 42688
lsb grpperm 86 2722 23
lsb descr 31 5734 47
lsb crdbtch 20 55308 661
lsb chargelog 4 169406 17259
lsb chargedata 3 28260 752
lsb feepkgfee 2 4211 222
lsb tlog 2 3429500 1250000
lsb atmrpln 2 5274 129
lsb upload 2 957042 55950
lsb custdet 1 276880 28721

and statistic runs time after time, database is restarted after 60 days
 
A lot of the I/O is going to :

14 /dev/rtlogidxa 2010132 521059 1489073 27.8

Where is TLOG table's indexes.The table is 4Gb big. mayby we can recreate indexes ?
 
by statistics i meant the output of onstat, not the table statistics created by "update statistics". this is a point i would refer to later.

concerning the select of seqscans you could have a look at then order_det table, which has >700000 rows and was scanned 422 record by record. That must not be problem and can be normal, but you could have a look if an index would make sense. (this is not our main topic atm)

Maybe you could create a schedule which runs "onstat >> /tmp/onstat.lst" daily, so the statistics are collected, and a "onstat -z" afterwards to reset them to zero, so we have something to monitor.

i did not understand your statement concerning the fuzzy-checkpoints. what is the problem with these? are you not allowed to use them? for what reason?

 
ok, seems we have quite some work to do here :)

lets have look at your table-extents...

Code:
database sysmaster;
select tabname Table, count(*) Fragments, sum(size) size
from sysextents
where dbsname = 'yourdatabasename'
group by tabname
order by 2 DESC, 1;

ibm recommends 2 extents for a table ...

concerning the i/o :

chunk-io : onstat -g iof
vp-io: onstat -g iov

 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top