fsck stuck in pahse 1

[michaelvv]

I am running fsck on AIX 4.3.3 on a 7026-6H1. The filesystem is on a single EMC drive, 64 GB in size. The disk is between 99 and 100% busy, it has been running for 3 1/2 hours, and is still in phase 1. There are no errors displayed, or by runnig errpt. Does anyone have any ideas?

 

[samjoo]

Probably! your filesystem seems to be have something wrong but fsck have many option. try to run with "fsck -y".

 

[MoshiachNow]

Are you sure you have unmounted the volume prior to fsck ?
Please post the output from fsck promts and "errpt|head -20".
Thanks

Long live king Moshiach !

http://www.7for70.com/

 

[michaelvv]

I am positive the filesystem was unmounted. I had to stop fsck and mount it again. There was absolutely no output from fsck beyond the Phase 1 header. There were massages displayed by errpt, but totally unrelated to the drive where this filesystem resides. The next window to try this again is after 5:30 PM (eastern time).

 

[11011110000]

Corrupted supberblock?

 

[michaelvv]

Both copies? Why no error message(s)? Why is the disk busy 99-100% of the time (only while fsck is running)?

 

[MoshiachNow]

If your drive is a RAID - possibly you have some issue with your RAID drivers drivers settings - queue_depth,etc.

I onece have spent several months having RAID volumes beeing 100% busy,till we have decreased the queu_depth.
It really looks on the level of the OS versus your disk driver,more then a FS issue.
Also possibly bad vmtune params are involved.

Please post the results of :

1.lsattr -El hdiskX
2./usr/samples/kernel/vmtune

Long live king Moshiach !

http://www.7for70.com/

 

[michaelvv]

I hope this is readable, thanks for the effort!

# nydrs2011:/home/mikev <1040> lsattr -El hdiskpower8
pvid_takeover yes Takeover PVIDs from hdisks True
pvid 000dfc8a7738270c0000000000000000 Physical volume identifier False
queue_depth 8 Queue DEPTH True
q_type simple Queue TYPE False
q_err no Use QERR bit True
clr_q yes Clear Queue (RS/6000) True
rw_timeout 40 READ/WRITE time out True
start_timeout 180 START unit time out True
reassign_to 120 REASSIGN time out value True
scsi_id 0x7a1013 SCSI ID False
lun_id 0x3d000000000000 LUN ID False
ww_name 0x5006048accef40df World Wide Name False
reserve_lock yes Reserve device on open True
location Location True
unit_id 'Array Serial Number-FLU' or WWN False
max_transfer 0x40000 Maximum transfer size True
max_coalesce 0x20000 Maximum coalesce size True
target_id SCSI/LOOP/ALPA ID of SP False
parent Real SP or ATF Parent Device False
FLU Flare Logical Unit Number (FLU) False
cgname Consistency Group Name False

# nydrs2011:/home/mikev <1042> /usr/samples/kernel/vmtune
vmtune: current values:
-p -P -r -R -f -F -N -W
minperm maxperm minpgahead maxpgahead minfree maxfree pd_npages maxrandwrt
209508 838032 2 8 120 128 524288 0

-M -w -k -c -b -B -u -l -d
maxpin npswarn npskill numclust numfsbufs hd_pbuf_cnt lvm_bufcnt lrubucket defps
838852 12288 3072 1 93 305 9 131072 1

-s -n -S -L -g -h
sync_release_ilock nokilluid v_pinshm lgpg_regions lgpg_size strict_maxperm
0 0 0 0 0 0

-t
maxclient
838032

PTA balance threshold percentage = 50.0%

number of valid memory pages = 1048565 maxperm=79.9% of real memory
maximum pinable=80.0% of real memory minperm=20.0% of real memory
number of file memory pages = 149491 numperm=14.3% of real memory

number of compressed memory pages = 0 compressed=0.0% of real memory
number of client memory pages = 0 numclient=0.0% of real memory
# of remote pgs sched-pageout = 0 maxclient=79.9% of real memory

 

[MoshiachNow]

vmtune looks reasonable.
try setting the queue_depth to 1 (or at least 3) and run fsck again.
It maybe slower now ,but if works - will point us in the right direction ...

Long live king Moshiach !

http://www.7for70.com/

 

[michaelvv]

Cannot unmount FS before 5:30 PM (unless it crashes). I can do that tomorrow, unless they send us home early for Thanksgiving, in which case it's going to be Monday. Thanks a lot for the effort.