Corrupt filesystems in AIX 5.2

[michaelvv]

I have two L-pars on a p650, both running AIX 5.2 (obviously). For no apparent reason, most of the filesystems become corrupt within a short time of fixing them. Has anybody else encountered this problem? Does anybody have a solution?

Thanks,
Mike

 

[ogniemi]

what does it "mean become corrupt within a short time of fixing them"?

what the filesystems are you talking about...belonging to rootvg or to the other VG located for example on some external storage....

(maybe you have located them mirrored LVs and one of the copies of each LVs where not synced (stale PPs) due to the disks were not accessible for a short period of time..., maybe quorum was lost for a shor period of time and then you lost your VG....hard to say without additional details.. )

 

[michaelvv]

First, thanks for the reply.
Here is the scenario:

df -k shows the filesystem (JFS2, just over 100 GB on EMC storage) 100% full.
du -sk shows 60 GB in use.

I unmount the filesystem, run fsck, it gets repaired and the two numbers match. By next day I have the same problem!!

 

[mrberry]

I have seen problems like this before where the is a discrepency between df and du.

This has occured where a process has been killed off (with a -9) that was writing to a file (usually some sort of log file). When the process dies the open file appears to be still being written to, but is does not register with a du.

The only solution I found was to unmount and fsck.

Not sure if that is your problem, but the result sounds silimar. Any processes being killed during the night?

 

[michaelvv]

Not that I am aware of. What bothers me is that it happens to just about every filesystem (did not look at any of the 'rootvg' filesystems however).

 

[MoshiachNow]

HI,

Please post the output of the following:

1.errpt |head -40
(must be for the time PRIOR to the FS "corruption")
2.fsck prompts that runs on the problematic FS.
3.df -k (taken immediately AFTER the FS becomes 100% full)

Thanks

"Long live king Moshiach !"

http://www.7for70.com/

 

[kh1978]

Hello,
wich ML have you installed. In AIX 5.2 ml1 there was a many problems with jfs2 and fsck/defragfs. This problems was fixed in ml2.

 

[RodKnowlton]

This symptom (df disagrees with du) also happens when a process hold an open filehandle for a file that has been deleted by another process.

Could there be a skulker type process that's deleting files that other processes have open?



Rod Knowlton
IBM Certified Advanced Technical Expert pSeries and AIX 5L

 

[michaelvv]

Wow I'm overwhelmed by the attention! Unfortunately (not for me), I'm away from the office for a few days, I'll be back in the office on Wednesday. I'll continue with this mistery then. Thanks guys,
Mike

 

[dblair]

This behavior is not restricted to AIX..