Disk space issue

[bigdaddy01]

Hey guys I have a very strange disk space problem, any help on this is greatly appreciated.

Who ever set the orginal system up didn't knpw what they where doing and put /, /var, /usr, /opt, and export all in root, definatly a big no no. I am getting a increasing file which is pushing the disk space up yet can't find the offending file/s. I have checked logs and deleted/moved everything I can find that would make a differnce.

The first place i started is with oracle, but nothing of oracle is on the root partition. BSM is set up and being used i have removed all of the security audit logs which where taking up some space.

I am pulling my hair out (whats left) as i can't find whats pushing the space up?

help !

 

[KenCunningham]

First choice for possible problems are files in /var/adm such as wtmp, wtmpx, messages, sulog etc, which if you are happy to do so can be zeroed (cat /dev/null > filename) this saving some space. Is the increase a gradual process or does it tend to happen all at once? Would suggest, if you have an opportunity, that you should consider repartitioning the box, but realise it's not always easy in a production environment. Cheers.

 

[jad]

if you delete the above files and processes are still using them, then they will still exist on the disk for allocating storage etc but won't be anywhere on the disk.

zero'ing the files as KenCunningham said is much better.

 

[bigdaddy01]

It is slowly increasing up to 4 % every 24 hours, with no large file in site.. there is an nfs link on the box is it possible this could cause the problem ?
I have found alot of BSM auditing files in /var/audit which "touch" won't work on.. so i deleted the old ones, also I have counted up the root file system using a "du -sk *" there is no individual directory that is overly large, take a look below... is it possible df -k is reporting incorrectly?

The root size is 14.5g of which veritas says that 2.5 gig is free...

#_>du -sk *
1 DeadLetters
1 Mail
96 TT_DB
1 _-_No_PrGlOg_-_
1 bin
2 cdrom
1763 dev
83 devices
1 display.sh
9992 etc
1419366 export
0 home
47060 kernel
1 lib
8 lost+found
0 mailx
1 mnt
0 net
1 nsmail
744 nsr
792621 opt
19572 platform
218331457 proc
12780 sbin
0 start
1336 tmp
965677 usr
148299 var
0 vol
9 xfn
6195 u06

 

[jad]

can you give us a 'ps -ef' from your system?

 

[SamBones]

Try this. It isn't perfect, but it'll show files that are changing size. You'll need [tt]lsof[/tt] on your system for it to work...

#!/bin/ksh

export THIS=/tmp/growth.now
export PREV=/tmp/growth.prev

[[ -f ${PREV} ]] && rm -f ${PREV}

while ( /usr/bin/true )
do
    print "Sampling files at $(date)"

    lsof -s|grep VREG| awk '{print $9, $7}'|sort|uniq > ${THIS}

    [[ -f ${PREV} ]] && sdiff -s ${PREV} ${THIS} | awk '$1 == $4 {print $1 " went from " $2 " to " $5}'

    mv ${THIS} ${PREV}
    print

    sleep 60
done

Since it's using [tt]lsof[/tt] (list open files), it will only report those that were open for two consecutive samplings. But, it does run pretty quick since it's only getting open files and not scanning every file on the system.

The latest distribution of [tt]lsof[/tt] is available via anonymous ftp from the host vic.cc.purdue.edu. You'll find the lsof distribution in the [tt]pub/tools/unix/lsof[/tt] directory.

Hope this helps.

 

[bigdaddy01]

Hi Sam,

I can't configure the lsof for a 64 bit kernal...
the error message is
LSOF NEEDS TO BE CONFIGURED FOR A 64 BIT KERNEL, BUT |
! THE VERSION OF SUN C AVAILABLE DOESN'T SUPPORT THE !
! -xarch=v9 FLAG. LSOF WILL BE CONFIGURED FOR A 32 BIT !
! KERNEL.
Any Idea's how I can change this to 64 ?

cheers

 

[SamBones]

Try this site for a 64 bit Solaris 8 kernel.

http://ftp.u-strasbg.fr/securite/lsof/binaries/sun/solaris/8/

Hope this helps.