UC Module full - No obvious large files 1

[jwptrek]

My 11.1 UC module is showing over 90% disk usage. df-h shows that it's /dev/sda3, mounted at / that is full. I've done the standard find the nohup files but the only one is in ./opt/Avaya/oneXportal/openfire/bin/and shows a size of 0. I also get a number of errors when running the command find -name nohup.out. See below:
./opt/Avaya/oneXportal/openfire/bin/nohup.out
find: ‘./proc/32582’: No such file or directory
find: ‘./proc/32587’: No such file or directory
find: ‘./proc/32611’: No such file or directory
find: ‘./proc/32680’: No such file or directory
find: ‘./proc/32681’: No such file or directory
find: ‘./proc/32682’: No such file or directory
find: ‘./proc/32683’: No such file or directory
find: ‘./proc/32684’: No such file or directory

I got similar errors when running du -a | sort -n -r | head -n 5 to see where the usage is.
-bash-4.2# du -a |sort -n -r |head -n 10
du: cannot access ‘./proc/11257/task/11257/fd/4’: No such file or directory
du: cannot access ‘./proc/11257/task/11257/fdinfo/4’: No such file or directory
du: cannot access ‘./proc/11257/fd/4’: No such file or directory
du: cannot access ‘./proc/11257/fdinfo/4’: No such file or directory
du: cannot access ‘./proc/13270’: No such file or directory
du: cannot access ‘./proc/13271’: No such file or directory
du: cannot access ‘./proc/13272’: No such file or directory
du: cannot access ‘./proc/13691’: No such file or directory
du: cannot access ‘./proc/13692’: No such file or directory
du: cannot access ‘./proc/13693’: No such file or directory
12066208 .
4774100 ./var
3038520 ./opt
2853040 ./media
2825268 ./var/log
2825268 ./media/sda2
2673204 ./var/log/audit
2673204 ./media/sda2/audit
2182104 ./opt/Avaya
1910940 ./var/cache

It shows . as the largest folder but it's only showing about 12 MB. Last time I got the issue, somehow I did a series of reboots and all of a sudden I was down to 39% usage. I cannot replicate the issue. I did clear out logs from /opt/Avaya/apache-tomcat/logs to get down to 89% usage but there's still something big I cannot find. I am also concerned about things since I am getting the above errors. Upon doing some further searchung with the du commnand, I found /var/log/audit had a lot of files dating from after the last time I had a space issue so I cleared those out. Still, that's only about 2GB.

 

[derfloh]

I would use the following command:

du -sh /
It will show the folder and file size in a human readable size.

From there you can go deeper I.e.
du -sh /var
And so on.

 

[jwptrek]

I've tried that and gone through all the folders I see in /. I can only account for about 7GB of space when the system says / is using almost 17GB space. This makes me think there's either some hidden files that ls commands and du -sh doesn't see or a symbolic link being misinterpreted. Not sure what is next to do here.

 

[NYSTECH]

Similar to what derfloh said
du -hsx /* | sort -rh | head -10 will list the 10 largest directories. Then you can drill down into those. An example drill into var directory
du -hsx /var/* | sort -rh | head -10.

 

[jwptrek]

I did a variant of that but now it's seemingly pointless. I did some more searching online and found a manual for the UC module. It can be turned off by holding the "top" button (v1 of the UC module) or the button (v2 of the UC module) for 2 seconds. The LEDs then change and the unit shuts down. Once it's shut down, the same button will turn the unit back on. I shut down my UC via button and rebooted it. I then got an alert in system status that there was a BIOS error reported by the UC. I shut it down again with the button and booted it up again and it booted fine. Once I logged into the UC module, it showed it was back down to 39% disk usage. Therefore, it seems that rebooting via the button once or twice triggers some kind of automated cleanup. Does this tell anyone anything here?

 

[derfloh]

In Linux the finde are deleted but space is not released. Had that on the past. The reserved space becomes free after a reboot. It should work different in my opinion

 

[sizbut]

...or rebooting caused more log files, eventually passing some threshold which, as the OP suspects, triggered a housekeeping process.

...or restarting deleted numerous files from pending but uncompleted due to some error voicemail email messages.

...or...

 

[jwptrek]

All those theories seem possible. I agree it doesn't make sense that it takes a reboot for the reserved space to become free. The unit was rebooted after I deleted the log files I'd deleted and I did see the change there. I'd not deleted anything else yet so the unit was sitting at 89% usage until the reboot using the button. It seems like rebooting that way was what somehow (set a flag or whatever) triggered the housekeeping process.