CMS Backup Failure

[cmsterm]

The logs I'm being referred to aren't helping me isolate what the issue is, maybe someone here will understand? Has this happened to anyone, what fixed it for you, or what should I do?


At the end of the backup through Terminal Emulator it stated:
.................
765729 blocks
Request failed. See /cms/install/logdir/backup.log for more information.


The logs in that directory only showed:

===== Begin backup Tue Oct 9 20:39:15 CDT 2018

Converter started Tue Oct 9 20:39:16 CDT 2018
Converter completed successfully Tue Oct 9 20:39:16 CDT 2018
771115 blocks

backup failed
/tmp/cpio.inlist23067 contains the list of files to back up
/nfsbu/CMSADM-r17jb.a-181009203928-df07vmi4008 has been removed

===== Backup failed Tue Oct 9 21:27:28 CDT 2018


===== Begin backup Wed Oct 10 05:31:44 CDT 2018

Converter started Wed Oct 10 05:31:44 CDT 2018
Converter completed successfully Wed Oct 10 05:31:44 CDT 2018
765729 blocks

backup failed
/tmp/cpio.inlist878 contains the list of files to back up
/nfsbu/CMSADM-r17jb.a-181010053236-df07vmi4008 has been removed

===== Backup failed Wed Oct 10 06:16:23 CDT 2018


How do I view the cpio.inlist878, or is that necessary? I read an example of listing the files but I'm not familiar with the cmd or if it would be useful in troubleshooting.


So I checked out the logs in /cms/maint/backup/back.log:

status: Backing up Historical data: wvector
(ACD = 1)
status: Updating backup history ...
state: 1
error:
status: Last backup finished 10/10/2018 03:29:04.
state: 0
/cms/install/bin/compress_backup successfully finished: Wed Oct 10 03:29:04 CDT 2018



Looking at the backup.success in /cms/main/backup it shows the correct timestamp but 0 KB.



Not sure if this was just the CMSADM backup for the UNIX OS backup, or both.

 

[kyle555]

I'd have to go thru the admin guide, but I think that 0 byte file is good. It's the old school unix way of letting you know something happened.

Without digging into it too deep, I'm thinking like the backup in terms of crunching the datasets was good but maybe getting them to their final destination wasn't. What release are you on? Tape? NFS share?

 

[cmsterm]

I believe we use tape because of the reference to cpio, it's possible though we use both because of the mount points I found. We have NetApp and that goes to tape so, tape? We use CMS R17

 

[joe2938]

Replace the tape and try a manual back up, this is what crashed our back ups in the past.

 

[kyle555]

Linux or Solaris? If it's an older Sun upgraded to R17, possibly tape. If newer, probably Linux. And the maintenance costs on Sun gear probably eclipse (pun intended) the cost of doing it on Linux, where you'd more likely use NFS.

Either way, the stuff under the hood is old school, so it's probably referencing tapes no matter what. AAM backups make references to tapes too despite no tape drive being used on the platform ever.

 

[cmsterm]

You both make very good points and I believe you're right on the money. I think our tapes were full at the time of the backup attempting to be ran. Finding it difficult to confirm but trying again soon!

 

[cmsterm]

It looks like all backups are successful, aside from the filesystem backup kicked off via the Avaya call management system under cmsadm. Unfortunately no error to follow as it got cancelled after taking too long...

 

[cmsterm]

Learned it's not tape, the filesystem is backed up to "other" where it's a nfs share

 

[kyle555]

do a df -h

https://downloads.avaya.com/css/P8/documents/101015305

I recall the first time I set it up it didn't persist through reboots

Have fun playing with Linux mount points!

 

[cmsterm]

Thank you for the resource! I will Am starting on page 148 about CMSADM backups.

df -h
the backup directory is only 11% used
The filesystem overall shows 79% used

 

[avayatech2018]

Following along for any useful info. We have CMS R17 and are trying to figure out why we can't do a CMSADM backup. It appears ours are actually backing up to USB thumbs on the front of the server. It appears the system is doing a daily maint backup. But when we try and do a CMSADM backup it tells us command not found.

40 years in Telecom. New to Avaya! Go easy.

 

[kyle555]

I don't do much CMS...you do the ADM backup thru cmssvc right? Go in the Linux CLI as root, type cmssvc, and there's a choice for the adm backup? are you doing su - root or just su root?

Does it say what command is not found?

 

[cmsterm]

Despite what df -h said, as well as what all other errors displayed and what technical documentation suggested, what resolved this for me was deleting a previous large backup.

 

[avayatech2018]

I'm uploading a screen shot of the error. Logging in as su. Let's see if the image loads.

40 years in Telecom. New to Avaya! Go easy.

 

[kyle555]

maybe su - instead of su.

The minus means you inherit the environment variables of the user you're su'ing to. If I just logged in as root and did 'useradd avayatech2018' and you have nothing in your path, you can't just use commands like ifconfig

whatever user you were before the su command, do a echo $PATH
That'll show ya what folders you can run commands from the CLI without explicitly stating the directory. Just like how in Windows, you have c:\windows\system32 in your path, so you can just run "cmd" and not have to type C:\Windows\System32\cmd.exe

It would appear that just su with no - is making your root shell have whatever environment variables the previous user account had, and if echo $PATH doesn't have /sbin or /user/sbin in there, you can't run those commands.

you can do a find / -name "ifconfig" to see where ifconfig is exactly - can't remember if it's /sbin or /usr/sbin, etc, but if you su - root I'm pretty sure it'll work.

 

[avayatech2018]

Thanks Kyle I'll give that a shot and let you know.

40 years in Telecom. New to Avaya! Go easy.

 

[avayatech2018]

Kyle I think you were spot on! I did the su - and it changed my directory structure and I did the cmsadm and received the correct menu items. I plan to perform the backup in the morning before things open. I will post back how it goes. Thanks again for you help!!!!!

40 years in Telecom. New to Avaya! Go easy.