Client backs up full everytime

[ravashaak]

Hi,

I have a client running redhat 8 and networker 6.12, which backs up as a full backup, even when it is supposed to be an incremental. I've performed nsrck -L6 against the client, I've reinstalled the networker client software, and I've even went so far as to delete the client and its index directory in /nsr/index. I've rebuilt the client resource, but still to no avail.

In my savegroup completion reports, each filesystem reports as follows:

flanker:/: No full backups of this save set were found
in the media database; performing a full backup

In addition to the report above, there are some other symptoms that may be important. For one, when I attempt to browse save sets through nwadmin (for this client), I see no new save sets, even after a supposed "full" backup. The client also reports as having backed up its index with the size of 1 KB, 1 file. Another odd thing I have noticed is that in the nwadmin gui, in the "sessions" field, this client has an oddity during its backup. Most clients will display something like the following during backups:

nova: /export/home saving to pool 'DLT01' (010112) 23 GB

This problem client displays its backup status in the session window much like above, with one exception: in place of the client name, is the fqdn of our backup server.

Could this be media database corruption? Some name resolution issue? Will Batman escape and leave the whining Robin to boil in the acid vat? Any ideas, suggestions, etc are most welcome.

FYI: server runs Solaris 8 and Networker 6.12

 

[Chapter11]

I would first check name resolution.

 

[wallace88]

You renamed this client at one point in time?

On the backup server, run:
mminfo -avot -r client,clientid | sort | uniq | grep -i flanker

How many different client id's show up for this client?

 

[605]

If anything else works fine, a corrupted media db is obvious.

 

[ravashaak]

Wallace88,

If the client was renamed, it was done prior to my arrival on this job. What is more likely is that the system name was retired, then re-used for this client system. It's possible that another backup client existed at one time with the name "flanker".

Output for your mminfo command resulted in a single client ID.

 

[wallace88]

Since the client-clientid is unique, it's unlikely a client id issue.

I concur with Chapter11. Test this by populating the host files on the client and backup server with the client's and server's:

ip hostname FQDN

Make sure that the nsswitch.conf looks at host file first. Then test backups.

 

[ravashaak]

Thanks for the input so far everyone!

Here's where things stand currently. I have tested forward and reverse lookups from the client and the server. I tested against hostname, fqdn, and ip. I probably won't be able to run the host files test until after the weekend. But once I do, I'll post another update.

As far as media database corruption is concerned: What would be the best way to test for media database corruption? I did a grep on all my daemon.log files. I did have one WISS error approximately one month ago. Is this indicative of media database corruption? Is the fact that I received no more such errors indicative of the problem having vanished? And if corruption has indeed occured, how exactly should I resolve it?

Also, on a slight tangent: What sorts of words or strings should I grep for when examining my daemon.log files? I looked for "error" and "WISS". And are there any other logs I should examine? Sorry to rapid-fire so many questions, but I'll reward the answers.

 

[LGTObug]

The WISS error is from the media database.
To resolve the problem to following:

1.) Stop Networker Services
2.) delete all files from the /nsr/tmp or C:\Program Files\nsr\tmp Directory
3.) in /nsr/mm delete the two files: nsrim.pv & cmprssd
4.) In the /nsr/mm/mmvolume6 Directory keep the files: "VolHdr, vol.0, ss.0 and clients.0." and delete all others.
5.) Start Networker Services

After you started the services Networker run nsrim -X

that's it. Your media-database should be repaired.

When you take a look at the complete error message it guides you to an media-database error:

"WISS error: Unable to mount /nsr/mm/mmvolume6: bad database header"

regard's

 

[wallace88]

Sounds like name resolution is ok so far.

Your method of checking the daemon.log is fine... other method is to look though every line.

You usually see WISS errors when NetWorker starts up.

Check the following: on the server, go to /nsr/index/flanker. Look at the directory structure. Is there only one subdirectory called db6? Or is there something like /nsr/index/flanker/FQDN/flanker/FQDN...?

 

[ravashaak]

My apologies for the delayed response. It's been a very hectic holiday season for me thus far. And thanks to all for the ideas and help so far.

Wallace88: there is only a db6 subdirectory
in /nsr/index/flanker

I'm still performing some testing on the client and server to clarify this problem. Scheduling downtime has been very problematic lately. I'll post further updates as soon as I can.

 

[ravashaak]

Ok, I've found some time to perform a few more tests.

LGTObug: I've followed the procedure you provided for repairing media database corruption. Although it didn't help with this particular issue, it's something I needed to be able to do. Thanks.

I've also reinstalled the client on the client (hehe). I've rebooted the client. I've stopped the networker daemons on the server, completely removed all references to the client and its entire /index directory structure. I've restarted the networker daemons on the server and rebuilt the client resource from scratch. The problem still persists.

I have managed to glean a few new facts: The beginning of these failures coincided precisely with some library patching on the client. This patching was not entirely successful apparently. Upon closer examination, up2date was no longer functioning on the client.

A fellow admin made some repairs to the client and we'd hoped this would fix things. Apparently, the fixes did not work to resolve this issue. I'm putting the ball back in his court, since it looks like it's a client-side issue that's not directly networker-related. I'm still open to any ideas or additional suggestions from you guys. And I'll award some points as soon as I know I am done with this thread. Thanks again and HAPPY NEW YEAR!!!!