New script to total nitely backup totals 1

[hermxl1]

Hey, I just figured out how to total up the number of GB I'm backing up nightly all in one simple script. I realize there many ways of doing this already but most require you to export data to something else to add up all the savesets. Plus, I'm not big on UNIX scripting so this was a significant accomplishment for me.

Anyone interested, Please take a look:

hermxl1

#!/bin/ksh
SEARCHDATE=$1

TOTAL=`mminfo -av -r 'savetime,totalsize' | grep $SEARCHDATE | awk '{sum += $2} END {print sum}'`

let TOTALGB=$TOTAL/1024/1024/1024
echo $TOTAL
echo On $SEARCHDATE $TOTALGB GB of data was backed up.

 

[eskolnik]

Does this script match the output of mminfo -X, because when I run it its off by about 10 GB


chi-backup# mminfo -X
Save Set Type Summary:
1318 save sets 3106252 files 2357/ss 498 GB 377 MB/ss 160 KB/file
314 full 464423 files 1479/ss 85 GB 271 MB/ss 183 KB/file
652 incremental 852484 files 1307/ss 303 GB 465 MB/ss 356 KB/file
245 intermediate 5361 files 22/ss 612 MB 2500 KB/ss 114 KB/file
107 ad-hoc 1783984 files 16672/ss 108 GB 1015 MB/ss 61 KB/file
0 archive
0 migration
192 empty
258 purged 7009 files 27/ss 2365 MB 9170 KB/ss 337 KB/file
260 index 7057 files 27/ss 2366 MB 9100 KB/ss 335 KB/file
0 incomplete

96% of full files per incremental
100% of files are on-line

Recent Save Set Usage Summary:
582 today 1231147 files 2115/ss 234 GB 403 MB/ss 190 KB/file
736 2/18/04 1875105 files 2547/ss 263 GB 357 MB/ss 140 KB/file


ksh sizing.ksh '02/18/04'
DISPLAY = 170.200.11.100:0
269668257680
On 02/18/04 251 GB of data was backed up



Ed Skolnik

 

[hermxl1]

Well,

My script grabs all of the saveset sizes for a given day in bytes using mminfo -av -r 'savetime,totalsize(20)'. If you add up all the bytes and then convert to gigabytes you get the total amount of GB's backed up that day.

My mminfo -X number doesn't match either, so I did the math by hand. ( and with Excel) Using calc and Excel I got the same answer as the script which is approx. 10 to 12GB off from what mminfo -X reports. The only thing I can figure is that Legato is not doing all of its calculations in bytes and then converting to GB's. If its start in MB's or KB's then there could be rounding errors with the conversion to GB's, but 10 to 12GB is a lot of rounding error.

Does anyone else have a clue with what's going on here? I feel pretty confident that the results of the script are correct. Any other thoughts?

Hermxl1

 

[eskolnik]

Well I got to the bottom of this, the mminfo -X shows the actual amount of data sent to the backup media and the
" mminfo -av -r 'savetime,totalsize' " tells you the actual amount of data BEFORE compression from the client side.

So both numbers are correct.

I actully found this on the Legato Web site!!!!!!!!


Solution Title: Questions about mminfo -X
Solution ID: legato19292

Here is the solution:
1. To calculate, roughly, how many bytes of data to be backup:

# savegrp -n -E -l full -c <client> -G <group>

For example, place a roughly 4.5 megabyte log file in D:\testlog

2. Run the above command

C:\>savegrp -n -E -l full -c liszt -G Default
liszt.legato.com:\testlog 5 records 1 KB header 4492 KB data
liszt.legato.com:\testlog 4493 KB estimated
liszt.legato.com:index:ef8cab46-00000004-3cde1b28-3cde1b26-00020000-89456633 3 records 1 KB header 2 KB data
liszt.legato.com:index:ef8cab46-00000004-3cde1b28-3cde1b26-00020000-89456633 2 KB estimated

As you can see , it shows 4493 KB

3. Backup the file with NT "Compression" Directives turn on.

4. Run mminfo -X command

C:\>mminfo -X -q "name=D:\testlog"
Save Set Type Summary:
1 save sets 7 files 7/ss 918 KB 918 KB/ss 131 KB/file
1 full 7 files 7/ss 918 KB 918 KB/ss 131 KB/file
0 incremental
0 intermediate
0 ad-hoc
0 archive
0 migration
0 empty
0 purged
0 index
0 incomplete

Recent Save Set Usage Summary:
1 today 7 files 7/ss 918 KB 918 KB/ss 131 KB/file
0 5/28/02
0 5/27/02
0 5/26/02
0 5/25/02
0 5/24/02
0 5/23/02



The 918 KB is the actual bytes stored in the "TAPE" volume.
Because the saveset was compressed.

So in summary
=============

1. Use savegrp -n -E to look at how much data reside in OS that will be backup.

2. The totalsize in mminfo -X corresponds to the number in which the size of the saveset are KEPT/occupy in the TAPE volume.


Here is the problem or goal:
mminfo -X output

Calculate bytes to be backed up

Use savegrp -n -E -l to calculate data bytes to back up


Problem Environment:
NetWorker for UNIX

mminfo

savegrp


Ed Skolnik

 

[eskolnik]

RFE LGtpa30967 was already created to have a new switch that shows the original size of the data and to correct the man pages to reword how -X displays its information.

I have added you as an interested customer on this RFE.


Ed Skolnik

 

[hermxl1]

Gretat work Ed!!!!


I was sure that there was some byte to gigabyte conversion error in the mminfo -X. Glad to see there wasn't. Now I can continue using my script with confidence.

Thanks,

Hermxl1

 

[hermxl1]

Hey Ed,

New Issue. Turns out the numbers for mminfo -X does not match the Networker Management Console monthly report numbers. I ran mminfo -X -q 'savetime>02/01/04,savetime<03/01/04' and got a different amount of data backed up as well as a different number of save sets done. At first I thought it was just the date being wrong but that wasn't it.

The Console reports 54.5TB for and mminfo reports 47TB. Any ideas on who's right???

Herm

 

[eskolnik]

If you look at the man page for mminfo the query spec "-q" isn't not supported with the -X option, so who knows what you would get.

Maintenance Commands MMINFO(1m)

NAME
mminfo - NetWorker media database reporting command

SYNOPSIS
mminfo [ -avV ] [ -o order ] [ -s server ] [ report ]
[ query ] [ volname... ]

< report >: [ -m | -p | -B | -S | -X | -r

reportspec ]
< query >: [ -c client ] [ -N name ] [ -t time ] [ -q
queryspec ]

Ed Skolnik

 

[Nerom]

Hello!

I also have problems with the results of the mminfo and Management Console. I run the next command:

mminfo -a -r
name,ssid,client,savetime,type,ssflags,volume -q ssid=1550964
name ssid client date type ssflags
volume
oposARCH200408091846 1550964 icm1000x.icm.es 09/08/2004 adv_file
vF ICM1000HD.001
oposARCH200408091846 1550964 icm1000x.icm.es 09/08/2004 adv_file
vF ICM1000HD.001.RO

In the Management Console 3.1: [Networker Backup Statistics-->SaveSet Details] The output of the same query is nothing.

Anybody knows why?

Thanks for all.

 

[ptwork]

Herm, you may want to look at your figures in CSV to see if the numbers come closer. Rounding with that amount of data could be significant (from the release suppliment)...

Console provides numeric formatters that visually simplify the large numbers in
reports, dividing them by a scale. For example, 92127177708 (bytes) is rendered as 90
(GB). All large numbers appearing in reports are rounded off in this manner.
If column data appears in bytes, kilobytes, or megabytes, and the value is less than 1,
it appears as 0.
Note: To see the actual, unrounded figures for this data, you can export the report into
CSV (comma separated value) format.

 

[ptwork]

Nerom,
Not sure how you used Console to do the same command, as it doesn't have the ability to accept ssid as your query used. Plus, those savesets look a bit wacky, and you didn't collect info like size, duration, level, so I'm not sure what/how you were trying to compare. Usually, when I try to get a query right, I use one of the drill downs and start with a client or server I am interested in. Are you getting ANY information?

If you are, it is possible that the constraints you choose were to tight and eliminated everything.

 

[Nerom]

Hi ptwork,

When i run thequery in MC I setup the server name and client, the savetime and the name of the saveset. If i know the ssdi i know the name of the savetset too. The query is something like this:

server: icm1000p
client: icm1000x
time from: 09/08/2004 0:00
time to: 09/08/2004 23:59
SaveSet: oposARCH200408091846

I think yhat this query is identical that the mminfo query. My propose is to be sure that the queries in MC are correct and shows a real information.

Thanks,

 

[ptwork]

Nerom, as I stated, I'd not use all those parameters at first. Start with just the server, do you get data? Add the client, do you get data? This will help you determine where the problem lies. This could also help determine if Console is having trouble with the date format, perhaps looking for a Sept 8 time frame instead of Aug 9? Unless your systems dates are messed up

 

[Nerom]

ptwork, It is not a date problem. We made a query without date and the Saveset that i looking for does not appear. I checked differents queries (ie: only server and client, only server, only client, and the result is the same). I thing that MC does not work correctly, a bug?

Thanks,

 

[goony]

There's also the old "how are you counting" problem... e.g. is a "gigabyte" 10^9 bytes or 2^30 bytes?

10^9 = 1,000,000,000 bytes
2^30 = 1,073,741,824 bytes

This can also possibly account for some differing results.