Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations derfloh on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
Retention time for tapes 1
- Thread starter jl123
- Start date
-
1
- #2
I feel like responding with "why?"
TSM works quite differently from other backup products and while I suppose you could, I feel as though it'd be a disservice to come up with a way in which you could. The fact that you're asking implies you don't really understand how TSM works.
TSM works quite differently from other backup products and while I suppose you could, I feel as though it'd be a disservice to come up with a way in which you could. The fact that you're asking implies you don't really understand how TSM works.
- Thread starter
- #3
In part you are right, but my major issue is lack of control over my backups and need to start at some point and making a full backup and removing the tapes from the robot seams to me to be a good way to start. After I know I have full backups of my servers maybe then I can start to change settings and try to tune my solution.
At this time I make a full backup and after 1 month the tapes are clean...(or at least some of them)
At this time I make a full backup and after 1 month the tapes are clean...(or at least some of them)
ProbablyDown
MIS
Yrrk, my thoughts exactly...
You are giving yourself too much work, man... If you really want true offsite data security, then set up DRM.
TSM manages all tapes in a stg pool... if you start trying to manually control this, you are actually giving yourself alot more work...
You are better off to set up TSM correctly the first time instead of trying to fix it later, when you have all kinds of nodes on there. You will save yourself lots of headaches...
You are giving yourself too much work, man... If you really want true offsite data security, then set up DRM.
TSM manages all tapes in a stg pool... if you start trying to manually control this, you are actually giving yourself alot more work...
You are better off to set up TSM correctly the first time instead of trying to fix it later, when you have all kinds of nodes on there. You will save yourself lots of headaches...
- Thread starter
- #5
Ok lets see if I can change my question 
I want to make full backups once a month. I also want to be able to get information from the last six month's.
Since my robot only has 24 SDLT and my information won't fit all in there I need to switch tapes now and then.
So having this in mind what couple of rules do you suggest me to follow?
Regards
Jorge
I want to make full backups once a month. I also want to be able to get information from the last six month's.
Since my robot only has 24 SDLT and my information won't fit all in there I need to switch tapes now and then.
So having this in mind what couple of rules do you suggest me to follow?
Regards
Jorge
Let me try it another way.. with a little education on how TSM is different..
TSM does not have a concept of a "full backup" when you're doing file system backups. This is one of the fundamental key differences that seperates TSM from other products. Therefore in a way, you're trying to fit a square peg in a circular hole in your line of thinking. Lets talk briefly about how TSM works differently..
Most backup products deal with Full/Incremental/Differentials and your retention is based on when those were taken and how many of them you wan't to keep. So it makes perfect sense to do what you're asking with other products. You might want to keep your weekly fulls going back 6 months but throw away your daily incrementals (which would obviously mean any version of that file backed up to an incremental would potentially be lost).
In TSM, file system backups are *always* incremental. The first time you backup a server, it backs up all the files since they've never been backed up before. Every single subsequent backup it only backs up files that have changed. There is no need to backup files that haven't changed because the system already has them. It manages what needs to be backed up and what doesn't in it's database. Therefore it consumes far less tape in the long run.
Retention in TSM is also quite different. It's more granular. Instead of retaining based on F/I/D backups, you retain at a file level. You decide how many versions of a file you want to keep and for how long (any any combination of the two). This can be quite powerful. You can easily retain .MP3 files differently then .DOC files and so forth. You can also retain whole filesystems, paths, regular expressions, whatever differently with a high degree of control.
What does this do to the tapes where the data is ultimately stored? Well first of all it really opens up options on where TSM stores the data. Which tape? Where on the tape? Intermixed with other clients or not? This is quite configurable but by default it generally keeps each client's data together and ideally on it's own tapes since that's better come restore time.
The other thing it does to data on tapes is over time as certain files get expired, it creates "holes" in the tape. Initially when you first backup a server, it may fill an entire tape. Down the road, the system likely expires some of those files on that first tape but not all of them (no need to, they haven't changed). This is where concepts such as "reclamation" come in where the system reclaims a tape once the tape reaches a certain percentage of utilization. Intially it's 100% utilized. Over time that drops.. When it reachs whatever reclamation threshold you set, the system can take the "good" data still on that tape and copy it over to a new tape so you can reclaim the portion of the tape not in use anymore.
In your case, if you find your robot filling up but you have ample blank tapes, you have three solutions..
One, check out some of the FULL volumes from the library and put in scratch (blank) tapes. If a restore request comes along that requires a tape that is not in the library, an operator will need to mount the particular tape.
Two, adjust your retention settings to something more realistic that coincides with your need/desire to have ALL backed up data in the library and the library's capacity. This requires some evaluation on your part in determining what's sucking up all the space and making appropriate policy changes.
Three, buy a larger library.
All in all you'll find it very powerful and best to let TSM manage your media for you. You do need to learn the TSM concepts of retention and reclamation which is why i highly recommend you go educate yourself. There is terrific material available. Read my FAQ on here on how to learn TSM as a starter. It's my guess that you're not doing your homework on why you're consuming all that tape. With a little homework and understanding of how TSM retention works, you should easily be able to reign in the consumption to what you want. Most people are lazy (or unknowledgable), however, and get themselves into trouble by not adequately setting retention policies and/or architecting a proper system to handle their backup and restore needs.
TSM does not have a concept of a "full backup" when you're doing file system backups. This is one of the fundamental key differences that seperates TSM from other products. Therefore in a way, you're trying to fit a square peg in a circular hole in your line of thinking. Lets talk briefly about how TSM works differently..
Most backup products deal with Full/Incremental/Differentials and your retention is based on when those were taken and how many of them you wan't to keep. So it makes perfect sense to do what you're asking with other products. You might want to keep your weekly fulls going back 6 months but throw away your daily incrementals (which would obviously mean any version of that file backed up to an incremental would potentially be lost).
In TSM, file system backups are *always* incremental. The first time you backup a server, it backs up all the files since they've never been backed up before. Every single subsequent backup it only backs up files that have changed. There is no need to backup files that haven't changed because the system already has them. It manages what needs to be backed up and what doesn't in it's database. Therefore it consumes far less tape in the long run.
Retention in TSM is also quite different. It's more granular. Instead of retaining based on F/I/D backups, you retain at a file level. You decide how many versions of a file you want to keep and for how long (any any combination of the two). This can be quite powerful. You can easily retain .MP3 files differently then .DOC files and so forth. You can also retain whole filesystems, paths, regular expressions, whatever differently with a high degree of control.
What does this do to the tapes where the data is ultimately stored? Well first of all it really opens up options on where TSM stores the data. Which tape? Where on the tape? Intermixed with other clients or not? This is quite configurable but by default it generally keeps each client's data together and ideally on it's own tapes since that's better come restore time.
The other thing it does to data on tapes is over time as certain files get expired, it creates "holes" in the tape. Initially when you first backup a server, it may fill an entire tape. Down the road, the system likely expires some of those files on that first tape but not all of them (no need to, they haven't changed). This is where concepts such as "reclamation" come in where the system reclaims a tape once the tape reaches a certain percentage of utilization. Intially it's 100% utilized. Over time that drops.. When it reachs whatever reclamation threshold you set, the system can take the "good" data still on that tape and copy it over to a new tape so you can reclaim the portion of the tape not in use anymore.
In your case, if you find your robot filling up but you have ample blank tapes, you have three solutions..
One, check out some of the FULL volumes from the library and put in scratch (blank) tapes. If a restore request comes along that requires a tape that is not in the library, an operator will need to mount the particular tape.
Two, adjust your retention settings to something more realistic that coincides with your need/desire to have ALL backed up data in the library and the library's capacity. This requires some evaluation on your part in determining what's sucking up all the space and making appropriate policy changes.
Three, buy a larger library.
All in all you'll find it very powerful and best to let TSM manage your media for you. You do need to learn the TSM concepts of retention and reclamation which is why i highly recommend you go educate yourself. There is terrific material available. Read my FAQ on here on how to learn TSM as a starter. It's my guess that you're not doing your homework on why you're consuming all that tape. With a little homework and understanding of how TSM retention works, you should easily be able to reign in the consumption to what you want. Most people are lazy (or unknowledgable), however, and get themselves into trouble by not adequately setting retention policies and/or architecting a proper system to handle their backup and restore needs.
mbourne2006
IS-IT--Management
Sorry Yrrk, but I have to voice a slightly different view ...
"In TSM, file system backups are *always* incremental."
This *can* be true, but it depends on your requirements and what you are willing to accept within the constraints of your system.
One of the TSM environments that I manage has a requirement to perform regular full backups of certain systems. We achieve this by running an admin schedule on a Friday afternoon whose job it is to activate a different policy set containing (via management classes) copygroups with parameter "copymode=absolute". Sure, the clients still run a TSM incremental backup, but the "absolute" setting in the associated copygroup forces all objects in the client filesystem to be backed up. We then run another admin schedule on Saturday afternoon to switch back to copymode=modified, thereby reverting to the more common type of incremental backups.
Although this approach is unnecessary in terms of maintaining backup copies of unchanged files, it does have a major advantage in that client data becomes more contiguous on the backup media, and so reduces (a) the number of tape mounts required and (b) reduces the tape scanning time (active and inactive objects are not quite so mixed up) whenever a significant restore is required.
I don't make this point to fuel the debate, but I think it's important to be aware of all the options
Kind Regards,
Matthew Bourne
"Find a job you love and never do a day's work in your life.
"In TSM, file system backups are *always* incremental."
This *can* be true, but it depends on your requirements and what you are willing to accept within the constraints of your system.
One of the TSM environments that I manage has a requirement to perform regular full backups of certain systems. We achieve this by running an admin schedule on a Friday afternoon whose job it is to activate a different policy set containing (via management classes) copygroups with parameter "copymode=absolute". Sure, the clients still run a TSM incremental backup, but the "absolute" setting in the associated copygroup forces all objects in the client filesystem to be backed up. We then run another admin schedule on Saturday afternoon to switch back to copymode=modified, thereby reverting to the more common type of incremental backups.
Although this approach is unnecessary in terms of maintaining backup copies of unchanged files, it does have a major advantage in that client data becomes more contiguous on the backup media, and so reduces (a) the number of tape mounts required and (b) reduces the tape scanning time (active and inactive objects are not quite so mixed up) whenever a significant restore is required.
I don't make this point to fuel the debate, but I think it's important to be aware of all the options
Kind Regards,
Matthew Bourne
"Find a job you love and never do a day's work in your life.
- Thread starter
- #8
Good point Matthew that there is that as well as other ways to force a backup of all the files (archives, selectives, etc). I'm not going to dispute that but this is less about that and more about retention and a capacity problem in his library. Given his level of knowledge it didn't make a whole lot of sense to me exploring how one might do a full *and* solve his retention problem when what i'm reading is he doesn't really understand how TSM handles retention. I am willing to bet this is simply an issue of setting a bit more aggressive retention policies and voila - capacity problem solved. The truth is TSM is highly configurable and often there are many different ways to do the same thing.
I'm a little surprised you find it worthwhile to alter your copymode as you do to achieve the benefits you suggest. Between collocation and tuning how migration and/or reclamation works to suit your needs I find it surprising there would be any benefit, let alone that the benefits of doing so would outweigh the costs.
I'm a little surprised you find it worthwhile to alter your copymode as you do to achieve the benefits you suggest. Between collocation and tuning how migration and/or reclamation works to suit your needs I find it surprising there would be any benefit, let alone that the benefits of doing so would outweigh the costs.
- Status
Similar threads
- Locked
- Question