Well I have been working an issue with their support for 2 weeks now and it seems there is a bug in 5.9 that casues backed up data to get lost when you migrate to 6.1. Their answer is "opps sorry, we had a bug and your backup data is loast, nothing we can do!". Typical Commvault crap. Their software is buggy and quirky and their support is weak. These guys should not be playing in the Enterprise space.
Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
Galaxy Losses data... 2
- Thread starter kluken
- Start date
- Thread starter
- #41
Wow bkpadmin and frbutler, very impressive. I still don;t understand, do you guys rotate tapes, do you not have issues with the fact that CV routinely refuses to prune some jobs off tapes and thus holds the tape? I am constantly manually managing media and that is a waste of time, they constantly tell me it si bugs of what ever as to why the media does not get pruned. Maybe becasue I am a medium sized shop and we watch our tapes we see this more, maybe in a huge shop there are so many tapes you don't notice a few lagging. I also have noticed that CV is slower than Veritas, so with that much data I would think performance issues woudl be compounded.
kluken - I dont see CV being slower then any other product backup product most of which is dependent on your backend.
My mission critical policies are very media intensive -
primary to mag
sych to mag
sych of mag sych to lto2 (i have some lto3 drive but not sure how to go about introducing mixed drives with CV any pointers will be welcomed)
Selective of synch mag to lto2
it is possible if there were a media return problem affecting the primary i wouldnt noticed because I dump out to mag first.
Now that you bring it up ill give this a thorough once over.
On the other hand it seems in your case CV went from losing data to not letting go!
My mission critical policies are very media intensive -
primary to mag
sych to mag
sych of mag sych to lto2 (i have some lto3 drive but not sure how to go about introducing mixed drives with CV any pointers will be welcomed)
Selective of synch mag to lto2
it is possible if there were a media return problem affecting the primary i wouldnt noticed because I dump out to mag first.
Now that you bring it up ill give this a thorough once over.
On the other hand it seems in your case CV went from losing data to not letting go!
- Thread starter
- #43
yeah it is funny, it loses data on the bug, but the annoying problem has been not pruning tapes. As for mixed media, we actually ran SLDT 220 and SDLT 320 in 2 seperat libraries for a while then went LTO3. The biggest issue I had with mixed mixed media was AUX copy mixed media woudl waste space, if we were going from a 220 -> 320 then it woudl read the 220 tape to the 320 then when it ejects the 220 for the next tape it does the same to the 320, the AUX copy is essentially an "image" or bit copy so there was wasted tape, so we just bought a dual LTO3 48 slot library. Not sure how you want to use your mixed media, but if you are going to allocate certain polices to certain meida in the same library you proabbly would have to use bar code rules and generate your own bar codes then associate various bar codes to media pools and then let polices pull from specific media pools. That is what we do. Of my 48 slots I 4 different media pools based on bar codes. I know a given policy will only pull specifc tapes. If you have mixed drives you coudl then use data paths to restrict it further. We customize all our own bar codes using netc,llc software.
mybackupfailed
MIS
BkpAdmin writes:
"I have 9 Commcells / 27 MA's and so many clients I had an MR created in v5.0 for the client group feature you see in v5.9 and I am easily able to manage all of them with QNET."
I write:
See, if you were able to one commcell with multiple master servers you wouldn't need to buy QNET (how much did that cost?). You can't manage 9 Commcells at once since it can't scale up that high. You have to spend $$$ on 9 Commcells and then spend more $$ on QNET just to manage the mess.
BkpAdmin writes:
"Anyway you slice it when managing an enterprise a centrailzied application is key."
I write:
The fact that you need 9 Commcells shows that they don't have a good centralized application model because they can't scale. So they write a kludge like QNET and sell that to you for $10,000. They should have architected it write in the first place so you could have one Commcell that can split into 9 sites and have built in centralized management.
BkpAdmin writes:
"Why don't you report that you do not like the software to your management? Why is it you don't have a choice - your bitterness is astounding......"
I write:
You definitely have me there, I am bitter! Don't tell anyone I used to work there. Once an IT dept spends tens of thousands of dollars they don't want to hear an admin tell them they made a mistake.
"I have 9 Commcells / 27 MA's and so many clients I had an MR created in v5.0 for the client group feature you see in v5.9 and I am easily able to manage all of them with QNET."
I write:
See, if you were able to one commcell with multiple master servers you wouldn't need to buy QNET (how much did that cost?). You can't manage 9 Commcells at once since it can't scale up that high. You have to spend $$$ on 9 Commcells and then spend more $$ on QNET just to manage the mess.
BkpAdmin writes:
"Anyway you slice it when managing an enterprise a centrailzied application is key."
I write:
The fact that you need 9 Commcells shows that they don't have a good centralized application model because they can't scale. So they write a kludge like QNET and sell that to you for $10,000. They should have architected it write in the first place so you could have one Commcell that can split into 9 sites and have built in centralized management.
BkpAdmin writes:
"Why don't you report that you do not like the software to your management? Why is it you don't have a choice - your bitterness is astounding......"
I write:
You definitely have me there, I am bitter! Don't tell anyone I used to work there. Once an IT dept spends tens of thousands of dollars they don't want to hear an admin tell them they made a mistake.
mybackupfailed
MIS
Frbutler writes:
"I also know it may not be the best but it is what I like and what I think to be the easiest to work with based on what I have been exposed to.
It may not be for everyone. But it works for me."
I write:
Can't argue with that. Seems like you have it all under control. Watch out for 5.9 scripts when you upgrade to 6.1, they will have similar problems you had with Veritas since they completely rewrite their script interface.
"I also know it may not be the best but it is what I like and what I think to be the easiest to work with based on what I have been exposed to.
It may not be for everyone. But it works for me."
I write:
Can't argue with that. Seems like you have it all under control. Watch out for 5.9 scripts when you upgrade to 6.1, they will have similar problems you had with Veritas since they completely rewrite their script interface.
I would think that if you were able to report to upper management that a specific application does not work and that they are vulnerable, they would have to listen.
I know if I were bitter about the application I was "Forced" to work with I would find another job that does not require me to do something against my will.
Exactly how much data is passed between a MA and the CS during backup operations? I think the concept behind multiple CS's is to reduce WAN traffic between sites.
If I were to kickoff 200 backup jobs a night and had them all reporting their status across the WAN to the CS would that not be an issue? I think it would.
I think it would be with CV or NBU.
It sounds like.. correct me if I am wrong... you just do not like the way Commvault works. If you know what you want in a product, you should find one that meets your needs and then write up a justification to management on why you should switch....
I mean what do companies like Nokia, Compusa and Fidelity know about backups. They must be crazy or staffed by incompetent people to use (in your words) a product that is "a mess"
If I were forced to use NBU soley I would complain only because I do not know it like I know CV.
How a backup landscape is engineered has a big impact on how efficiently it runs and how easy it is to manage.
Single master server concept is single point of failure.
If a WAN link goes down (that never happens) all backups fail.
If your master server dies no backups run anywhere
I think it all comes down to how much are you willing to spend and how much risk are you willing to take.
I know if I were bitter about the application I was "Forced" to work with I would find another job that does not require me to do something against my will.
Exactly how much data is passed between a MA and the CS during backup operations? I think the concept behind multiple CS's is to reduce WAN traffic between sites.
If I were to kickoff 200 backup jobs a night and had them all reporting their status across the WAN to the CS would that not be an issue? I think it would.
I think it would be with CV or NBU.
It sounds like.. correct me if I am wrong... you just do not like the way Commvault works. If you know what you want in a product, you should find one that meets your needs and then write up a justification to management on why you should switch....
I mean what do companies like Nokia, Compusa and Fidelity know about backups. They must be crazy or staffed by incompetent people to use (in your words) a product that is "a mess"
If I were forced to use NBU soley I would complain only because I do not know it like I know CV.
How a backup landscape is engineered has a big impact on how efficiently it runs and how easy it is to manage.
Single master server concept is single point of failure.
If a WAN link goes down (that never happens) all backups fail.
If your master server dies no backups run anywhere
I think it all comes down to how much are you willing to spend and how much risk are you willing to take.
mybackupfailed
MIS
One more post from me. The reason you have to run with multiple CS's is not really because of fault tolerance but because of hard limits. You can only install so many clients, so many media agents, and run so many concurrent jobs (ask Commvault! They know all about it). So when you hit a certain number you have to spend lots of $$ and install another CS. No CS is fault tolerant unless you cluster it so having multiple CS's only spreads the load and eases your dependence on one CS. But each CS is a single point of failure no matter how many you have. Not much data passes between media agents and CS's and going over the WAN between a CS and MA is not really a problem. MA's write to an index so the index needs to be local. 200 jobs running on a MA with a CS over the WAN isn't a problem (we do it nightly). Multiple CS's are (in my opinion) a band-aid for the current design because of the limitations of their software. Why can't all your CS's talk to each other and update each others databases? Then you could manage everything at once, run one set of reports, and have fault tolerance. I think (don't know for sure) using multiple Veritas master servers accomplishes this.
My main complaint with Commvault is your vulnerability to bugs. You are more likely to find a critical problem then they are and you have to pull teeth to get them noticed.
Not trying to be nasty but judging the software by the customer list shouldn't be a barometer of software quality. When a critical restore has to be run and you find a bug for the first time you won't be impressed anymore by the list of big companies who also bought the stuff.
My main complaint with Commvault is your vulnerability to bugs. You are more likely to find a critical problem then they are and you have to pull teeth to get them noticed.
Not trying to be nasty but judging the software by the customer list shouldn't be a barometer of software quality. When a critical restore has to be run and you find a bug for the first time you won't be impressed anymore by the list of big companies who also bought the stuff.
interesting article in September's ComputerWorld..anyone seen it regarding Netbackup?
Apparantely a lot of users are holding off upgrading to v6.0 (which was released October 2005) because of too many bugs....kind of puts the argument about Veritas doing more Q&A to bed doesn't it ;-)
Apparantely a lot of users are holding off upgrading to v6.0 (which was released October 2005) because of too many bugs....kind of puts the argument about Veritas doing more Q&A to bed doesn't it ;-)
- Thread starter
- #51
I guess they are all the same, but I just feel CV does nto care. We reported a buug last year in 5.9 that still is not fixed and latest we are being told is that it will bt 7.x before they fix it. On windows clients you get random intermittent jsystem state backup failures due to RSM. They knew about this bug last year. The down side is when that job run with errors it casues all associated childs jobs to not get pruned until the next successful full. Well that means tapes may be held active longer than you think as there are jobs that should have pruned that are not getting pruned. There answer is to manually prune the jobs or apply a registry key to ALL clients telling them not to backup RSM. Great fix huh. It goes back to none of the vendors are perfect, but it is the attitude that counts and Cv does not care about fixing bug, just spitting out new releases with new features. Took 3 days to get a answer on this one even though the bug has been there for ever and i havd to esclate to the director of Tech support to get an answer in 3 days! and then it was like, hey we gave you work arounds.
Kluken - Your arguement is running out of steam as you seem shocked to find out there are bugs with software and your tout of switching to Veritas has been sullied by dataloss bugs on a grand scale which were known and not fixed for over a year.
I have copied the article from Computer World for others to read as well as yourself.
September 05, 2006 (Computerworld) -- Users are saying that problems with Symantec Corp.'s latest version of Veritas NetBackup are keeping them from rolling out the application even though the data backup and recovery product was shipped more than a year ago.
Veritas Software Corp. announced NetBackup 6.0 in April 2005 (see "Veritas announces next version of NetBackup, EVault") and began shipping it in October, just after Symantec announced that it was buying Veritas (see "Sidebar: Planned purchase leaves some Veritas users cold").
Users on at least two Web site forums dedicated to Symantec customers said they are hoping that Symantec's Maintenance Pack 4, due out around October, will fix bugs they're finding in NetBackup 6.0. The most recent maintenance pack, No. 3, was made available on June 30.
Bank of Canada, whose job it is to issue money for that country like the Federal Reserve in the U.S., plans on staying at NetBackup Version 5.0 "until it's apparent things with 6.0 have improved," said Paul Keating, an IT manager at the bank, which is based in Ottawa.
Problems cited by other users include issues with the vault service, which automates tape rotation; the scheduler, which lets users specify the times that backups are run; and the Job Manager service, which accepts backup jobs and then runs them. When these features don't work, automated backups can't be performed and aren't saving the data properly when they do, users said.
"The vault service won't start," said Steve Bally, systems engineer for RadiSys Corp., a Hillsboro, Ore., engineering company that has been using NetBackup for a number of years. Without the vaulting service, Bally said, he can't perform tape duplication or send backups off-site because of problems with NetBackup.
Bally said he doesn't plan on rolling out NetBackup 6.0 until the vaulting service issues are fixed, so he has not yet experienced the scheduler problem described by several other users on the Web site forums for Symantec customers.
There are about 36,000 users of NetBackup, according to Symantec. Mike Adams, group manager for Symantec's NetBackup product marketing, said he could not say how many of those users have migrated to Version 6.0. "Anecdotal evidence" is that there is "good uptake," he said.
One major issue Veritas users cited with NetBackup 6.0 is that when a backup job indicates it's complete, it often has only backed up 1K files, so there's no data to restore. Bally characterized Veritas 5.1, which he was using before 6.0, as "solid" but said he wanted to use Version 6.0 for new services, such as the ability to restore Microsoft SharePoint Portal Server services.
Another major problem reported by users is an issue with the Job Manager. Due to problems with this feature, "NetBackup management went from maybe 30% of my time to 60% or more," said Phil Rand, senior systems analyst and systems administrator for Seattle Pacific University. Not only did backup jobs fail, but they would disappear from the activity monitor so that Rand might not notice they were missing, he said. Rand has received binaries from Symantec, in advance of Maintenance Pack 4, to help him fix the problem, he said.
Other users are reporting problems with multistreaming backups and restores, which establish multiple connections, or threads, from a single system to the backup server. This is useful for users with a large system with multiple I/O devices and large amounts of data that need backing up. Some jobs would retry hundreds or thousands of times, even though the actual backup was completed successfully, Rand said.
Muhammed Shafi, senior Unix administrator at Pactiv Corp., a Lake Forest, Ill.-based manufacturer of packaged products such as Hefty garbage bags, said it took Symantec more than a month to resolve an issue where a multistreamed backup was able to restore files from only one stream. He had been able to perform restores with multiple streams in Version 5.1, he said.
Users also reported that the software's more sophisticated features were difficult to use because of a lack of documentation on how to configure them. Mike Sponsler, a Unix/Linux systems administrator for a Washington office of Northrop Grumman Corp. who migrated to NetBackup 6.0 from EMC Corp.'s Legato NetWorker, said he was generally happy with the product but that he was having configuration issues with advanced features such as database backup and restore.
Sponsler also said NetBackup Operations Manager, Veritas Security Services and Vault Manager also were difficult to manage. "Things such as [Veritas Security Services], the general consensus is to not use it unless it's a matter of life or death," he said.
Cupertino, Calif.-based Symantec said its fourth maintenance pack is due out this month or next -- on its normal quarterly release schedule -- and that the number of reported bugs in Version 6.0 was not out of the ordinary for a release of its size. "Any time you come out with a big-scale release, over time, the same number of requests come in," Adams said.
Adams said he had not heard specifically of the issues raised by the users and did not know what problems Maintenance Pack 4 intends to correct.
For users holding off on upgrading from Version 5.1 to Version 6.0, Symantec typically sells a product for two years or until the next major release, and Version 5.1 came out in June, 2004, Adams said. While users can still buy components of NetBackup 5.1, it has been pulled off the active sale list. However, he said he expects it to continue to be supported until 2008 or 2009
I have copied the article from Computer World for others to read as well as yourself.
September 05, 2006 (Computerworld) -- Users are saying that problems with Symantec Corp.'s latest version of Veritas NetBackup are keeping them from rolling out the application even though the data backup and recovery product was shipped more than a year ago.
Veritas Software Corp. announced NetBackup 6.0 in April 2005 (see "Veritas announces next version of NetBackup, EVault") and began shipping it in October, just after Symantec announced that it was buying Veritas (see "Sidebar: Planned purchase leaves some Veritas users cold").
Users on at least two Web site forums dedicated to Symantec customers said they are hoping that Symantec's Maintenance Pack 4, due out around October, will fix bugs they're finding in NetBackup 6.0. The most recent maintenance pack, No. 3, was made available on June 30.
Bank of Canada, whose job it is to issue money for that country like the Federal Reserve in the U.S., plans on staying at NetBackup Version 5.0 "until it's apparent things with 6.0 have improved," said Paul Keating, an IT manager at the bank, which is based in Ottawa.
Problems cited by other users include issues with the vault service, which automates tape rotation; the scheduler, which lets users specify the times that backups are run; and the Job Manager service, which accepts backup jobs and then runs them. When these features don't work, automated backups can't be performed and aren't saving the data properly when they do, users said.
"The vault service won't start," said Steve Bally, systems engineer for RadiSys Corp., a Hillsboro, Ore., engineering company that has been using NetBackup for a number of years. Without the vaulting service, Bally said, he can't perform tape duplication or send backups off-site because of problems with NetBackup.
Bally said he doesn't plan on rolling out NetBackup 6.0 until the vaulting service issues are fixed, so he has not yet experienced the scheduler problem described by several other users on the Web site forums for Symantec customers.
There are about 36,000 users of NetBackup, according to Symantec. Mike Adams, group manager for Symantec's NetBackup product marketing, said he could not say how many of those users have migrated to Version 6.0. "Anecdotal evidence" is that there is "good uptake," he said.
One major issue Veritas users cited with NetBackup 6.0 is that when a backup job indicates it's complete, it often has only backed up 1K files, so there's no data to restore. Bally characterized Veritas 5.1, which he was using before 6.0, as "solid" but said he wanted to use Version 6.0 for new services, such as the ability to restore Microsoft SharePoint Portal Server services.
Another major problem reported by users is an issue with the Job Manager. Due to problems with this feature, "NetBackup management went from maybe 30% of my time to 60% or more," said Phil Rand, senior systems analyst and systems administrator for Seattle Pacific University. Not only did backup jobs fail, but they would disappear from the activity monitor so that Rand might not notice they were missing, he said. Rand has received binaries from Symantec, in advance of Maintenance Pack 4, to help him fix the problem, he said.
Other users are reporting problems with multistreaming backups and restores, which establish multiple connections, or threads, from a single system to the backup server. This is useful for users with a large system with multiple I/O devices and large amounts of data that need backing up. Some jobs would retry hundreds or thousands of times, even though the actual backup was completed successfully, Rand said.
Muhammed Shafi, senior Unix administrator at Pactiv Corp., a Lake Forest, Ill.-based manufacturer of packaged products such as Hefty garbage bags, said it took Symantec more than a month to resolve an issue where a multistreamed backup was able to restore files from only one stream. He had been able to perform restores with multiple streams in Version 5.1, he said.
Users also reported that the software's more sophisticated features were difficult to use because of a lack of documentation on how to configure them. Mike Sponsler, a Unix/Linux systems administrator for a Washington office of Northrop Grumman Corp. who migrated to NetBackup 6.0 from EMC Corp.'s Legato NetWorker, said he was generally happy with the product but that he was having configuration issues with advanced features such as database backup and restore.
Sponsler also said NetBackup Operations Manager, Veritas Security Services and Vault Manager also were difficult to manage. "Things such as [Veritas Security Services], the general consensus is to not use it unless it's a matter of life or death," he said.
Cupertino, Calif.-based Symantec said its fourth maintenance pack is due out this month or next -- on its normal quarterly release schedule -- and that the number of reported bugs in Version 6.0 was not out of the ordinary for a release of its size. "Any time you come out with a big-scale release, over time, the same number of requests come in," Adams said.
Adams said he had not heard specifically of the issues raised by the users and did not know what problems Maintenance Pack 4 intends to correct.
For users holding off on upgrading from Version 5.1 to Version 6.0, Symantec typically sells a product for two years or until the next major release, and Version 5.1 came out in June, 2004, Adams said. While users can still buy components of NetBackup 5.1, it has been pulled off the active sale list. However, he said he expects it to continue to be supported until 2008 or 2009
- Thread starter
- #53
Not sure how my argument has lost steam. Sure they are having issues at Veeitas, but I don't see LOST data or unable to recoever data as a underlying theme. I see vaulting issues so sure some tapes are not getting duplicated correctly, but the data is still there. CV lost our data, no recovery as an option. Like I said it goes to attitude more, Veritas seems ot be working with cusmtoer and even is quoted as supplying updated binaries ahead of the SP. I can't CV to reply let alone admit anything or take action.
let me quote it for you since you missed it "One major issue Veritas users cited with NetBackup 6.0 is that when a backup job indicates it's complete, it often has only backed up 1K files, so there's no data to restore"
what this translates too is - Veritas reporting your 1TB job as complete but in reality has only backed up 1kilobyte.
let me put a scenario in place for you - its 2 years in the future from this moment - you have to perform a restore from a backup performed with this bug - you attempt the restore and there is nothing there event though your report shows the job as successful.
I can also comment on how the Veritas multistreaming commonly overwrites valid media and performs stream mistep - however thier forums say it best.
yes kluken i concede that they have blessed thier customers with new binaries after a year of this being in the field. Wow that is fantistic customer service!
what this translates too is - Veritas reporting your 1TB job as complete but in reality has only backed up 1kilobyte.
let me put a scenario in place for you - its 2 years in the future from this moment - you have to perform a restore from a backup performed with this bug - you attempt the restore and there is nothing there event though your report shows the job as successful.
I can also comment on how the Veritas multistreaming commonly overwrites valid media and performs stream mistep - however thier forums say it best.
yes kluken i concede that they have blessed thier customers with new binaries after a year of this being in the field. Wow that is fantistic customer service!
- Thread starter
- #55
Well if that data loss issues is true, and I would suppose it is then that is extremely bad of them to. I would also like to see more details, as is the job saying "complteed with exceptions". Is it a problem with someone glossing over backup report info and not picking up that a backup ran, but did not run successfully, versus, says it ran 100% and did not really? So what we are both saying is backup vendors suck, none of them have a decent product or QA.
- Thread starter
- #58
Well that is a said state of Data Protection vendors. Honetly our expieriences with Backup Exec in 75 locations over last 5 years have been flawless, I assumed Veritas would have same quality on the NBU product, but evidently being anchored in ancient outdated code makes it no better than the fly by the seat of your pants vendors like CV.
- Status
Similar threads