How to efficiently delete a lot of data?

[DrDDT]

Hi,

We have to remove several gigabytes of old data.
What is the best way to do this, as fast as possible?

Can I bypass recycle bin somehow, without disabeling it for users?

 

[appnair]

Here's a thought.I am not entirely sure it will work or not.We used this kind of method when we used PVA's trashcan.That was possible because the trashcan did not have a schema dependency.If recycle bin does not need to be on every server,temporarily configure a server,perform the delete(using lapi,oscript) on that server.If this method works nothing on your prod servers need to change.

Well, if I called the wrong number, why did you answer the phone?
James Thurber, New Yorker cartoon caption, June 5, 1937

 

[DrDDT]

Unfortunately, Recycle Bin uses schema changes.
I tried it, and it didn't work.

Could we use lapi/oscript to delete both?
First delete the node, then remove it from recycle bin?

I have to do this montly, so it would be nice to have some sort of script to remove this data automatically every month.

 

[ggriffit]

I guess you could create a patch that would override the move to Recycle Bin method and then suspend the DB triggers for Recycle Bin and Notifications so that they are not triggered and then you can do the delete.

Greg Griffiths
Livelink Certified Developer & ECM Global Star Champion 2005 & 2006

http://www.greggriffiths.org/livelink/

 

[appnair]

or if you want to read the objects form the recycle bin as a list and use a lapi.delete on it and see if it is a real delete.

Well, if I called the wrong number, why did you answer the phone?
James Thurber, New Yorker cartoon caption, June 5, 1937

 

[DrDDT]

Does object importer do a 'real delete'?

 

[Stelex]

Do you want to physically remove the data, or just remove the refefence to the data? If you want to simply remove the reference - a SQL call might be in order.

If not, and I suspect this is the case, then you will probably have to disable the recycle bin.

One customization I have seen though is to have the document moved into a hidden folder, and the documents from that folder were then able to be deleted with going through the recycle bin - but then, this was a customization and I can't even remember how they did it. This way the company had an out - if they needed to restore a document that had been deleted they could have an administrator pull it out of this hidden folder where he could move it wherever the user needed it to be moved and eveything was auditable - sort of a customized recycle bin with audit trail, if you can imagine this.

 

[DrDDT]

I need to physically remove the data, because it uses several GB of SAN space I need back.

I'll just delete it twice. (Or I can try to hold the shift key when deleting

 

[Stelex]

Hmmm - something that would be possible is to:

1. Create a secure instance with the documents in
question installed.

2. Attach the new instance to the SQL database

3. Set the Purge Documents setting to a minimum value

4. Move the files in question to a single folder

5. Delete the folder

6. Allow the system to purge the deleted documents.

7. Run a Level 5 database verification to identify any
lingering SQL entries for the database.

8. Identify from the SQL verification utilities any
documents that exist in the data store without SQL
entries and remove the appropriate documents.

It's a time consuming process, but one that probably works the best.