Batch Script Ideas

[clk430]

Winzip doesn't like us, and we don't like it. We have a batch script that calls wzunzip.exe to unzip large files for Mercator to pick up. Unforutnatly, wzunzip is too slow unzipping and trips a double trigger flag "source not available" because the file is locked until all files are unzipped.

Here is the basic script called from the output batch adapter:

wzunzip %1 E:\EC\Appdata\ChoicePoint_PoliceRepts\

Basically, it grabs the *.zip, unzips it and places it in the folder above.

My goal is to have the file saved as a temp extension then until it is complete, then rename it. I succeeded with the same issue with gpg.exe. Originally, it was:

gpg -e -r xxkeyxxx %1

and I modified it to be:

gpg -o "%~dpn1.tmp" -e -r xxkeyxxx "%~f1"
ren %~dpn1.tmp %~n1.gpg

This saved it to a temp extension, then renamed it when it was done. I just can't figure out how to do it with unzip. Any ideas? Cliff notes for my question?

 

[eyetry]

Winzip uses temporary file names duringthe unzip process and renames the file(s) when complete.

 

[BocaBurger]

It does from the GUI, but does it from the command line? What compression do you havge set, the lowest compression is the fastest to process. You might want to contact WinZip's support for this question.

You did buy a license?



BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[clk430]

Yes, we bought the license, but the customer support is, well, not helpful at all. I'll look into the compression as you suggested.

 

[clk430]

Alright, what about commands with the batch adapter?

For:
wzunzip %1 E:\EC\Appdata\ChoicePoint_PoliceRepts\

Is there a command that could specify to write the output of the wzunzip executable to a temp file first until it has completed? Use it in conjucntion with the -Poll command? -File command?

This problem has plauged our systems for 4 years now. I think my company will write you a nice check if it can be fixed?

 

[BocaBurger]

Batch adapter works just like calling a batch file from the OS.

Possible solution: wunzip to a different directory, rename the file to the trigger directory with a .tmp extension, rename the file to the correct value. If the rename to the .tmp is fast enough, you might combine steps 2 & 3.

Can't accept the check.






BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[clk430]

The zip file has many files inside with the only common denominator being .tif as the extension. I would assume the map that renames the files to a .tmp extension and then back would run into the same problem as the trigger map is getting.

 

[eyetry]

First, yes from my experience winzip uses temp file names from the command line. I've watched it unzip large files and rename at the end. Not sure if there are special WinZip settings though as we simply install winzip (v8) and run.

Second, we have two control maps that execute a runmap that unzips files via batch files. The zip file is the trigger file. Once the decompression is completed successfully the control map executes the next step in the mapping process, formatting data to be loaded to temp tables. Regarless, temp file name or no, everything is processed without contentions because the unzip map releases it lock when complete. Problem is that you need to know the name of the output file(s) that will be processed. So, if you can't control that this process would be more difficult to implement.

 

[BocaBurger]

The file renaming and transfer would have to be done outside of DataStage TX. A batch file could be called to do this.

BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[clk430]

Ok guys, check this out. I've been working with BocaBurger in another thread and changed the following .ini settings to address another problem:

ListenSleepTime from 200 to 400
InitPendingHigh from 0 (unlimited) to 200
InitPendingLow from 0 (unlimited) to 100
WatchMaxThreads from 20 to 8
MaxThreads from 20 to 12
RunMapCaching from ;300 (disabled) to 100

By doing this, I fixed the "source not availble" error!!! Well, fixed 95% of the errors that this one map generates.

I suspect it's the listenSleepTime?

 

[BocaBurger]

Probably reduced the load on the server and improved performance. This changed the timing between components.
I always say a well tuned system runs better and has less problems.





BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[clk430]

I always say a well tuned system runs better and has less problems."

Amen brother!