UNIX Reports into PDF to server over web 1

[Haybails90]

Hey Folks,

I just stepped into a new company this month and I am trying to wrap my brain around a current scenario which they would like solved.

This is a relatively large private company with yearly business in the billions coming via slightly more than 150 'branch' locations around the US. Now the company's backbone is running on AT&T UNIX System 5 Release 4 (with possible plans to switch over to AIX 5.2 UNIX) which poops out about two to three thousand UNIX reports PER MONTH! The plan is to make month end reports available on our secure, asp based, web site. I don't need to make ALL of these available each month (LOL) but a select few. Now, they say they'd originally thought about PDF versions of the reports but, as some of the larger branches are still on dial-ups, they originally nixed the idea for two reasons:
1.) they were originally manually taking each report and saving it as PDF
2.) some of the reports can get up to a thousand or more pages in length and the upload via dial-up was scaring them.

So, my very basic quest right now is for any feed back on what may be the best way to grapple with this scenario. Should I be thinking PDF? Or would I be better off walking into the 'Crystal Reports' frame of mind? Or, still yet, looking elsewhere for a better solution?

Thank you, one and all, for any thoughts and feedback.




Haybails90
(I also posted this in the 'Adobe:Acrobat' forum)

 

[DreXor]

well if you have the branches ZIP or LZH or GZip them and upload them to a "handling" folder, the nice part is they're in their raw format, which will compress nicely (assuming plain text or the likes) for easy dialup compliance, and in turn gives you full happy acess to any info in the file, vs a pdf where you have to kinda twist things about to say read page 400 of report X instead of opening report X in a PDF viewer and scrolling to page 400



DreX
aKa - Robert
if all else fails, light it on fire and do the happy dance!

 

[Tarwn]

I agree with Drexor, the reports will have to be made available to you in, and setting up a cron job on the Unix servers to compress them and save them to a locally accesible folder on your web machine might be the best way to handle it. Unfortunatly you then run into theproblem of unzipping them, etc.

I think my solution would be to first identify exactly wat information is needed in the monthly reports, then set up a BASH or Perl script on the individual servers to extract that exact information, place it in a single file, then upload it as plain text. The reason I suggest doing this on the original servers (despite the work involved in adding this as a cron job to every single server) is because it will cut down immensely on communications time, bandwidth needs, etc to transfer the files from the servers. It will also mean that your web server won't have as much work to do in terms of parsing the files (you could set up a vbs batch file or perl script and have it scheduled to run every sunday night, or something) as most of the irrelevent data will have been removed. You also save on disk space on your web server, etc.

Now, as far as PDF vs other formats, I find that if a report is long enough to warrant concerns about PDF downloads (like 400 pages) then it is probably to long for anybody to actually read. You may be better off breaking the data down into two sets: (1) The stuff people will actually want to read every single week/month/whatever and (2) the wekkly/monthly/etc stuff they would like accesible at some poit in the future but probably would only look at once in a blue moon. Aty this point you could use your script to create a weekly/monthly/etc 5-10 page report (at the most) and shove the rest of the data into a database with an ASP front-end so that users could look at the data on demand rather then all at once, every week.

---
So in a nutshell, here was my proposal:
1) Get details on exactly what data is needed
2) Build a Shell or perl script to extract that data from existing files on a Unix server and save it as a text file
3) Upload that file to a specified directory on the windows server
4) Once that works, deploy the script to all your Unix servers, change settings as needed
5) Sit down [with mngmt] and figure out exactly what info they want regularly and wat info they just want available at some point in the future
6) Write a VBScript or Perl Script to parse the incoming Unix data files, sending the data to the database
7) Write a second script (or extend the first) to create PDF documents to build the weekly/monthly/etc need-to-know reports based on meeting in (5)
8) Write an ASP front-end to allow users to view data in some meaningful way from the database

Advantages:
Majority of parsing load moved to Unix servers,
Disk space needs on Windows server reduced,
data communications between Unix servers and Windows server reduced,
PDF [or alternative] Report size reduced to manageable [and readable] size,
Reduce needed resources for building of reports on Windows server,
Reduce download time/bandwidth of PDF [or alt.] reports

Disadvantages:
Requires a meeting or two extra with management
Application has more possible points of failure since it relies on a more distributed model
More know-how required since part of the work will be executed on Unix servers (ie, if you don't know bsh scripting or Perl scripting, it may be time to learn)


Anyways, I think thats how I would handle the problem. As I mentioned befoe, I don't think anyone gains much from reading a 40 page report. Trimming it down and makingthe data available in a website may be a better way to approach this. That way everyone can get the information that everyone wants, while minimizing the transfer of information that individuals don't bother with.

-T

 

[Haybails90]

'DreXor' and 'Tarwn',

Thank you both for your responses. 'DreXor', thank you for diving right in with the first response, and 'Tarwn' thank you for a well thought out response. This is some GREAT initial brainstorming and project planning that you've offered here. I really appreciate your thoughts and I plan on using your input to help me catalogue my strategy and plan of attack.




Haybails90