What is the max size of output files that CE10 can handle?

[khartman]

Hello all,

Here is the issue. I am running CE10 on two clustered (full-install) Windows 2000 servers. The users view their report via the CE portfolio and authenticate via Windows AD.
We have some reports that are self-scheduled to the Excel format (not the data only version) by the users. The issue we are having is that the amount of data being generated in a single report is huge. The output files I have seen in the FRS are anywhere from 16MB to 67MB - these are Excel files - in size.

So what ends up happening is that when the users attempt to open these via the ActiveX viewer (note we are all running WinXP Pro desktops) the rptviewer script launches and the ActiveX viewer opens only to end up closing a few seconds later without warning. It does not even get to the point where it asks if the user wants to view the output or save it to disk.

I have tried to explain to the users that it is the file size which is causing the problem and that the web component server cannot handle transferring such a large file to the viewer. Am I correct in this? Does anyone know what the file size limit is and if it can be adjusted in the registry?

I have even copied the files directly from the output repository to my desktop and tried opening in Excel directly and it takes over 5 minutes to open the 67MB file. By the time it does open there are numerous worksheets, each one containing the maximum 65K+ records.

Any help would be appreciated.
Thanks, Ken

Kenneth Hartman
Manager, EIS
Hughes Network Systems

http://www.hns.com

 

[Turkbear]

Hi,
First, how is the ActiveX viewer showing Excel spreadsheet files?( I assume you mean they attempt to export to that format and then open ( with Excel) the file when asked)

Second, I never let users anywhere near the FRS directories.

Users should only export to Excel on a shared or local disk, not to the FRS and CE scheduled runs/exports should use some other disk location and have your users run them in Excel from that site.
( IIRC, Excel has a max row limit of @65000 )






To Paraphrase:"The Help you get is proportional to the Help you give.."

 

[khartman]

Turkbear,

Thanks for responding so quickly. You are correct, the users are using the "view latest instance" option to open the report. Generally they are then asked if they want to view the report, which then launches the Excel application, or if they want to save it to a local or network disk location - NOTE: they cannot save anything to the FRS , the users do not have access to the FRS at all - they don't even know where it resides.

I should have been more concise, I am the CE Admin and have access to the FRS. I only went to the Output FRS to see what the actual file size was in order to validate my assumption that this is what was causing the problem.

Although the max limit in excel is 65K rows, CE builds the additional worksheets to accomodate the excess records. I think the core of the issue is the fact that there are large number of records being returned because the report designer has not limited the user to a specific maximum date range (e.g., only three months of data are returned no matter what date range the user put in).

For example, one report which is in the Excel data-only format returned over 315K rows. This file was not able to be retrieved via the "view latest instance" option. I haven't tried this yet, but I am thinking that the report would be retrievable if the format was left as a Crystal Report format - of course I am not sure whether they would be able to export such a large file which may just leave me in the same position. My only other alternative is to have the report designer limit the date range to bring back a more manageable number of records at a time.

Thanks,
--Ken

Kenneth Hartman
Manager, EIS
Hughes Network Systems

http://www.hns.com

 

[Turkbear]

Hi,
You can also limit the # of records returned by setting a number for the max in the CMC for the page server(s).

Users hate that, however, since no message tells them that it is limited..









To Paraphrase:"The Help you get is proportional to the Help you give.."

 

[khartman]

TurkBear,

I thought of that, but your right I would have users all over me for that one. After the last post I did a little experiment. I scheduled one of the problem reports to the standard Crystal Report format and was able to open it fine in the viewer (1611 pages!!!!)[/color red] - I really have to wonder what analyst is going to go through that many pages!! Anyway, I was able to then export the file to Excel; however, it has been several minutes since I tried to open the file in Excel and it is still trying to load it....so the bottom line is I need to educate the users on running reports that are going to focus them on the subset of information that they really need to work with - no more of this "shotgun" approach. I set up a meeting with the report designers for the finance group to discuss them putting limitations on the their reports - if a user goes outside of that limit they can always display a message in the header of the report informing the user that the result set was truncated automatically.

I will let you know how it turns out. BTW, do you know if there is anything published by BusinessObjects which states that there is a limit to the size of the files which can effectively be viewed via the web portal?

Thanks again,
Ken

Kenneth Hartman
Manager, EIS
Hughes Network Systems

http://www.hns.com

 

[Turkbear]

Hi,
Not that I am aware of, but someone viewing thread this may know..( Synapse? )...





To Paraphrase:"The Help you get is proportional to the Help you give.."

 

[paulmarr]

I am experiencing a similar problem - a report is scheduled in CE to to output a Excel spreadsheet to Unmanaged disk.

The report runs fine as a Crystal Report (albeit taking 2 hours) but as soon as we schedule to unmanaged disk and excel - it processes for two hours - and as soon as it tries to dump the file I get a "Not enough memory for operation" error.

Depending on the parameters - sometimes the record set is small enough that the schedule succeeds - but again - try viewing the Latest instance and I get a Web Component error.

I would love to read if anyone knows the answer to this thread!

Cheers
paulmarr

 

[synapsevampire]

Paul: You may be exceeding the maximum number of rows in Excel.

A report that takes 2 hours is probably poorly designed, I would guess that the record selection formula is amiss, and you're not passing the criteria to the database.

-k

 

[sijeffs]

Paul,

Unmanaged Disk
I've found that the largest file I've managed to successfully schedule to unmanaged disk is about 720mb and its always run out of memory if any larger.

Excel
I frequently have reports which output approx 350k rows in Excel and they will always open within a few seconds. I generally work on the basis of about 50mb being the limit for the viewer.

Regards.

Simon

 

[khartman]

Simon,

Can you give me a little more info on the Excel output you are bringing to your users via the viewer? This is where I am having issues. The reports they are generating are around 60+MB in size and it seems that the viewer will not open the file. It asks if they want to view or save, in either case it never finishes. Is your output in the Excel data-only format or the Excel format? These users requested the out put be in Excel format, my recommendation was to put it in the data-only format to avoid the excess formatting.

Also are you running CE10 or XI/XIR2?

--Ken

Kenneth Hartman
Manager, EIS
Hughes Network Systems

http://www.hns.com

 

[sijeffs]

Ken,

Apologies for the delay in responding.

I'm using CE10 by the way.

I think that you need to reduce the size of your output files.

You could try the following;

Reduce the amount of white space (Blank cells).
Get rid of any non critical columns.
Where possible concatenate data i.e. Surname and FirstName into a single column.

I'm not sure how much of a saving there will be by using Data only though I'd be interested to know if you find out.

Are you getting any messages when the viewer fails to open?

Regards.

Simon