Cube processing - Disk space consumption

[johnem]

Dear Listers,

At the request of one of our developers, I was asked to monitor C:\ drive disk-space consumption during cube processing. At the commencement of the processing we had approx. 2.8GB available. The partition data files, ie. (dbname.agg.rigid.data), peaked to approx.100MB. DIsk space continued to be consumed.

I ran a search (incl. system/hidden files), over the disk ordered by size and could not view any files that were increasing in size. The processing was eventually canned just prior to all the 2.8GB being consumed.

Can anyone assist as to what AS is doing that would consume this space and where should I direct further investigation?

TIA

John

 

[MDXer]

John

When analysis services processes a cube it actually builds it in a temp file that has 0byte file size rarely will you see this file report anything over 0bytes, but doing a properties on the file will show it in megs or gigs depending on your cube size. After the processing is successfully completed it then overlays the new data onto the old files. So what you end up with is 2 copies of your Cube during processing. However there is an edit you can make and I do reccomend making the change. In the registry under
HKey_Local_Machine\Software\Microsoft\OlapServer\CurrentVersion
there is a key for TempDirectory. This defaults to the Directory Analysis Services installs to as does the data directory. You can modify the location of both the data and temp directory to place them on drives that will have less IO issues. I recommend placing the Data Directory on it's own drive and the temp on a diffrent drive as well. You can have up to 2 temp drives both of which can live on diffrent drives. I like configs that look something like this
c Drive - OS and Apps
d Drive - Data Files
e Drive - TempDrive
f drive - tempdrive (if needed)

Processing of
SUM
MIN
MAX
COUNT
are done on disk. Whereas DistinctCount is processed in memory which can also cause performance problems when building and aggregating.


"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck