Exploration Warehouse 1

[loulou01]

We have recently developed a data warehouse which has both;
1. fully normalised data
2. dimensional data

Currently we are using Cognos for Management trend analysis type reporting. Although we do not yet perform data mining, we intended to (with the purchase of data mining tools ) eventually use our data warehouse for this purpose.

However, I have just finished reading an article by Bill Inmon "Information Management: Charting the course: What a data warehouse is not " which indicates that for large amounts of data, data mining requires an Exploration warehouse which has a different structure than a data warehouse.

Can anyone provide some explanation of what the structure of an Exploration warehouse is ?

 

[DiegoErnesto]

Hi;

In my experience, a data warehouse isn't fully normalized, the structure is a star scheme, a fact table and nodes around this.

This schema is the more appropriate and facilitates queries eliminating the joins between tables.

I hope to satisfy your question.

Diego Ernesto.

Cognos have good tools for that... Good luck...

 

[blom0344]

Very probably next to the information you gathered about why datamining should be done on another structure than a classic datawarehouse is a document by the guru himself about the subject:

http://www.billinmon.com/library/whiteprs/whitecrs.pdf

I personally suspect that the upperchiefs in the field are just as vulnerable to a bit of hyping to keep themselves occupied. If a trend is really strong enough to matter in business terms you will also stumble over it with a datawarehouse structure if you can get those commercial guys to state exactly what they are interested in (which in my opinion is the bottleneck in the entire process)

look at : thread354-90013 in the datamining forum T. Blom
Information analist
Shimano Europe
tbl@shimano-eu.com

 

[Guest_imported]

loulou01
STOP!!!
Do NOT listen to that Exploration warehouse concept that Inmon is pushing... without learning more about the alternatives. Our friend above (Blom) had a great comment ~ the guru's are looking for ways to sell books.

This exploration concept is bogus (in my opinion). Basically, it says that you should build yet another physical structure for data mining analysis. Another "movement" or "extraction" of data... duplicated data that you have to maintain & support Why do this? Why encur more cost? Why not "mine" the data inside the database? Inside your warehouse? Why not let the database do the heavy processing for you? Let the tools like SAS, SPSS, TeraMiner & others do "the work" inside the RDBMS.

I must also admitthat I disagree with DiegoErnesto too. StarSchema's limit organizations ability to change. They are modeled after expect business queries and not after the general business model. The "hub & spoke" model is dying and more 3NF data warehouses are on the up. Why? Well, both the Star Schema & even Inmon's "Exploratio Warehouse" were developed because the underlying database can't handle 3rd Normal form model. These architectures (StarSchema & Exploration Warehouse & DataMarts) are workarounds to avoid joins & complex queries that most business users would prefer.

There are databases designed specifically for data warehousing that operate in full parallel environments that can scale to meet any of these needs. I suggest you talk to the analysts like Gartner or Meta for more details. The database is an extremely important factor in determining if you need all these "workarounds" for data mining.

 

[loulou01]

Thanks for the tips on exploration databases. I will definately keep your comments in mind.

As to star schema's (dimension models) no longer being needed due to advances in data warehouse databases to handle 3NF, doesn't it depend to some degree on the tools being used to report on this data and the type of end user (expert or not).

Our non expert end users are currently using COGNOS Impromptu 5.0 & Powerplay 6.5, which use the star schema's.