DW Vendors

[Rob999]

Hi everyone. I'm looking at beginning a DW project for CRM puposes primarily and wonder what those who've gone through the process think about some of the major vendors. I've used MS SQL Server 2000 to good effect, but my new company doesn't want to go that route. I'm in the Casino business and everyone's enamored with the AS/400 and DB2. Volume and scalability are big issues as we have a number of locations with almost 10,000 hotel rooms and over a million rated players with a terabyte of application data residing on our current application server.

Some of the choices we're looking at include using a new AS/400 with IBM's DB2 OLAP services. Also we're looking at Business Objects, Cognos Power Play and Hyperion Essbase. Additionally, we've seen vendors including Compudigm and are looking at a build vs. buy decision, or perhaps build the database and use one of the reporting tools mentioned above (which all do more than report now) to finish off the app.

If anyone has any input I'd like to hear it, especially if they've had first had experience with any of the above, or if they've looked at these and choose something else.

Thanks!

 

[avjoshi]

Rob,

To comment on your last paragraph; we can never generalize response times. Saying that it will take a certain amount of time to run queries in a warehouse system implies that all your queries are going to run in specific amount of time. It would be wrong on the part of that vender when they made that statement. It would also be wrong on your part to expect less than five second response time for every single query in a multi-terabyte environment. Properly created Relational Model in not any slower that a Multi-dimensional cube. The only benefit I see in some of the major Relational Databases over multi-dimensional is that they can handle large volume of data. Some of them also offer ROLAP to MOLAP continuity.

And when you say that MicroStategy or a Business Objects - don't subscribe to or build around the dimensional model did you mean physical-MOLAP implementation of dimension modeling? Because, what I remember working with them is they do subscribe to and can build around the dimensional model.

Remember Dimensional Modeling is a concept in Data Warehouse development methodology. Dimensional Model can be physically implemented in Relational Databases in way of Fact and Dimensional tables or cubes in MOLAP databases.


Anand

 

[Rob999]

avjoshi,
I think you misinterpreted a few things I said. First Vendors CAN and DO generalize a number of things when asked - it's part of making a sale. The person who made this particular comment on performance was a programmer on the system, so for the type of query he was running he knew how long it would take in most circumstances. All the vendors we have talked to provided technical personnel to answer those types of questions.

Regarding my expectations that every query take less than five seconds, I said "I know that some things will take a while regardless of which system you use...". I was talking about if the bulk of the queries take minutes each then it could take an executive an hour or more to investigate a series of questions - not what OLAP is supposed to be. Apparently the way around this is to throw more hardware at it - I don't know and that's why I asked.

And of course you can build a dimensional layout in a ROLAP vendor's warehouse. The question was can they take advantage of it ("use it maximum effect")? I'm curious with regards to performance on a large database when it comes to drilling down and drilling through large amounts of data where no preaggregation is being used.

 

[avjoshi]

Have you looked at TPC-H benchmarks? (

http://www.tpc.org/tpch/default.asp)

It might help you make some of the decisions.

Anand

 

[Rob999]

Not yet, but it is definitely on my ever increasing to do list!

 

[MDXer]

if you can get your company to fork out the fee OlapReport.com is a good source of info.

"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck

 

[entaroadun]

I'm a MSTR ROLAP guy, and I'm not too familiar with MOLAP implementations, but here goes...

I like MSTR because it has a clean delivery implementation. The product set has been organically grown, and all of the pieces, from the app server up, are integrated and easily customizable. I've evaluated Cognos and their implementation on the information delivery end, and it is not nearly as clean or as easy to custom code. If you will be assembling web portal apps, I would encourage you to look seriously at MSTR, simply because the staffing cost and time to market for web customization is so much lower.

As for performance, if you preagg an RDBMS the same way a MOLAP cube is preagg-ed, you will get similar performance. Yes, it will most likely be slower, simply because RDBMS's (regardless of what the sales people say) haven't been fully tuned for VLDB reads.

However, ROLAP solutions don't necessarily have to be tied to an RDBMS. ROLAP solutions generate SQL. You can easily point the SQL to your cube. If your cube doesn't support ANSI SQL (and who doesn't?) you can create views in a bridge RDBMS to make the cube look like a table. There is maintenace cost, but it's minor compared to what you'll be saving on the front-end.

MSTR also offers more sophisticated custom functions, since it isn't limited by the dimensional model. However, it does embrace the dimensional model fully; most generic implementations of MSTR are of a dimensional model rendered in an RDBMS, simply because that's what most DW consultants know how to do.

 

[jonhoward]

In the past I worked on a retail warehouse built on DB2/400 using MSTR. One of the SLA's was to have *any* report come back in <10 seconds, be it a precanned report or an adhoc report built using a report builder. It worked because there was a (comparatively) large amount of pre-aggregation done on the database. Basically, there was an aggregate for almost all possible query eventuality.

If you consider the trade-off triangle of report flexibilty vs query performance vs maintenance, this particular company decided that they would take the hit on maintenance so that performance would be optimal.

I wish you luck Rob999 )

 

[Rob999]

Thanks very much for the input!

Just how much work is involved in building preaggregations for all of these combinations of data elements selected for reporting / cubes? Is it - as I believed was mentioned here - a matter of looking at what has been run consistently and then building the common ones?

Also, am I correct in my assumption that the three primary ways to do this are to -

1. Build it all yourself.
2. Build it using vendor tools and their consultation (ie, Cognos).
3. Buy it from a vendor.

#1 seems ridiculous, unless your doing what I did with SQL Server, which has its limitations. This project is clearly too big for that.

#2 appears to be about selecting the right vendor for the job and this is how we're leaning.

#3 our execs want control and don't want a "turn key" approach which is very expensive and often requires you to wait for them for mods and extensions. This is not likely for us.

The vendor search process has been slowed a bit while we continue to gather the requirements and define the functionality. But I still appreciate any feedback on the major vendors. Thanks again!

 

[johnherman]

Rob999 - you and your execs are wise to avoid choice 3. Quite simply, if you buy your Data Warehouse or Data Mart off the shelf, you are getting no competitive edge over your competition (unless they don't have one). And believe me, you definitely want the opportunity to gain competitive advantage via your DW (even if your business folks can't figure out how).

Regarding choice 2. If you can find a single vendor who can provide what you need, you have once again failed to gain competitive advantage. The obvious problem with more than one vendor is that they tend to point fingers (blame) on each other when their components don't work together. Choosing vendors who are strategic partners with each other helps minimize this. For example. IBM and Data Stage. Informatica and Sun and Oracle. Microsoft and Microsoft.

Number 1 is the number 1 ultimate solution. However, if you do not have the in-house expertise in DW/DM, and don't have deep pockets to hire good consultants, you will quickly find yourself becoming a DW failure statistic.

Best of luck, and let us know what your execs decide.

Sometimes the grass is greener on the other side because there is more manure there - original.

 

[Rob999]

Thanks for the input JH.

Because of numerous competitive advantages we have, the Data warehouse may not be as much about "keeping up" with our competition, as it is staying ahead. So I don't know that building the industry's best data warehouse (it's not going to happen - too many others have 10's of millions invested and a decade or more head start) is our objective. We just want to better identify and serve our guests and a CRM application (with the correct business processes) should help us out there. As the industry becomes more and more competitive and the other properties get more sophisticated with respect to identifying the profitable customers and giving them they attention most people want, we need to step our efficiency up as well.

With respect to consultants, if we were to go with a major BI vendor (Cognos, Hyperion, MSTR, etc) how much help could we expect - or rely on - from them?? Is it industry standard that metrics and implementation ideas come with the ETL tool and dashboards?? Can we expect to "pump" a vendor for information, perhaps even in the sales/proof of concept phase?? Our users are good, but most of them have no experience with high end analytical work and it's easy to imagine them missing some key details that might be a pain to incorporate after the fact.

 

[johnherman]

With respect to consultants, if we were to go with a major BI vendor (Cognos, Hyperion, MSTR, etc) how much help could we expect - or rely on - from them??

A difficult question to answer. I would word the contract carefully so that they don't relegate you to low man on the totem pole after you buy their stuff. Most vendors have two separate consulting organizations; one for pre-sales and one for post-sales. The pre-sales people will bend over backwards until you sign on the line. This branch of the consulting organization is a money losing venture. The post-sales consulting, however, is purely profit driven.

Is it industry standard that metrics and implementation ideas come with the ETL tool and dashboards??

Most BI tools have some sort of EIS or dashboard to display KPI's, metrics, and other "balanced scorecard" measures. Identifying those metrics will be up to your executives and, other than some basic accounting and finance numbers, will be dependent on your industry. Examples of metrics: average price per pound, transportation cost per mile, etc.

Can we expect to "pump" a vendor for information, perhaps even in the sales/proof of concept phase??

Go for it! As I stated before, vendors will bend over backwards in the pre-sales phase of a project, so you can get lots of time for free. It also helps to have competition. For instance, even if you know you want to go with (say) Business Objects, letting BO know that Cognos and Hyperion are in competition will sweeten the free consulting you get. You can probably even "force" the vendor(s) into a proof of concept or prototype and get one of you applications built for free. Not only that, the vendor's consultants will probably be using their best practices as far as tool capabilities, so you will have a good template to build upon. In fact, you can request that as part of the contract/deal.

Good Luck with the vendors.

Sometimes the grass is greener on the other side because there is more manure there - original.

 

[MDXer]

Agreeing with John on the letting the vendors know who else you are looking at. I was tasked with running an RFI to possibly replace proclarity. When I got it down to the final 4 they all wanted to know who they were competeing against. Telling them was probably the best thing that happened, no longer were we just communicating with sales people and presales engineers, we were given access to Senior developers and district managers. BO & Cognos are highly competitve during my RFI BO & Cognos managers both swore they would not lose the ontract based on cost.

&quot;Shoot Me! Shoot Me NOW!!!&quot;
- Daffy Duck

 

[Rob999]

Hello again.

We're wrapping the requirements gathering stage (and my wife had our first baby a few weeks ago), so it's time to get back to round #2 of the BI vendors derby.

We've been pretty straightforward with each vendor about who else we were looking at and some of them had no hesitation to "dish dirt", so we are assembling a list of things to look for, and to look out for.

One of our analysts has arranged for he and I to go to a Seibel presentation - and according to Gartner they are the leaders in CRM among the SAP/Peoplesoft/Epiphany/Onyx crowd, which leads me to a question: What's the diff between this business space and the Cognos/Mstr/Hyperion segment? Is it that SAP offers a product that sits on (with their help) what you have, while Cognos offers tools that allow you to build what you need?

Once again, thanks again to all.

 

[MDXer]

When I was running an RFI much like yours I found some very useful questions to be.

1) Why do you feel your product is better than compititors product?
2) What are your products Strengths and weakneses
3) What are competitors strengths and weakneses
4) Who do you see as your main competitors.

The CRM systems such as Seibel and PeopleSoft are more desinged for management systems. They do come with some limited analytical capabilities. SAP I know has extensive Analytical capabilities and has modules for a number of vertical markets. Hyperion, Cognos, Business Objects are purely reporting systems. If your looking for DW and Olap reporting your better off staying with vendors of OLAP tools such as Cognos and BO.

&quot;Shoot Me! Shoot Me NOW!!!&quot;
- Daffy Duck

 

[Rob999]

Thanks MDXer,

We do have a very detailed set of questions we're asking. The vendors for the most part are hesitatnt to dish dirt on the opposition, but a lot of that seems to be based on a lack of info on their competitors. Of course, Microstrategy's website dished a lot of dirt, particularly on Cognos and Business Objects. I found it very informational. Plus a number of them have sent me selected excerpts from the Gartner Reports and the OLAP Survey.

We're still mainly in the Reporting tools space when it comes to looking at vendors. We also have two companies that are basically consultants offering custom modified solutions built around their products, and have just added Teradata and Siebel as possible choices - bring the current total to 10 possibilites!

If anyone has any info on them, or more commentary about the Seibel, Peoplesofts vs. Hyperion and Microstrategy - or any additional input, please let me know. Thanks!