DW Vendors

[Rob999]

Hi everyone. I'm looking at beginning a DW project for CRM puposes primarily and wonder what those who've gone through the process think about some of the major vendors. I've used MS SQL Server 2000 to good effect, but my new company doesn't want to go that route. I'm in the Casino business and everyone's enamored with the AS/400 and DB2. Volume and scalability are big issues as we have a number of locations with almost 10,000 hotel rooms and over a million rated players with a terabyte of application data residing on our current application server.

Some of the choices we're looking at include using a new AS/400 with IBM's DB2 OLAP services. Also we're looking at Business Objects, Cognos Power Play and Hyperion Essbase. Additionally, we've seen vendors including Compudigm and are looking at a build vs. buy decision, or perhaps build the database and use one of the reporting tools mentioned above (which all do more than report now) to finish off the app.

If anyone has any input I'd like to hear it, especially if they've had first had experience with any of the above, or if they've looked at these and choose something else.

Thanks!

 

[tdatgod]

There was an article in the October 2003 issue of Casino Journal about Harrah's and how they use CRM.

http://www.casinojournal.com/cjarchives/Oct2003/solutions1.htm

---

 

[MagicThighs]

Rob,
I'm building a DW in the UK. We went through a major
eval and chose Oracle on Solaris. Start schema based,
it demonstrated scalability well into the 10s of Tb
space and has an excellent toolkit.
The down side is that you would probably encounter
major culture shock moving from an DB2, AS400 mindset.
Good luck,
Don.

 

[Rob999]

Thanks for the response guys. Excellent link, and coincidentally, I have the actual magazine and Compudigm, the company in the article working with Harrah's has made two presentations for us - but I don't know that enough of the exec's were convinced. Great visualization tool, particularly for the gaming floor, but we're looking for something more enterprise wide - Hotel, Point of Sale, Gaming, Conventions, etc. That's why we're likely going to build something - hopefully for less than the $120 Mil Harrah's has accumulated over the years. But apparently it's been worth it for them.

Yes, "MagicThighs" - I'm afraid Oracle would probably be too big a culture shock for the group - but it's early on, and we're looking at a number of things still.

 

[MDXer]

Not presenting a solution because the vendor isn't "widely adopted" is not a valid reason to not propose that solution. When you are considering an enterprise solution ROI should be the key focus.

The first step should be to gather information on all implementations that can support your implementation. With a huge investment in the detailed technical aspects: Security considerations,
Storage modes,
Transactions Per Minute,
Cost of Transactions Per minute.

http://www.tpc.org/

is a great site to get benchmark comparisons.

To fairly represent the project the results should be given in a way that rates each platform according to benefits versuses cost. The software platform should not be the focus. The DW is a combination of Hardware and software. Server specs should be included as well as Storage systems.

Other elements that should be considered is talent pool. Development time.

the deciding factor should be ROI.

As for a possible change of platforms I don't see this as an issue as some of the Largest Hotel Casino Resorts in the country have already started migrating off AS400.

I have no doubt that Any of the major vendors can handle the data volume you are looking at, but is it worth the cost? A combination of technologies may also prove to be best. I was involved in a DW initiative that was terabyte size. We utlisized the MS products and had a production quality system that performed as required, was scalable and responsive. New managment came in with a very blatant Anti MS attitude and decided to change platforms well that was less than a year ago and the company no longer exists.

"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck

 

[Rob999]

Hi MDXer!
Not sure where the quotes from "widely adopted" came from - I don't think anyone said that. If you're saying that I should present ALL practical solutions - that isn't practical where I'm at now due to time constraints. We're just looking at major BI reporting vendors as well a couple of products such as the aforementioned Compudigm.

Your advice is very sound and much appreciated - I too have implemented a Microsoft DW utilizing SQL Server 2000 at my prior position (in fact you helped me with a question I had on that and got a star for it quite a while ago!) which pretty much got me hired here a few weeks ago. But there is a perception that exists here that we need a major player for safety's sake (ie, Cognos, Business Objects, Hyperion, IBM, etc) and where my MS warehouse cost about $35,000 ($30k server was primary expense) this will likely be a 7 figure project - which is what the executives at my last place said the SQL Server 2000 warehouse was easily worth.

Because I've only been here a short time I'm inclined to go with the flow and investigate the aforementioned selected major players because that was my directive. Also I won't be the developer on this one, this is clearly a safety first department, our competition is spending or has spent millions on BI/CRM - and they aren't as profitable as we are or growing at anywhere near the same pace.

As you said, we are looking at the AS/400 because that's pretty much our talent pool - but IBM has the least developed product offering so the others are going to get a good look as well. By looking at about 6 or 7 choices (between vendors and products and the combinations of that mix) I think we can come up with something good. Once again, thanks for the input!

 

[MDXer]

the "widely adopted" came from my knowledge of the casino industry in the location I believe you to be located.

just be aware that
BO is primarliy ROLAP I doubt you will get acceptable reporting from a VLDB with ROLAP technologies,
Hyperion is MOLAP but Builds Nulls into the dataset inflating the actual Cube size (as of my last discussions with someone well versed in Hyperion)
As of a year ago IBM was using Essbase. Of the ones your looking at I would say Cognos would be your best bet.

"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck

 

[Rob999]

Thanks MDXer. I'm getting ready to call these vendors and I will slyly insert some of that info into the questions (I won't tell them you told me though!).

 

[avjoshi]

How do you know one of them is not reading this post? ;-)

Just kidding.

Best of luck with your pursuit.

Anand

 

[johnherman]

As someone with experience on well over a dozen DW projects, I do not recommend warehousing with the AS/400. DB2/UDB for Unix would be better if you are enamoured with IBM. I am not enamoured with IBM and believe their DW solution is a bunch of piecemeal products held together by IBM's reputation. In addition, IBM is way more expensive than the others, both as an initial purchase and for ongoing maintenance (including staffing).

For DW using RDBMS, I recommend Oracle or SQL Server. Yes, SQL Server is scalable for your 1+ Terabyte DW. I just finished a gig for a health insurance company (100+ million claims, 10+ million members, 5 years history, etc) and about 3TB on SQL Server. Oracle and SQL Server seem to have a better integrated solution than IBM. I know some IBM employee or zealot will be on my case for these remarks, but that's based on my experience. Just the facts, Jack.

As far as using MDBMS (Multidimensional Database versus Relational), I think Hyperion is a good idea. BTW, IBM's OLAP solution is to sell you the Hyperion product and mark up the price. There are other players in the MDBMS arena, too, such as Teradata.

As far as front-end, I would recommend a series of dog and pony shows for your end user community to determine which vendor has the look and feel and ease of use your users want with their front end. Sophisticated users will opt for Microstrategy and Business Objects while users desiring a simple solution will choose Brio. Cognos falls in between. The problem with the simple Brio solution will arise when the users want something the product cannot deliver. The other products have the ability to customize your solution somewhat by writing some scripts, queries, etc.

For further information, or to hire me to assist with your platform evaluation.............. ;-)

Good Luck

Sometimes the grass is greener on the other side because there is more manure there - original.

 

[Rob999]

Thanks JH.
Yes, SQL Server does have something of a reputation problem, especially in the AS/400 community - it's not even being considered seriously. But I think that some of them are starting to see that the IBM approach has a number of problems, price being a big hurdle (not that any of them are "cheap").

The Hyperion Rep pointed out that IBM uses their product - and yes, they are financially compensated when IBM makes an OLAP sale according to him. But it is comforting to some guys here to know that the product can actually sit on the 400, versus a PC type server that everyone else uses.

As far as the front end goes, we're going to do the dog and pony thing but most of these guys are now offering ETL tools that may include some type of graphical modeling feature for setting up a star schema destination as well as the user interfaces. They're also big on consulting services and "knowledge transfer" regarding their products and the process. So it's very possible we'll go with Hyperion, Cognos or Business Objects based on their one stop solution - according to the sales reps anyways.

Speaking of Brio, it looks like Hyperion just purchased them within this past year (and in response Business Objects purchased Crystal Decisions - or was it the other way around??). The Hyperion rep was unable to give me much of a price estimation because they now have a number of products so we're going to have a follow up call to give them some of our data specifics. But it looks like we'll be meeting with IBM, Bus Objects, Hyperion and probably Cognos, whose rep hasn't called us back yet.

Thanks again everyone - if any one has any additional input I'd love to hear it. And if the thread keeps going, I'll let the group know what we've decided to do. A decision should be within a couple of months - hopefully.

 

[MDXer]

YOu may want to ask Hyperion what they use as their backend. Last I knew the IBM HYperion Relationship went both ways. Meaning if you bought an inclusive DW OLAP PAckage from IBM you recieved DB2 and EssBase, if you Contracted with Hyperion for a DW OLAP Solution you recieved DB2 and EssBase. Hyperion does have the ability to sit on top of other RDMS platforms. I haven't worked with Oracle RDBMS or their BI Tools but have read and done research on them and They are not as full featured as people believe especially in their MOLAP offering.

"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck

 

[Rob999]

MDXer - thanks, I will ask about that.

Also, we're now considering MicroStrategy as well. Major player, good CRM features, it's been mentioned by some of the people I've talked to (including here) and honestly - they're website doesn't suck like most of the others. A lousy website from a BI company is not a good sign in my admittedly less than fully informed opinion. So if anyone has any info on them as well as the others, please feel free...

 

[Z3]

Sure - I will comment on MicroStrategy as your requested. MS is primarilly a ROLAP company. Technically they are the best in BI. They are kinda on the edge and define all the rest of BI technically, but sometimes they lost focus and don't know where to go next. So it's a bit risky. If you are looking for Web deployment, have high scalability requirement, over 100 GB of data, complicated reporting /analytical requirement, definitely check MS out. Unlike Cognos, MS is primarily an organic software company, means all their softwares work with each other. MS is the true platform company, backend IT people like it a lot because it is more flexible and can fit in a lot of applications. Price-wise, MS is the less expensive to own than Cognos, but sometimes you can get almost free BO software if you get BO bundled. MS uses virtual cubes/relationship, so it's more flexible, but can be a little tricky to set-up correctly.

So it really depends on what you want with your warehouse. If you have very little data and you know what you want and data won't grow much and doesn't change much, go for Hyprion or Cognos, or even Microsoft OLAP. If you are looking for high end Web deployment with a lot of data and some complicated analysis and perhaps many projects beyond current one, go check out MicroStrategy. If you are somewhere in the middle, and can get a major discount on BO for free, go with BO. They all work pretty well, personally I like them all, but depends really on your needs.

 

[Rob999]

Thanks for the input Z3. Yes, we are looking at a "huge" amount of data (at or slightly more than 1 terabyte - which will grow quickly and will close to double in 18 mos when a new tower goes online). The idea of an organic company in an environment when everyone I'm talking to just bought a new reporting company and are "working on" integrating their products sounds pretty good. Much of MicroStrategy's web site pounds on how their products integrate seamlessly and you don't need to write everything twice because the tools don't talk to each other.

One question for anyone though - with these ROLAP companies do you need some kind of "super server" to make up for the lack of preaggregated cubes and the relational layout? A couple of the companies use this and I don't yet know if it's a real performance disadvantage for multi dimensional analysis or not...

 

[Z3]

Most companies have enough hardware to support ROLAP, with some degree of aggregation or optimization. MS is pretty much a pure ROLAP shop, and their product generate only SQL to the database via ODBC. So Materialized View would work, other performance enhancing software/hardware will work as long as it supports SQL from ODBC (who won't?). You can manage you own aggregate tables as many MS users do. One good thing about MS is, it's aggregate aware. Means after you made a new aggregate table, you just drag it in to MS schema and update the catalog, and it will start using it right away without restarting service and so on. Same is true for taking the aggregate table away, or changing it. It's kinda like you can build your own cubes in the WH, if you want. MS can also do some small scale of MOLAP on their server, but it's childish compare to Cognos or Hyperion.

 

[MDXer]

ROLAP will always be slower than MOLAP. I haven't used MicroStrategy but was invited to their user conference last year in Las Vegas. I liked the integration aspect and it seemed to some degree to be intuitive, but after seeing demos I couldn't accept the ROLAP performance.

One demo I witnessed was giving by a company that dealt in the same type of data my company did infact part of their data was included in our data. There DW was 4,000,000 - 5,000,000 records annually my companies was 1.3 - 1.7 billion. We had similiar reports that the company using Microstrategy would take 4+ hours to run, where our system ran the same reports under MS AS in less than 1 minute.

IF your DW is going to be used for occasional reporting then this type of perfrormance may be acceptable but if this is to be a true Business Intelligence tool then I would say it isn't. One of the Senior members of Aspirity has said something along the lines of "Business Intelligence is Analysis at the Speed of thought". If it takes 3 hours to run a report that you may then need to further analyze then you may be able to accomplish 3 or 4 levels of analysis in a day. Which in my opinion isn't acceptable.

"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck

 

[avjoshi]

I would say it purely depends on the design of the warehouse database. You can achieve he same degree of efficiency with aggregate tables on ROLAP databases. Moreover, ROLAP design will allow you build multiple aggregate tables off of the base STAR design, depending on your reporting needs.
I don't pretend to know much about MOLAP databases and certainly not a person to comment of their performance. But I know that most of the prominent ROLAP database systems inherently handle very large amount of data and are highly scalable.
In my humble opinion, ROLAP databases are suitable to build corporate information repository and MOLAP for the business specific reporting.


Anand

 

[MDXer]

avjoshi

I do agree with what you say. In fact you kind of bring light to something that in my opinion causes a lot of DW/BI initiatives to Fail or grow into nightmares. This is the lack of definition, and specifications. I believe that many companies do not fully understand or communicate what it is they want and expect from this type of implementation. I think 2 of the last things to be considered should be hardware and software. You really can't predict what you need until you know the scope of the project, and then you need to allow the time to develop the system, in a manner that allows for current needs and growth.

I have seen to many people develop with the attitude this is what we need right now so lets just do that. Only to have it blow up when someone say but we need it to do XYZ as well and in 6 months the data will double.

I think ROLAP and MOLAP both have their places, but you don't know what best suits yours without research and requirements.

"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck

 

[Rob999]

I agree there are places for both as well. However the point that MOLAP is good for one thing, ROLAP for something else is moot if the vendor is committed to one exclusive of the other. If it's a MOLAP vendor, it's an easy matter to set up your own 'Data Store', use tables from a staging area, or set up your own specific relational tables to optimize a few high use queries that aren't multi dimensional.

But conversely, can you set up a MOLAP star schema and use it to maximum effect when the vendor - like a MicroStategy or a Business Objects - don't subscribe to or build around the dimensional model?

Also, it appears we're going to build with scalability in mind, so we will invest in a server that's going to be able to handle a VLDB right out of the gate - but with that size database performance remains a concern. One of the demos we saw - a company that uses a relational model - says that 2 to 3 minutes is not a long time to wait for a query. That's not what I thought OLAP was supposed to be and I know some executives who will start hitting the escape key after 45 seconds. I know that somethings will take a while regardless of which system you use, but if the relational model's slower, why compound the problem? What's the benefit?