Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations strongm on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
ETL Tools selection 10
- Thread starter ADB1
- Start date
-
2
- #21
THe best example is analysis services. In 2 releases they went from the entering the OLAP market to being Number 1, and from what I have seen and heard about yukon's olap offering it will continue to dominate the market. I doubt it will take 10 years 3-5 years depending on how long it is between yukon and future releases.
"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck
ok.. are talking ETL or BI then? there are many types of BI - 5 to be exact... OLAP is but one component of BI - M$ is #1 out ther in OLAP - where did you get this from... other products like Microstrategy, BO and Hyperion Essbase rule the market... BO being the biggest... M$ #1 in BI ....hmmm - can you qualify your information please?
regards,
Peter
regards,
Peter
The comment was not offered to demonstrate a fact about ETL but rather to support Microsofts history in entering a particular market. I'm sorry if you are unable to differentiate between fact, opinion and that offered to support ones opinion.
Here is a link to current market shares. Olap report is one of the top resources for olap and olap vndor related information. In return why don't you provide evidence of your statements that BO is the #1 in the BI market as-well-as Microsofts' inability to gain market share.
"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck
Here is a link to current market shares. Olap report is one of the top resources for olap and olap vndor related information. In return why don't you provide evidence of your statements that BO is the #1 in the BI market as-well-as Microsofts' inability to gain market share.
"Shoot Me! Shoot Me NOW!!!"
- Daffy Duck
I'm glad you posted the link; I'm wondering if you are able to read and understand what you post and base your opinions on... read on I have copied some info from your link:
"The larger vendors — Microsoft, Oracle, SAP, Business Objects — cannot even measure their OLAP business themselves, because their OLAP capabilities are often delivered as part of larger, bundled products. For example, Microsoft Analysis Services and the OLAP capabilities of SAP BW are not sold separately, but are included as part of product suites. "
-first point - difficult to measure because of a bundling of services... I guess analysis services comes with every version of SQL Server sold, whether they use it or not is another question... this skews the results... cause it is probably not being used... why? read on... (from your link again)
"Microsoft had its third consecutive year with no new OLAP product releases, though many of its partners delivered improved front-end tools which increased the attraction of the Microsoft OLAP platform. This meant that Microsoft continued to grow faster than the market, though at a slower rate than previously. Much the same is expected in 2004, with Microsoft again relying on its partners for product enhancements, with the major ‘Yukon’ release of SQL Server delayed into 2005 and Microsoft’s own weak OLAP front-end capabilities also not expected to be significantly upgraded during 2004."
So microsoft ain't doing anything.. no revision no nothing... probably cause many people are not using the featurs of analysis services so there is no need to upgrade.. hence false measurement in terms of OLAP market share... it's bundled in with the DB that people use; not anaysis services...
As far as the rest of the players go:
"The large non-specialist vendors, such as Microsoft and Oracle, are stronger in the rest of the world than the smaller BI specialists, who tend to be UNDER-REPRESENTED outside the major markets. By vertical market, MicroStrategy is particularly strong in retail, Applix in finance and insurance and Microsoft and Business Objects in the IT industry."
Again, the above is from your link; so it's important to see who is gaining market share... market share isn't gained by a product being sold in a bundle... it's gained by a product being used!!!
I hope you can see my point; once again the microsoft marketing machine out to prove to the world that it is gaining market share... maybe... but nobody uses their product; in this specific case - Analysis services... packaged with a bundle doesn't mean it's gaining market share... I've worked in many IT shops where they use SQL server and avoid analysis services cause it's a weak offering...
what do you have to say to that?
"The larger vendors — Microsoft, Oracle, SAP, Business Objects — cannot even measure their OLAP business themselves, because their OLAP capabilities are often delivered as part of larger, bundled products. For example, Microsoft Analysis Services and the OLAP capabilities of SAP BW are not sold separately, but are included as part of product suites. "
-first point - difficult to measure because of a bundling of services... I guess analysis services comes with every version of SQL Server sold, whether they use it or not is another question... this skews the results... cause it is probably not being used... why? read on... (from your link again)
"Microsoft had its third consecutive year with no new OLAP product releases, though many of its partners delivered improved front-end tools which increased the attraction of the Microsoft OLAP platform. This meant that Microsoft continued to grow faster than the market, though at a slower rate than previously. Much the same is expected in 2004, with Microsoft again relying on its partners for product enhancements, with the major ‘Yukon’ release of SQL Server delayed into 2005 and Microsoft’s own weak OLAP front-end capabilities also not expected to be significantly upgraded during 2004."
So microsoft ain't doing anything.. no revision no nothing... probably cause many people are not using the featurs of analysis services so there is no need to upgrade.. hence false measurement in terms of OLAP market share... it's bundled in with the DB that people use; not anaysis services...
As far as the rest of the players go:
"The large non-specialist vendors, such as Microsoft and Oracle, are stronger in the rest of the world than the smaller BI specialists, who tend to be UNDER-REPRESENTED outside the major markets. By vertical market, MicroStrategy is particularly strong in retail, Applix in finance and insurance and Microsoft and Business Objects in the IT industry."
Again, the above is from your link; so it's important to see who is gaining market share... market share isn't gained by a product being sold in a bundle... it's gained by a product being used!!!
I hope you can see my point; once again the microsoft marketing machine out to prove to the world that it is gaining market share... maybe... but nobody uses their product; in this specific case - Analysis services... packaged with a bundle doesn't mean it's gaining market share... I've worked in many IT shops where they use SQL server and avoid analysis services cause it's a weak offering...
what do you have to say to that?
With respect to selecting an ETL tool... you also need to consider the programming resouces that must accompany the tool... given the other tools have been out for many years; there are many programmers available and experienced to use them... IT is a consultant and vendor driven business... the people with experience will bring in the proven tools... NOT the tools on the perceived horizon that don't exist and can't deliver on mission-critical projects.
regards,
Peter
regards,
Peter
johnherman
MIS
I wonder why you even bothered to ask our opinions since you obviously seem to know everything already.
Sometimes the grass is greener on the other side because there is more manure there - original.
Sometimes the grass is greener on the other side because there is more manure there - original.
JOHNHERMAN regarding your opinions... they are not valid and that's why I respond to your emails; to clear things up so people who read this don't get side tracked by the inferior ETL and BI technologies that M$ currently has to offer! this isn't a pro-M$ forum... it's a an area for discovery - and thanks for posting the link to olapreport; that clearly shows who dominates a particular vertical markets.
Anyways, getting back to the ETL topic (and trying to avoid who will dominate in 3 years - is there another forum for that... John, maybe you'd like to create one!)
We have evaluated Hummingbird Genio, and it's a mid-weight ETL tool... great interface and metadata management but poor support for command line execution and management of internally built processes.
Also, examples documentation is limited and so are the transformation objects within the tool; no support for surrogate key generation and slowly changing dimension - these 2 scenarios must be dealt with programatically.
regards,
Peter
Anyways, getting back to the ETL topic (and trying to avoid who will dominate in 3 years - is there another forum for that... John, maybe you'd like to create one!)
We have evaluated Hummingbird Genio, and it's a mid-weight ETL tool... great interface and metadata management but poor support for command line execution and management of internally built processes.
Also, examples documentation is limited and so are the transformation objects within the tool; no support for surrogate key generation and slowly changing dimension - these 2 scenarios must be dealt with programatically.
regards,
Peter
-
2
- #29
There is a small New Zealand company out there with a product designed specifically for Data Warehouses called Wherescape.
Their Data Warehouse Tool is called red. Check out their site, I think you can download a demo to check it out.
We are using them for our data warehouse and I find it most effective.
Their Data Warehouse Tool is called red. Check out their site, I think you can download a demo to check it out.
We are using them for our data warehouse and I find it most effective.
For better or worse, this thread is all my fault as I asked my colleague ADB1 to raise the initial question for me.
Hopefully the above won't have put everyone off, because I wanted to ask icecore for a follow-up on his BO eval - how far have you got, icecore?
We're in the middle of one too. So far so good - the interface looks good and easy to use with decent traceability and re-use possibilities. We should have some (predicted) throughput volumes soon, of interest to Rob999, possibly. BO are wrestling with a (we think) tough POC for us at the moment - I'll let you know how this goes, if anyone's interested?
Richard.
Hopefully the above won't have put everyone off, because I wanted to ask icecore for a follow-up on his BO eval - how far have you got, icecore?
We're in the middle of one too. So far so good - the interface looks good and easy to use with decent traceability and re-use possibilities. We should have some (predicted) throughput volumes soon, of interest to Rob999, possibly. BO are wrestling with a (we think) tough POC for us at the moment - I'll let you know how this goes, if anyone's interested?
Richard.
Richard,
so far BO, first impressions of BO are favorable... I'm installing the unix engine this week to run some tests... I like the fact you only have to work with one interface to run and create jobs... it's a great setup.
I like the fact that it has real-time message capability out of the box and is included in the price...
Communicating with java processes can be challenging however, BO provided a webex showing how we would interface with a java routine and retrieve data record by record; after the presentation I asked them for the java code they demo'd and they would not give it to us! In essence BO's POC is essentially a signoff on them coming in and proving their product works to your requirements and if it does you have to buy it... I don't quite like that strategy when other vendors are willing to come in without a purchasing commitment on our end... I guess BO is using aggressive sales tactics... but as I said before so far so good with the BO eval.
We have Pervasive in this week to install and participate in our POC;
out of curiosity, what makes your POC so difficult? what are some of the difficult tasks you are asking of them?
Also, what are they charging you for the software?
regards,
Peter
so far BO, first impressions of BO are favorable... I'm installing the unix engine this week to run some tests... I like the fact you only have to work with one interface to run and create jobs... it's a great setup.
I like the fact that it has real-time message capability out of the box and is included in the price...
Communicating with java processes can be challenging however, BO provided a webex showing how we would interface with a java routine and retrieve data record by record; after the presentation I asked them for the java code they demo'd and they would not give it to us! In essence BO's POC is essentially a signoff on them coming in and proving their product works to your requirements and if it does you have to buy it... I don't quite like that strategy when other vendors are willing to come in without a purchasing commitment on our end... I guess BO is using aggressive sales tactics... but as I said before so far so good with the BO eval.
We have Pervasive in this week to install and participate in our POC;
out of curiosity, what makes your POC so difficult? what are some of the difficult tasks you are asking of them?
Also, what are they charging you for the software?
regards,
Peter
Peter,
Well, we think it's difficult, which may not be the same thing! We have a need to calculate the total duration of a number of events that may overlap, so our current (hand-coded) processing loops through, altering the start and end dates and times of the appropriately-ordered events until they don't overlap, calculates the resulting durations and them sums them to find the total duration. It's the iterative nature of the task that we're interested to see if DI can cope with or, as important, isn't impenetrable to re-visit for maitenance or updating.
They (BO UK) are quoting £85.5k plus £17k s&m ($133k, $26.5k resp), which seems a bit steep. We may be able to get it cheaper through our US parent company, though... Have you had any price info yet?
Richard.
Well, we think it's difficult, which may not be the same thing! We have a need to calculate the total duration of a number of events that may overlap, so our current (hand-coded) processing loops through, altering the start and end dates and times of the appropriately-ordered events until they don't overlap, calculates the resulting durations and them sums them to find the total duration. It's the iterative nature of the task that we're interested to see if DI can cope with or, as important, isn't impenetrable to re-visit for maitenance or updating.
They (BO UK) are quoting £85.5k plus £17k s&m ($133k, $26.5k resp), which seems a bit steep. We may be able to get it cheaper through our US parent company, though... Have you had any price info yet?
Richard.
ArtieChoke
Programmer
rmkg,
You want to be aware that price quotes are just the starting point - get a really tough negotiator to work with whomever you select - you will probably get a 1/3 to 1/2 reduction from the starting price - these guys are probably still hungry!
"I think we're all Bozos on this bus!" - Firesign Theatre![[jester]](/data/assets/smilies/jester.gif "[jester] [jester]")
You want to be aware that price quotes are just the starting point - get a really tough negotiator to work with whomever you select - you will probably get a 1/3 to 1/2 reduction from the starting price - these guys are probably still hungry!
"I think we're all Bozos on this bus!" - Firesign Theatre
Yes, Enterprise licence for Win2k OS and up to 4 processors.
We will be having a Dev & Test, yes, but the enterprise license allows for three instances.
Do you know the basic differences between the enterprise and departmental licenses?
And don't worry Artichoke, we'll haggle! Be nice to know what we're aiming for, though!
We will be having a Dev & Test, yes, but the enterprise license allows for three instances.
Do you know the basic differences between the enterprise and departmental licenses?
And don't worry Artichoke, we'll haggle! Be nice to know what we're aiming for, though!
-
1
- #38
I too am currently researching ETL tools for our DW running on Oracle 8i. My list includes Informatica, DataStage, SAS, and Cognos but I may add Oracle Warehouse Designer.
Currently we have no ETL tool, no metadata management, no licensed application development tool, and no modeling tool! The customers for this data use both Business Objects and Dataflux currently however they are in the process of evaulating Cognos for their needs. I was hoping to find an ETL package which would fit nicely with their choice but also be flexible for everyday ETL/Data Quality needs.
So far the Cognos offering seems a little too tied to its entire BI platform for what we are seeking. I used Informatica briefly at my previous position and really liked it, but their implementation required an administrator solely for that server full-time, something we really have no desire or budget for.
My feeling is that most of the popular ETL tools will be overkill for our smaller, more basic DW implementation so I'll be definitely checking out the lightweight tools mentioned in this article.
Currently we have no ETL tool, no metadata management, no licensed application development tool, and no modeling tool! The customers for this data use both Business Objects and Dataflux currently however they are in the process of evaulating Cognos for their needs. I was hoping to find an ETL package which would fit nicely with their choice but also be flexible for everyday ETL/Data Quality needs.
So far the Cognos offering seems a little too tied to its entire BI platform for what we are seeking. I used Informatica briefly at my previous position and really liked it, but their implementation required an administrator solely for that server full-time, something we really have no desire or budget for.
My feeling is that most of the popular ETL tools will be overkill for our smaller, more basic DW implementation so I'll be definitely checking out the lightweight tools mentioned in this article.
ArtieChoke
Programmer
Not sure why this was the case there, but we run 6.2 and don't need a "full-time" admin. Once it's set up properly, there's not much you need to do to keep it running.
"I think we're all Bozos on this bus!" - Firesign Theatre
ArtieChoke said:
He did all the test->stage->prod migrations of mappings/workflows/etc, new user setup, client installations, etc. I remember also asking him to refresh a few tables from different databases. I'm sure with as many sources/targets and projects as Nationwide Insurance has he was pretty busy.
There's always the possibility our DBA could handle those tasks since our team and DW is far smaller but I'm not sure.
How do you handle that at your organization?
- Status
Similar threads