On the subject of Data Warehouses, Data Cubes & OLAP....

[Will]

On the subject of Data Warehouses, Data Cubes & OLAP….

I would like to speak frankly about Data Warehouses, Data Cubes and
OLAP (on-line analytical processing). Has it dawned on anyone else
that these buzz words were created by some geek who decided to take a
stab at marketing? Knowing that to the backwoods manager who knows
little of technology that new innovative names for old concepts would
help to sale their products.

I mean seriously, what is the story here? In a nut shell, and please
stop me if you disagree, but isn’t a data warehouse simply a
database? Can’t you do everything on a conventional database
like SQL Server, Oracle or DB2 that you can do on these new
proprietary Data Warehouse constructs? I mean who are they trying to
fool?

Take a look, for instance, at Data Cubes. Who hasn’t noticed
the striking similarity between data cubes and views used in all the
more robust databases? Also, what about OLAP? OLAP is nothing more
than a report generator. There’s nothing you can do with these
million dollar price tagged Data Warehouse total solution packages
that I can’t do with SQL Server, Oracle or DB2…for that
matter Microsoft Access.

As an example some sales people for Metadata Corporation has the Vice
President of I.T. in Nashville, for Healthspring, sold on their total
solution data respository which is such a scam. All they had to do
was throw a couple of buzzwords at him and they have him hypnotized.

Personally, I feel that these kinds of marketing practices undermine
our industry. It helps to unravel what little standards or
consistency we have. What do you guys think?

Stuart

[John Keeley]

A very challenging post.
Nothing wrong with a bit of scepticism!

You are right about there being a huge amount of marketing hype.
Some people I suspect are not getting a return on their
datawarehouse/olap investment.
There are people (mainly sales people) in the Business Intelligence
industry who have little business intelligence.

However, despite saying this there is indeed a good case for having a
datawarehouse & olap.

Take a insurance company for instance.
They have a huge amount of data collected in their transactional
systems.
The money comes in over a certain time period & the money going out in
claims goes out over a longer time period.
Insurance companies can easily think they are doing better than they
really are unless they are analysing their data well.

The first point is the transactional data is usually in more than one
system - the data needs to be brought together.
The classic EXTRACT, TRANSFORM & LOAD.
This is no mean task bringing all the data together in a coherent way.
Often it is done wrongly, but the rewards are there if done well.
The reward can be a high as ensuring the survival of the business.
Sure, this datawarehouse is just a database, but it is a database
built for decision making.

And the second point is when making decisions, the analysis of the
data is done at a highly aggregated level - OLAP.
Yes, just another database, but a multidimensional one for speed
(pre-aggregates) & ease of end user navigation.

Many years ago when I first worked in a Management Information team we
just used spreadsheets for storing the data. Then we used an Access
database. Then we used OLAP; specifically TM1. The advantages were
big. The management could ask for just about anything & we could do it
in minutes. Now there is Microsofts SQL Server 2k & Analysis Services
bringing together the datawarehouse & OLAP. A powerful way of working.
Have you not looked at it?

I suggest you get a copy of Analysis Services & Reed Jacobson's book
"Analysis Services: Step by Step" play with it for a couple of weeks &
then see if you think ALL datawarehousing & olap is a con or not.

I wonder what brand of datawarehousing/olap you have come up against?
Name names!

Regards,

John

www.johnkeeley.com

Post by Will
On the subject of Data Warehouses, Data Cubes & OLAP….
I would like to speak frankly about Data Warehouses, Data Cubes and
OLAP (on-line analytical processing). Has it dawned on anyone else
that these buzz words were created by some geek who decided to take a
stab at marketing? Knowing that to the backwoods manager who knows
little of technology that new innovative names for old concepts would
help to sale their products.
I mean seriously, what is the story here? In a nut shell, and please
stop me if you disagree, but isn’t a data warehouse simply a
database? Can’t you do everything on a conventional database
like SQL Server, Oracle or DB2 that you can do on these new
proprietary Data Warehouse constructs? I mean who are they trying to
fool?
Take a look, for instance, at Data Cubes. Who hasn’t noticed
the striking similarity between data cubes and views used in all the
more robust databases? Also, what about OLAP? OLAP is nothing more
than a report generator. There’s nothing you can do with these
million dollar price tagged Data Warehouse total solution packages
that I can’t do with SQL Server, Oracle or DB2…for that
matter Microsoft Access.
As an example some sales people for Metadata Corporation has the Vice
President of I.T. in Nashville, for Healthspring, sold on their total
solution data respository which is such a scam. All they had to do
was throw a couple of buzzwords at him and they have him hypnotized.
Personally, I feel that these kinds of marketing practices undermine
our industry. It helps to unravel what little standards or
consistency we have. What do you guys think?
Stuart

[Will]

Funny you should mention insurance...cause I work for an HMO. We've
been talking to a company called Metadata Corporation I think. Anywho
their product is called Metadata Blah Blah. For the most part it is
NOTHING we don't already own. We have the SQL Server licensing and
two rough data warehouse models up and running. Instead of sinking
the two and sharing a link using a VPN the VP wishes to blow away all
that we have and start over from scratch. While he's at it he wants
to blow a million bucks!

How's that for marketing? The marketing reps throw some buzzwords at
him and now he's hypnotized into believing what they have to offer is
"magic". Don't know whether to laugh or cry ;)

What's your take? Are you in insurance?

Stu

Post by John Keeley
A very challenging post.
Nothing wrong with a bit of scepticism!
You are right about there being a huge amount of marketing hype.
Some people I suspect are not getting a return on their
datawarehouse/olap investment.
There are people (mainly sales people) in the Business Intelligence
industry who have little business intelligence.
However, despite saying this there is indeed a good case for having a
datawarehouse & olap.
Take a insurance company for instance.
They have a huge amount of data collected in their transactional
systems.
The money comes in over a certain time period & the money going out in
claims goes out over a longer time period.
Insurance companies can easily think they are doing better than they
really are unless they are analysing their data well.
The first point is the transactional data is usually in more than one
system - the data needs to be brought together.
The classic EXTRACT, TRANSFORM & LOAD.
This is no mean task bringing all the data together in a coherent way.
Often it is done wrongly, but the rewards are there if done well.
The reward can be a high as ensuring the survival of the business.
Sure, this datawarehouse is just a database, but it is a database
built for decision making.
And the second point is when making decisions, the analysis of the
data is done at a highly aggregated level - OLAP.
Yes, just another database, but a multidimensional one for speed
(pre-aggregates) & ease of end user navigation.
Many years ago when I first worked in a Management Information team we
just used spreadsheets for storing the data. Then we used an Access
database. Then we used OLAP; specifically TM1. The advantages were
big. The management could ask for just about anything & we could do it
in minutes. Now there is Microsofts SQL Server 2k & Analysis Services
bringing together the datawarehouse & OLAP. A powerful way of working.
Have you not looked at it?
I suggest you get a copy of Analysis Services & Reed Jacobson's book
"Analysis Services: Step by Step" play with it for a couple of weeks &
then see if you think ALL datawarehousing & olap is a con or not.
I wonder what brand of datawarehousing/olap you have come up against?
Name names!
Regards,
John
www.johnkeeley.com

Post by Will
On the subject of Data Warehouses, Data Cubes & OLAP….
I would like to speak frankly about Data Warehouses, Data Cubes and
OLAP (on-line analytical processing). Has it dawned on anyone else
that these buzz words were created by some geek who decided to take a
stab at marketing? Knowing that to the backwoods manager who knows
little of technology that new innovative names for old concepts would
help to sale their products.
I mean seriously, what is the story here? In a nut shell, and please
stop me if you disagree, but isn’t a data warehouse simply a
database? Can’t you do everything on a conventional database
like SQL Server, Oracle or DB2 that you can do on these new
proprietary Data Warehouse constructs? I mean who are they trying to
fool?
Take a look, for instance, at Data Cubes. Who hasn’t noticed
the striking similarity between data cubes and views used in all the
more robust databases? Also, what about OLAP? OLAP is nothing more
than a report generator. There’s nothing you can do with these
million dollar price tagged Data Warehouse total solution packages
that I can’t do with SQL Server, Oracle or DB2…for that
matter Microsoft Access.
As an example some sales people for Metadata Corporation has the Vice
President of I.T. in Nashville, for Healthspring, sold on their total
solution data respository which is such a scam. All they had to do
was throw a couple of buzzwords at him and they have him hypnotized.
Personally, I feel that these kinds of marketing practices undermine
our industry. It helps to unravel what little standards or
consistency we have. What do you guys think?
Stuart

[Jorge_Beteta]

Very interesting discussion. Also I'm just looking for answers of what
a datamart really is. If somebody could just take a look of my previos
question
"Am I correct of what a datamart is?"
http://groups.google.com/groups?dq=&hl=en&lr=&ie=UTF-8&group=comp.databases.olap&selm=5140b91e.0310141301.5c0920ac%40posting.google.com

I guess nobody did it :(

I think I have in some way the same interrogants. I got an answer I
would like to share:

"The reason the data mart is constructed differently than a
transactional system is because it's usage is different.

Basically, it's an off-line or near-line system where data is dumped
for the purpose of post-transaction processing. This could include
mining for fraudulent or erroneus transactions, aggregate and
statistical report processing, or sharing data with a class of users
that are not colocated (or even part of the same organization) and
don't require online data access. It's easy for these operations to
take a lot of system resources.

The DBMS resources used can also be quite different between TP and
analytical processing and are such tuned differently (ie procedure
cache vs data cache). The hardware itself (Raid 0 vs Raid 5) may be
setup differently if the database is read-only. Creating a separate
system for these uses solves a lot of potential problems.

The data structures themselves are sometimes no longer the same.
While it's the case that one could dump the data in its source
structure, it is often better to precalculate formulas and store
computed aggregates. Management reports often are looking for sum of
sums type data or cross-tabs.

RE 1. Users making queries w/o programmer assistance is more of a goal
than a indicator of whether a system is a data mart. Any system can
have an application or middleware running that pre-processes user
queries and converts them to SQL. Ultimately, the query mecahnism
ends up being similar or the same for both. Sometimes the interface
from the data mart is a dump into yet another system.

RE 2. I think that is essentially the case in AP systems. Some data
is not transferred, some data is copied wrote, and some is computed.

RE Questions:
2) You could have a data mart that accesses another data mart, or a
scientific system where the data wasn't transactional, but started off
life as a mass of observations."

"I also would clarify by indicating that a Data Mart is going to be a
more highly indexed subset than you would normally want your actual
data itself to be.

The concept seems to have emerged from the n-tier design concept where
you would ideally not want users all hitting your main data, but
prefer they hit a nearly-realtime highly indexed precompiled set of
views from an intermediate source whenever possible."

Jorge

[John Keeley]

Stuart,

The great irony in all this is there is little rationality in the
decision to buy decison support systems, when rationality is what a
decision support system is all about.

Such is the nature of humanity!

I previously worked in insurance & had to put up with an IT director
making statements like "I can't see any use for cubes" & "I want to
move away from analysts using Excel".

What do you do?
I just carried on doing what I knew was right.

Maybe you should try spreading the message that you can do just what
this expensive software can do at a fraction of the cost. Assuming you
are confident you can.

Indeed, SQL Server 2k, Analysis Services & a decent Excel add-in can
do most of what is needed.

I work for a consultancy called MIS - maybe we can convince you IT
director.
Look at www.misag.com

Regards,

John

Post by Will
Funny you should mention insurance...cause I work for an HMO. We've
been talking to a company called Metadata Corporation I think. Anywho
their product is called Metadata Blah Blah. For the most part it is
NOTHING we don't already own. We have the SQL Server licensing and
two rough data warehouse models up and running. Instead of sinking
the two and sharing a link using a VPN the VP wishes to blow away all
that we have and start over from scratch. While he's at it he wants
to blow a million bucks!
How's that for marketing? The marketing reps throw some buzzwords at
him and now he's hypnotized into believing what they have to offer is
"magic". Don't know whether to laugh or cry ;)
What's your take? Are you in insurance?
Stu

Post by John Keeley
A very challenging post.
Nothing wrong with a bit of scepticism!
You are right about there being a huge amount of marketing hype.
Some people I suspect are not getting a return on their
datawarehouse/olap investment.
There are people (mainly sales people) in the Business Intelligence
industry who have little business intelligence.
However, despite saying this there is indeed a good case for having a
datawarehouse & olap.
Take a insurance company for instance.
They have a huge amount of data collected in their transactional
systems.
The money comes in over a certain time period & the money going out in
claims goes out over a longer time period.
Insurance companies can easily think they are doing better than they
really are unless they are analysing their data well.
The first point is the transactional data is usually in more than one
system - the data needs to be brought together.
The classic EXTRACT, TRANSFORM & LOAD.
This is no mean task bringing all the data together in a coherent way.
Often it is done wrongly, but the rewards are there if done well.
The reward can be a high as ensuring the survival of the business.
Sure, this datawarehouse is just a database, but it is a database
built for decision making.
And the second point is when making decisions, the analysis of the
data is done at a highly aggregated level - OLAP.
Yes, just another database, but a multidimensional one for speed
(pre-aggregates) & ease of end user navigation.
Many years ago when I first worked in a Management Information team we
just used spreadsheets for storing the data. Then we used an Access
database. Then we used OLAP; specifically TM1. The advantages were
big. The management could ask for just about anything & we could do it
in minutes. Now there is Microsofts SQL Server 2k & Analysis Services
bringing together the datawarehouse & OLAP. A powerful way of working.
Have you not looked at it?
I suggest you get a copy of Analysis Services & Reed Jacobson's book
"Analysis Services: Step by Step" play with it for a couple of weeks &
then see if you think ALL datawarehousing & olap is a con or not.
I wonder what brand of datawarehousing/olap you have come up against?
Name names!
Regards,
John
www.johnkeeley.com

Post by Will
On the subject of Data Warehouses, Data Cubes & OLAP….
I would like to speak frankly about Data Warehouses, Data Cubes and
OLAP (on-line analytical processing). Has it dawned on anyone else
that these buzz words were created by some geek who decided to take a
stab at marketing? Knowing that to the backwoods manager who knows
little of technology that new innovative names for old concepts would
help to sale their products.
I mean seriously, what is the story here? In a nut shell, and please
stop me if you disagree, but isn’t a data warehouse simply a
database? Can’t you do everything on a conventional database
like SQL Server, Oracle or DB2 that you can do on these new
proprietary Data Warehouse constructs? I mean who are they trying to
fool?
Take a look, for instance, at Data Cubes. Who hasn’t noticed
the striking similarity between data cubes and views used in all the
more robust databases? Also, what about OLAP? OLAP is nothing more
than a report generator. There’s nothing you can do with these
million dollar price tagged Data Warehouse total solution packages
that I can’t do with SQL Server, Oracle or DB2…for that
matter Microsoft Access.
As an example some sales people for Metadata Corporation has the Vice
President of I.T. in Nashville, for Healthspring, sold on their total
solution data respository which is such a scam. All they had to do
was throw a couple of buzzwords at him and they have him hypnotized.
Personally, I feel that these kinds of marketing practices undermine
our industry. It helps to unravel what little standards or
consistency we have. What do you guys think?
Stuart

[Stephan Eggermont]

Post by John Keeley
Stuart,
The great irony in all this is there is little rationality in the
decision to buy decison support systems, when rationality is what a
decision support system is all about.

Well, most decision support systems are bought because someone feels
he doesn't have enough control over what is going on in the
business. An interesting question to ask that someone is why the
current systems don't provide the needed information, and how the
decision to build/buy them was taken. I haven't seen much rationality
there.

Stephan

[Mike Sherrill]

Post by Will
Has it dawned on anyone else
that these buzz words were created by some geek who decided to take a
stab at marketing?

Lots of us. Software is not like a technology industry; it's like a
fashion industry.

Post by Will
I mean seriously, what is the story here? In a nut shell, and please
stop me if you disagree, but isn’t a data warehouse simply a
database?

More like a report than a database.

Post by Will
Can’t you do everything on a conventional database
like SQL Server, Oracle or DB2 that you can do on these new
proprietary Data Warehouse constructs?

Yes and no. You can drive screws with a hammer, but it's not always
the best approach.

[Hans Forbrich]

Post by Will
On the subject of Data Warehouses, Data Cubes & OLAP….
I would like to speak frankly about Data Warehouses, Data Cubes and
OLAP (on-line analytical processing). Has it dawned on anyone else
that these buzz words were created by some geek who decided to take a
stab at marketing? Knowing that to the backwoods manager who knows
little of technology that new innovative names for old concepts would
help to sale their products.

While you have a small point with this - check the history of the
terminology behind "OLAP, OLTP, Cubes and Warehouses" before you spout
in this particular topic..
Part of the reason for the 'new terms' (which IIRC are roughly 15 years
old) is because of different lineage (Cubes did not oringally get stored
in a relational database) and a need to differentiate the design methods
(OLTP applications and OLAP applications have basic design differences
in order to meet business requirements.)

Post by Will
I mean seriously, what is the story here? In a nut shell, and please
stop me if you disagree, but isn’t a data warehouse simply a
database? Can’t you do everything on a conventional database
like SQL Server, Oracle or DB2 that you can do on these new
proprietary Data Warehouse constructs? I mean who are they trying to
fool?

Now you can "do everything on a conventional database". 5 years ago was
a different story.

Post by Will
Take a look, for instance, at Data Cubes. Who hasn’t noticed
the striking similarity between data cubes and views used in all the
more robust databases? Also, what about OLAP? OLAP is nothing more
than a report generator. There’s nothing you can do with these
million dollar price tagged Data Warehouse total solution packages
that I can’t do with SQL Server, Oracle or DB2…for that
matter Microsoft Access.

Which is a credit to some fairly serious skull sweat that's gone into
consolidating the different technologies.

Post by Will
As an example some sales people for Metadata Corporation has the Vice
President of I.T. in Nashville, for Healthspring, sold on their total
solution data respository which is such a scam. All they had to do
was throw a couple of buzzwords at him and they have him hypnotized.

Now you are getting to the meat of your email! I get the feeling your
issue really is that some management styles (some specific managers)
that reinforce the reality of Peter's Principle. Incompetence has it's
own rewards.

Post by Will
Personally, I feel that these kinds of marketing practices undermine
our industry. It helps to unravel what little standards or
consistency we have. What do you guys think?

I think the marketeers are doing what they are paid to do. So are the
sales guys. Doesn't mean I have to like their style or tactics.
However, those styles and tactics can yield great rewards - just look at
good ol' Microsoft (amongst others). And just like Pavlov, the industry
keeps rewarding and reinforcing some specific behaviours. C'est la vie!


As someone once said "You always have a choice. If you don't like it,
you can choose to leave"

Post by Will
Stuart