What is BIG DATA? 3

[johnherman]

What is big data? What makes it bigger than any data we've had before? What's the "big deal" about it - why is it such a subject of interest? Or is it just one of the latest buzzwords?

I'm being partly humorous, but I really don't get it. Could someone explain the concept and why it's such a hot topic?

The trouble with doing something right the first time is that nobody appreciates how difficult it was - Steven Wright

 

[BigRed1212]

If you are talking about super enourmous large data sets I think it is a hot topic because there are a growing number of these sets (pun?) and because there is concern that current tools, structures, and ways of doing things may not work well at that scale.

As for what makes things bigger than anythig before, I think it used to be that few folk added millions of records a day (just to pull a point from the air) and now by comparison more folk are adding millions of records an hour.

Hell, my home machine has a (what is normal now) terabyte drive in it. I remember working on a Sinclair and a Commodore and then thinking an IBM Model 30 was cool.

It's just the relentless pace of change.

 

[papadba]

because there is concern that current tools, structures, and ways of doing things may not work well at that scale.

Gently understated<g>

IMHO there is no hardware/software combination that has ever existed (or ever will) that can prevent "running out of resources". Left uncontrolled, "designers" can render any environment totally useless. . .

One of my recent clients has nearly 120 terabytes of online/realtime database storage (multiple database engines) and the term "big data" has never been used there.

From what i hear this is currently a new buzzword with no commonly accepted definition. Lots of vendors toss this in their presentations. . .

 

[r937]

big data" is anything large enough to get you thinking that you might want to store it in "the cloud"



r937.com | rudy.ca
Buy my new book Simply SQL from Amazon

 

[ArkM]

Look at Wikipedia article titled "Big data"...

 

[SQLScholar]

I would personally say much of the problem is time based.

Look back 50 years the only issue was how to file it, and where the filing cabnet was.

Look back 20 years - well databases were starting to become very popular. I imagine some databases have been running for well over 20 years (hopefully in a different format, on a different server) then they were on. Data has accumalated over that time -and people either

Are scared to delete data thats not required - incase it is required.
Are scared to archive the data incase its required quickly.
Are scared to touch a system that has been working.

So data just builds and builds and builds. I have seen systems with 15 years worth of data. I am sure there is more - and it isnt required even on a yearly basis - but no one archives it.

So it just gets bigger and bigger. Coupled with what other people have said about data going in quicker and quicker (internet has helped this!) you end up with BIG data!

Dan

----------------------------------------

http://www.fountain.me.uk

Be who you are and say what you feel, because those who mind don't matter and those who matter don't mind - Bernard Baruch

Computer Science is no more about computers than astronomy is about telescopes - EW Dijkstra
----------------------------------------

 

[johnherman]

I found a definition. Probably not the only one.....

http://www.zdnet.com/blog/big-data/big-data-defining-its-definition/109?tag=nl.e019

====================================
The trouble with doing something right the first time is that nobody appreciates how difficult it was - Steven Wright