Big Data

We never realize but whatever technology we use generates huge quantity of data every day. The most interesting part is how much data are piling up in the servers?  The answer is 1800 Exabyte/yr (1 EB = 1000 Petabyte = 1 million Terabyte = 1 billion Gigabyte) this data comes out of sensors, server logs, our Smartphone, web site, use of Social media.  Out of curiosity Facebook alone generates around 500 Terabyte of data every day and the same amount of data generates is generated by Airbus passenger Carrier flight in one day, The New York Stock Exchange generates about one terabyte of new trade data per day. All these data are unstructured data.

Big Data is a popular term used to describe the exponential growth and availability of data, both structured and unstructured.

Back in 2000’s, Big Data was defined as the three “V”s:

·        Volume: Many factors contribute to the increase in data volume. The exponential growth in the data storage out of sensor and machine-to-machine data Unstructured data streaming in from social media. Few years back excessive data volume was a storage issue, after the cost decreases there other issues comes up like analyzing the data founding the relevance within such a huge data.

·        Velocity: Velocity if Data can be understood by the data flow in social media, today peoples rely on social media for updating themselves with latest happening. Sometimes a few seconds old message also don’t interests user. They discard old messages and use to pay attention on the recent one. Data is streaming in at unprecedented speed and must be dealt with in a timely manner. Reacting quickly enough to deal with data velocity is a challenge for most organizations.

·   Variety: Today we are not just limited to Structured, numeric data stored in traditional databases. We are dealing with unstructured text documents, emails, videos, audios, and financial transactions. Something even the most famous organizations do struggle in managing, merging and governing all these different varieties of data.

In Short

Big data is not just huge amount of data, it is a concept of analyzing and finding new insights

form data regardless of their format and orientation.