Understanding Data Measurement in Big Data
Written on
Chapter 1: Introduction to Data Measurement
In the digital age, we are all familiar with the storage capacity of our devices, whether it's a smartphone with 16, 32, or 64 GB or a computer boasting 4, 8, or 16 GB of RAM. A device with 64 GB can hold approximately 64 x 1,073,741,824 bytes of information.
Data measurements are based on the binary system used by computers. For instance, (2^{10}) (two raised to the power of ten) equals 1,024 bytes. Understanding these measurements is crucial as we navigate the immense volume of data generated in the realm of Big Data.
Units of Data Measurement
1 Byte = 1 Character or 8 bits
1 Kilobyte (KB) = 1,024 Bytes
1 Megabyte (MB) = 1,024 Kilobytes
1 Gigabyte (GB) = 1,024 Megabytes
1 Terabyte (TB) = 1,024 Gigabytes
1 Petabyte (PB) = 1,024 Terabytes
1 Exabyte (EB) = 1,024 Petabytes
1 Zettabyte (ZB) = 1,024 Exabytes
1 Yottabyte (YB) = 1,024 Zettabytes
We utilize various measurements daily, such as kilograms for weight or meters for distance, and we gradually adapt to these data units. However, measuring data can be daunting, especially as we realize the exponential growth in volume that our devices often cannot accommodate, necessitating the use of cloud technologies offering virtually limitless storage.
Section 1.1: Measuring Data Size
A high-resolution photo typically consumes around 5 MB. If you take 100 photos, you'll need approximately 500 MB of storage. USB drives, which range from 32 to 256 GB, can be quite handy. Alternatively, services like Google Photos provide 30 GB of free cloud storage for your convenience.
As we delve into Big Data, we encounter measurements in the realm of terabytes and petabytes, moving far beyond the familiar kilobytes, megabytes, or gigabytes. Below is a comparative look at the vast numbers involved:
1 Megabyte (MB) = 1,048,576 Bytes
1 Gigabyte (GB) = 1,073,741,824 Bytes
1 Terabyte (TB) = 1,099,511,627,776 Bytes
1 Petabyte (PB) = 1,125,899,906,842,624 Bytes
1 Exabyte (EB) = 1,152,921,504,606,846,976 Bytes
Interesting Facts
- Google generates 100 petabytes of data daily.
- Facebook produces over 30 petabytes of data each day.
- Twitter generates around 100 terabytes of data daily.
- Spotify generates 64 terabytes each day.
- eBay also produces 100 petabytes daily.
- By 2007, we had created 300 exabytes of data; today, that number has skyrocketed to over 4,000 exabytes, with projections estimating an additional 40,000 exabytes (40 zettabytes) by 2020.
- Google stands as the largest Big Data company, processing 3.5 billion daily requests and storing 10 exabytes of data.
- Approximately 90% of the world's data has been produced in the last two years.
Section 1.2: Data Generated by Wearables
The rise of wearable technology, including smartwatches, fitness bands, and even smart clothing, is noteworthy. These devices, commonly used for tracking physical activities, can generate between 2 to 5 GB of data per day regarding a user's activities.
Over the course of a year, one might accumulate around 1.8 terabytes of data using such a device, which could amount to 18 terabytes over a decade. Personal data generated by individuals is becoming increasingly invaluable for data-driven companies like Google and Facebook, necessitating strict confidentiality.
Interesting Facts
- Wearables often include sensors that promote exercise and track weight loss programs.
- Common health metrics tracked by smartwatches include heart rate, body temperature, and steps taken.
- Smartwatches work in tandem with smartphones for data generation, storage, and analysis.
- Google Glass, a prominent wearable, was discontinued due to commercial challenges.
- There are over 100,000 health apps available for wearables and smartphones on major app stores.
- A black top worn by football players can measure athletic performance during games.
Chapter 2: Data Generated by IoT
Almost every object on Earth is set to become a data generator, including our homes, vehicles, and even our bodies. Look around, and you'll notice many devices are now internet-connected, from smartphones and computers to smart TVs and wearables.
Home automation allows control over lighting, temperature, and security through smartphones. It is estimated that 250,000 vehicles will be connected to the internet by 2020. Beyond residential use, various objects, such as traffic lights, ATMs, GPS devices, and airplanes, are also joining the network.
This extensive connectivity is referred to as the Internet of Things (IoT), creating a global network of devices that rapidly generate data, forming a conscious and autonomous system without requiring human intervention. Currently, there are approximately 8.3 billion connected devices, surpassing the global population, with projections of 50 to 75 billion devices by 2025.
Big Data and IoT are closely intertwined, as the data produced by IoT provides the volume and speed necessary for Big Data applications. The full realization of IoT's potential will depend on the rollout of 5G technologies.
Interesting Facts
- The global data volume is expected to reach 163 zettabytes by 2025 (IDC), with most of it being generated by IoT.
- By 2025, an average person is predicted to interact with connected devices at least 4,800 times daily.
- Applications of Big Data in conjunction with IoT include smart meters for electricity consumption, fleet management, automated inventory, patient monitoring, and connected vehicles.
Support the author's work by subscribing to the email list for more information on these articles.
This video titled "Estatística Dimensionamento de uma amostra - Aula completa - YouTube" delves into statistical methods for sample sizing, complementing our understanding of data measurement.
The second video, "Capítulo 2 - Como Dimensionar os Dados - YouTube," provides insights into how to effectively measure data, further enriching our exploration of Big Data concepts.