Читать книгу Big Data - Seifedine Kadry - Страница 48

1  What is big data? Big data is a blanket term for the data that are too large in size, complex in nature, which may be structured or unstructured, and arriving at high velocity as well.

2  What are the drawbacks of traditional database that led to the evolution of big data? Below are the limitations of traditional databases, which has led to the emergence of big data.Exponential increase in data volume, which scales in terabytes and petabytes, has turned out to become a challenge to the RDBMS in handling such a massive volume of data.To address this issue, the RDBMS increased the number of processors and added more memory units, which in turn increased the cost.Almost 80% of the data fetched were of semi‐structured and unstructured format, which RDBMS could not deal with.RDBMS could not capture the data coming in at high velocity.

3  What are the factors that explain the tremendous increase in the data volume? Multiple disparate data sources are responsible for the tremendous increase in the volume of big data. Much of the growth in data can be attributed to the digitization of almost anything and everything in the globe. Paying e‐bills, online shopping, communication through social media, e‐mail transactions in various organizations, a digital representation of the organizational data, and so forth, are some of the examples of this digitization around the globe.

4  What are the different data types of big data? Machine‐generated and human‐generated data can be represented by the following primitive types of big dataStructured dataUnstructured dataSemi‐Structured data

5  What is semi‐structured data? Semi‐structured data are that which have a structure but does not fit into the relational database. Semi‐structured data are organized, which makes it easier for analysis when compared to unstructured data. JSON and XML are examples of semi‐structured data.

6  What does the three Vs of big data mean? Volume–Size of the dataVelocity–Rate at which the data is generated and is being processedVariety–Heterogeneity of data: structured, unstructured, and semi‐structured

7  What is commodity hardware? Commodity hardware is a low‐cost, low‐performance, and low‐specification functional hardware with no distinctive features. Hadoop can run on commodity hardware and does not require any high‐end hardware or supercomputers to execute its jobs.

8  What is data aggregation? The data aggregation phase of the big data life cycle involves collecting the raw data, transmitting the data to a storage platform, and preprocessing them. Data acquisition in the big data world means acquiring the high‐volume data arriving at an ever increasing pace.

9  What is data preprocessing? Data preprocessing is an important process performed on raw data to transform it into an understandable format and provide access to a consistent and an accurate data. The data generated from multiple sources are erroneous, incomplete, and inconsistent because of their massive volume and heterogeneous sources, and it is pointless to store useless and dirty data. Additionally, some analytical applications have a crucial requirement for quality data. Hence, for effective, efficient, and accurate data analysis, systematic data preprocessing is essential.

10  What is data integration? Data integration involves combining data from different sources to give the end users a unified data view.

11  What is data cleaning? The data‐cleaning process fills in the missing values, corrects the errors and inconsistencies, and removes redundancy in the data to improve the data quality. The larger the heterogeneity of the data sources, the higher the degree of dirtiness. Consequently, more cleaning steps may be involved.

12  What is data reduction? Data processing on massive data volume may take a long time, making data analysis either infeasible or impractical. Data reduction is the concept of reducing the volume of data or reducing the dimension of the data, that is, the number of attributes. Data reduction techniques are adopted to analyze the data in reduced format without losing the integrity of the actual data and yet yield quality outputs.

13  What is data transformation? Data transformation refers to transforming or consolidating the data into an appropriate format that is acceptable by the big data database and converting them into logical and meaningful information for data management and analysis.