Book Description

The book begins laying a strong foundation with an overview of data lakes, data warehouses, and related concepts. It then delves into core Hadoop components such as HDFS, YARN, MapReduce, and Apache Tez, offering a blend of theory and practical exercises.

You will gain hands-on experience with query engines like Apache Hive and Apache Spark, as well as file and table formats such as ORC, Parquet, Avro, Iceberg, Hudi, and Delta. Detailed instructions on installing and configuring clusters with Docker are included, along with big data visualization and statistical analysis using Python.

Given the growing importance of scalable data pipelines, this book equips data engineers, analysts, and big data professionals with practical skills to set up, manage, and optimize data pipelines, and to apply machine learning techniques effectively.

Table of Contents
1. Introduction to Hadoop and ASF
2. Overview of Big Data Analytics
3. Hadoop and YARN MapReduce and Tez
4. Distributed Query Engines: Apache Hive
5. Distributed Query Engines: Apache Spark
6. File Formats and Table Formats (Apache Ice-berg, Hudi, and Delta)
7. Python and the Hadoop Ecosystem for Big Data Analytics – BI
8. Data Science and Machine Learning with Hadoop Ecosystem
9. Introduction to Cloud Computing and Other Apache Projects
Index