(HDP) DEVELOPER: APACHE PIG AND HIVE

Course details

HORTONWORKS DATA PLATFORM (HDP) DEVELOPER: APACHE PIG AND HIVE

This 4 day training course is designed for developers who need to create applications to analyze Big Data stored in Apache Hadoop using Pig and Hive. Topics include: Hadoop, YARN, HDFS, MapReduce, data ingestion, workflow definition, using Pig and Hive to perform data analytics on Big Data and an introduction to Spark Core and Spark ;
 
PREREQUISITES

Students should be familiar with programming principles and have experience in software development. SQL knowledge is also helpful. No prior Hadoop knowledge is required.

TARGET AUDIENCE

Software developers who need to understand and develop applications for Hadoop. FORMAT 50% Lecture/Discussion 50% Hands-0n Labs 
 
AGENDA SUMMARY

Day 1: Understanding Hadoop and the Hadoop Distributed File System (HDFS)

Day 2: Pig Programming

Day 3: Hive Programming

Day 4: Advanced Hive Programming, Hadoop 2 and YARN, Introduction to Spark Core 

DAY 1 OBJECTIVES
 
• List the Three "V"s of Big Data • List the Six Key Hadoop Data Types • Describe Hadoop, YARN and Use Cases for Hadoop • Describe Hadoop Ecosystem Tools and Frameworks • Describe the Differences Between Relational Databases and Hadoop • Describe What is New in Hadoop • Describe the Hadoop Distributed File System (HDFS) • Describe the Differences Between HDFS and an RDBMS • Describe the Purpose of NameNodes and DataNodes • List Common HDFS Commands • Describe HDFS File Permissions • List Options for Data Input • Describe WebHDFS • Describe the Purpose of Sqoop and Flume • Describe How to Export to a Table • Describe the Purpose of MapReduce • Define Key/Value Pairs in MapReduce • Describe the Map and Reduce Phases • Describe Hadoop Streaming 
 
DAY 1 LABS AND DEMONSTRATIONS
 
• Starting an HDP Cluster • Demonstration: Understanding Block Storage • Using HDFS Commands • Importing RDBMS Data into HDFS • Exporting HDFS Data to an RDBMS • Importing Log Data into HDFS Using Flume • Demonstration: Understanding MapReduce • Running a MapReduce Job 

DAY 2 OBJECTIVES 
• Describe the Purpose of Apache Pig • Describe the Purpose of Pig Latin • Demonstrate the Use of the Grunt Shell • List Pig Latin Relation Names and Field Names • List Pig Data Types • Define a Schema • Describe the Purpose of the GROUP Operator • Describe Common Pig Operators, Including o ORDER BY o CASE o DISTINCT o PARALLEL o FLATTEN o FOREACH • Perform an Inner, Outer and Replicated Join • Describe the Purpose of the DataFu Library 
 
DAY 2 LABS AND DEMONSTRATIONS 
• Demonstration: Understanding Apache Pig • Getting Starting with Apache Pig • Exploring Data with Apache Pig • Splitting a Dataset • Joining Datasets with Apache Pig • Preparing Data for Apache Hive • Demonstration:
Computing Page Rank • Analyzing Clickstream Data • Analyzing Stock Market Data Using Quantiles  Updated on 01 April, 2018