Big Data with Hadoop

Rationale

The workshop exposes you to the complex architectures of Hadoop and its components, guide you in the right direction to start with, and quickly start working with Hadoop and its components. It covers everything that you need as a Big Data Beginner. Learn about Big Data market, different job roles, technology trends, history of Hadoop, HDFS, Hadoop Ecosystem, Hive and Pig. In this course, we will see how as a beginner one should start with Hadoop. This course comes with a lot of hands-on examples which will help you learn Hadoop quickly.

Course Methodology

This course is a completely hands-on  exercise and participants are expected to learn by walking through the exercises as this would prepare them for the manipulation of big data a  linux operating system (ubuntu 14) would be installed together with the relevant hadoop cluster tools

Course Objectives

• You can develop real-time applications using MapReduce, Pig and Hive.
• You will easily understand the concept of Big Data and HADOOP.
• Sample Data and sample code is also included in this course for that project.

Target Audience

Data analysts, programmers, Developers, beginners in Bigdata  technology

Target Competencies

An understanding of distributed computing, basics of Linux operating system commands, and basic programming skills like java, python, and database fundamentals.

Course Outline

Big Data and Hadoop Concept

•         What is Data
•         What is Big Data
•         Data Sources of Big Data part
•         Traditional Analytics vs Big Data Analytics
•         Big Data Customers-Many Industrial Domains
•         Bigdata-attributes-volume
•         variety of data
•         Velocity of Data
•         Veracity of Data
•         Hadoop History
•         Hadoop Concepts
•         Hadoop Ecosystem
•         Hadoop Core Components
•         Hadoop Distributions
•         HDFS Blocks File Splits
•         HDFS Write Operation
•         hadoop-2.x-architecture

Understanding MapReduce and Hadoop Installation

•         MapReduce Components
•         Understand MapReduce Flow
•         Client Communication
•         Need of YARN
•         HDFS Architecture
•         NodeManager
•         Hadoop Cluster Modes
•         Secondary-Namenode
•         Hadoop 2.7.3 Installation
•         Hadoop 2.7.3 Configuration Files
•         Hadoop Basic Commands

Apache PIG (Concept)

•         Introduction.
•         Overview
•         Set up Cloudera on Windows for PIG
•         Basics Commands in PIG
•         Group by and Co-Group Operators Demo in PIG
•         Load and Store Functions in PIG

Calculate Averge Risk Using MapReduce

•         Calculate Average Risk using MapReduce
•         MapReduce Coding for calculating average risk
•         Execution Of MapReduce Code for average risk calculation

Calculate Averge Risk Using MapReduce per Location

•         Calculate Average Risk per Location using MapReduce
•         Execution Of MapReduce Code for Average Risk per Location

Calculate Average Risk Using MapReduce per Category

•         Calculate Average Risk per Category using MapReduce
•         Execution Of MapReduce Code for Average Risk per Category
•         Calculate Average Risk per Location and Category using MapReduce

Banking and Finance Domain Analysis using Pig

•         Calculate Overall Average Risk using Pig
•         Other Scenarios using Pig

Environment setup and Import Data using Sqoop

•         Setup Cloudera to work on Big-Data tools
•         Get Data RDBMS to HDFS using Sqoop
•         Banking and Finance Domain Analysis using HIVE