access
lifetime
content
13 Hours
Description
Big Data sounds pretty daunting doesn't it? Well, this course aims to make it a lot simpler for you. Using Hadoop and MapReduce, you'll learn how to process and manage enormous amounts of data efficiently. Any company that collects mass amounts of data, from startups to Fortune 500, need people fluent in Hadoop and MapReduce, making this course a must for anybody interested in data science.- Access 71 lectures & 13 hours of content 24/7
- Set up your own Hadoop cluster using virtual machines (VMs) & the Cloud
- Understand HDFS, MapReduce & YARN & their interaction
- Use MapReduce to recommend friends in a social network, build search engines & generate bigrams
- Chain multiple MapReduce jobs together
- Write your own customized partitioner
- Learn to globally sort a large amount of data by sampling input files
Loonycorn is comprised of four individuals--Janani Ravi, Vitthal Srinivasan, Swetha Kolalapudi and Navdeep Singh--who have honed their tech expertises at Google and Flipkart. The team believes it has distilled the instruction of complicated tech concepts into funny, practical, engaging courses, and is excited to be sharing its content with eager students.
Details & Requirements
- Length of time users can access this course: lifetime access
- Access options: web streaming, mobile streaming
- Certification of completion not included
- Redemption deadline: redeem your code within 30 days of purchase
- Experience level required: all levels
Compatibility
- Internet required
Terms
- Unredeemed licenses can be returned for store credit within 15 days of purchase. Once your license is redeemed, all sales are final.
Course Outline
- Introduction
- You, this course and Us (1:52)
- Why is Big Data a Big Deal
- The Big Data Paradigm (14:20)
- Serial vs Distributed Computing (8:37)
- What is Hadoop? (7:25)
- HDFS or the Hadoop Distributed File System (11:01)
- MapReduce Introduced (11:39)
- YARN or Yet Another Resource Negotiator (4:00)
- Installing Hadoop in a Local Environment
- Hadoop Install Modes (8:32)
- Setup a Virtual Linux Instance (For Windows users) (15:31)
- Hadoop Standalone mode Install (9:33)
- Hadoop Pseudo-Distributed mode Install (14:25)
- The MapReduce "Hello World"
- The basic philosophy underlying MapReduce (8:49)
- MapReduce - Visualized And Explained (9:03)
- MapReduce - Digging a little deeper at every step (10:21)
- "Hello World" in MapReduce (10:29)
- The Mapper (9:48)
- The Reducer (7:46)
- The Job (12:28)
- Run a MapReduce Job
- Get comfortable with HDFS (10:59)
- Run your first MapReduce Job (14:30)
- Juicing your MapReduce - Combiners, Shuffle and Sort and The Streaming API
- Parallelize the reduce phase - use the Combiner (14:40)
- Not all Reducers are Combiners (14:31)
- How many mappers and reducers does your MapReduce have? (8:23)
- Parallelizing reduce using Shuffle And Sort (14:55)
- MapReduce is not limited to the Java language - Introducing the Streaming API (5:05)
- Python for MapReduce (12:19)
- HDFS and Yarn
- HDFS - Protecting against data loss using replication (15:32)
- HDFS - Name nodes and why they're critical (6:48)
- HDFS - Checkpointing to backup name node information (11:10)
- Yarn - Basic components (8:33)
- Yarn - Submitting a job to Yarn (13:10)
- Yarn - Plug in scheduling policies (14:21)
- Yarn - Configure the scheduler (12:26)
- Setting up a Hadoop Cluster
- Manually configuring a Hadoop cluster (Linux VMs) (13:50)
- Getting started with Amazon Web Servicies (6:25)
- Start a Hadoop Cluster with Cloudera Manager on AWS (13:04)
- MapReduce Customizations For Finer Grained Control
- Setting up your MapReduce to accept command line arguments (13:47)
- The Tool, ToolRunner and GenericOptionsParser (12:36)
- Configuring properties of the Job object (10:41)
- Customizing the Partitioner, Sort Comparator, and Group Comparator (15:16)
- The Inverted Index, Custom Data Types for Keys, Bigram Counts and Unit Tests!
- The heart of search engines - The Inverted Index (14:41)
- Generating the inverted index using MapReduce (10:25)
- Custom data types for keys - The Writable Interface (10:23)
- Represent a Bigram using a WritableComparable (13:13)
- MapReduce to count the Bigrams in input text (8:26)
- Test your MapReduce job using MRUnit (13:41)
- Input and Output Formats and Customized Partitioning
- Introducing the File Input Format (12:48)
- Text And Sequence File Formats (10:21)
- Data partitioning using a custom partitioner (7:11)
- Make the custom partitioner real in code (10:25)
- Total Order Partitioning (10:10)
- Input Sampling, Distribution, Partitioning and configuring these (9:04)
- Secondary Sort (14:34)
- Recommendation Systems using Collaborative Filtering
- Introduction to Collaborative Filtering (7:25)
- Friend recommendations using chained MR jobs (17:15)
- Get common friends for every pair of users - the first MapReduce (14:50)
- Top 10 friend recommendation for every user - the second MapReduce (13:46)
- Hadoop as a Database
- Structured data in Hadoop (14:08)
- Running an SQL Select with MapReduce (15:31)
- Running an SQL Group By with MapReduce (14:02)
- A MapReduce Join - The Map Side (14:20)
- A MapReduce Join - The Reduce Side (13:08)
- A MapReduce Join - Sorting and Partitioning (8:49)
- A MapReduce Join - Putting it all together (13:46)
- K-Means Clustering
- What is K-Means Clustering? (14:04)
- A MapReduce job for K-Means Clustering (16:33)
- K-Means Clustering - Measuring the distance between points (13:52)
- K-Means Clustering - Custom Writables for Input/Output (8:26)
- K-Means Clustering - Configuring the Job (10:50)
- K-Means Clustering - The Mapper and Reducer (11:23)
- K-Means Clustering : The Iterative MapReduce Job (3:40)
