HDP Analyst Data Science

This course is designed for students preparing to become familiar with the processes and practice of data science, including machine learning and natural language processing. Included are: tools and programming languages (Python, IPython, Mahout, Pig, NumPy, Pandas, SciPy, Scikit-learn), the Natural Language Toolkit (NLTK), and Spark MLlib.

Target Audience

Computer science and data analytics students who need to apply data science and machine learning on Hadoop.

Prerequisites

Students must have experience with at least on programming or scripting language, knowledge in statistics and/or mathematics, and a basic understanding of big data and Hadoop principles.

Course Objectives

By the end of class, students will be able to:

• Recognize use cases for data science 
• Describe the architecture of Hadoop and YARN 
• Describe supervised and unsupervised learning differences
• List the six machine learning tasks 
• Use Mahout to run a machine learning algorithm on 
• Hadoop 
• Use Pig to transform and prepare data on Hadoop 
• Write a Python script 
• Use NumPy to analyze big data 
• Use the data structure classes in the pandas library 
• Write a Python script that invokes SciPy machine learning
• Describe options for running Python code on a Hadoop cluster 
• Write a Pig User-Defined Function in Python 
• Use Pig streaming on Hadoop with a Python script 
• Write a Python script that invokes scikit-learn 
• Use the k-nearest neighbor algorithm to predict values 
• Run a machine learning algorithm on a distributed data set 
• Describe use cases for Natural Language Processing (NLP)
• Perform sentence segmentation on a large body of text 
• Perform part-of-speech tagging 
• Use the Natural Language Toolkit (NLTK) 
• Describe the components of a Spark application 
• Write a Spark application in Python 
• Run machine learning algorithms using Spark MLlib