PySpark for Data Science - III: Data Cleaning and Analysis Methods
Dive into the world of big data processing with PySpark, the Python library for Apache Spark. Learn how to process, analyze, and derive insights from massive datasets using Python’s user-friendly interface. Elevate your data skills with PySpark. Dive deep into distributed data processing, machine learning, streaming, and more to navigate the vast oceans of big data.
- Learn how to process, analyze, and derive insights from massive datasets using Python’s user-friendly interface.
- Elevate your data skills with PySpark. Dive deep into distributed data processing, machine learning, streaming, and more to navigate the vast oceans of big data.
Prabhakaran SPrincipal Data Scientist
Included in All-Access, one membership for every course.
- Lessons
- 22, in 6 chapters
- Video
- 1 h 14 min
- Free to watch
- 1 lesson
- Certificate
- On completion
What you'll be able to do
- Identifying Variable Types
- Outlier Detection and Treatment
- Identifying and Removing Duplicates
- Feature Encoding with PySpark
- Missing Value Imputation with PySpark
- Feature scaling with PySpark
- Feature Extraction / Dimensionality Reduction
Course content
6 chapters · 22 lessons · 1 h 14 min · 1 free to preview
01Introduction to Data PreProcessing4 lessons · 14 min
- Data Preprocessing EssencePreview4:00
- Download Resources1:00
- Automate Variable type Identification8:16
- Self Assessment - 11:11
02Outlier Detection and Treatment6 lessons · 19 min
- Outlier Treatment Approaches6:52
- How to Detect and Treat Outliers using IQR method?3:59
- How to detect and treat outliers using the z-score?2:31
- Identifying and Removing Duplicates3:17
- Self Assessment - 21:04
- Course Review1:00
03Missing Value Imputation5 lessons · 15 min
- Missing Value Treatment approaches2:50
- Approach 1: Detect and drop the row with missing values6:20
- Approach 2: Check conditions and drop an entire Column2:25
- Approach 3: Impute the missing data appropriately1:46
- Self assessment - 31:30
04Feature Encoding2 lessons · 10 min
- StringIndexer4:34
- OneHot Encoding5:38
05Feature scaling2 lessons · 6 min
- Feature Min-Max scaling4:18
- Feature Standardization1:22
06Feature Extraction / Dimensionality Reduction3 lessons · 12 min
- Variance Inflation Factor (VIF)6:24
- PCA (Principal Component Analysis)3:24
- Final Self Assessment1:55
What you need
- Basics of Python
- Foundational knowledge of Data Science
- High school maths
Who this course is for
- Data Science Aspirants
- Data Science Professionals
- Software/Data engineers interested in quantitative analysis
- Professionals working with large datasets
- Data analysts, economists, researchers
Your instructor
Prabhakaran S
Principal Data Scientist
My name is Selva, and I am super excited to mentor you on this course!
I head the Data Science team for a global Fortune 500 company and over the last 10 years of my data science experience I've deployed 20+ global products. I'm also the Founder and Chief Author of Machine Learning Plus, which has over 4M annual readers.
I specialize in covering the in-depth intuition and maths of any concept or algorithm. And based on my existing student requests, I've put up the series of courses and projects with detailed explanations, just like on-the-job experience. Hope you love it!
- from 3,965 ratings
- 4.92
- learners
- 57K+
- courses
- 85
Questions
How do I get this course?
It is part of All-Access: one membership that opens all 85 courses. The plans and prices are on our course site. You can watch the free preview lessons before you join.
Can I get a refund?
Yes. If the membership is not right for you, ask for a refund within 30 days of your first payment.
Can I learn on my phone?
Yes. Lessons play in the browser on any phone, tablet or computer, and your place is saved for the next device.
I am already a member. Where are my courses?
On our course site, edu.machinelearningplus.com. Sign in there with the email you joined with. The courses move to this site later, with your access, and you will not pay again.
