PySpark for Data Science - V: ML Pipelines
Dive into the world of big data processing with PySpark, the Python library for Apache Spark. Learn how to process, analyze, and derive insights from massive datasets using Python’s user-friendly interface. Elevate your data skills with PySpark. Dive deep into distributed data processing, machine learning, streaming, and more to navigate the vast oceans of big data.
- Learn how to process, analyze, and derive insights from massive datasets using Python’s user-friendly interface.
- Elevate your data skills with PySpark. Dive deep into distributed data processing, machine learning, streaming, and more to navigate the vast oceans of big data.
Prabhakaran SPrincipal Data Scientist
Included in All-Access, one membership for every course.
- Lessons
- 18, in 5 chapters
- Video
- 1 h 2 min
- Free to watch
- 1 lesson
- Certificate
- On completion
What you'll be able to do
- PySpark Decision Tree
- PySpark Logistic regression
- PySpark Random Forest
- PySpark Gradient Boost
- PySPark XGBoost
Course content
5 chapters · 18 lessons · 1 h 2 min · 1 free to preview
01Recap Decision Trees3 lessons · 16 min
- Reading Decision TreePreview6:36
- Download Resources1:00
- How Decision Tree Works?8:07
02Build Decision Trees in PySpark8 lessons · 19 min
- Import required libraries and initialize SparkSession3:45
- Load the dataset2:11
- Prepare the data2:39
- Course Review1:00
- Building the DecisionTreeClassifier model4:12
- Evaluating the model on test data3:11
- Feature Importance1:22
- Improve the model (optional)0:56
03Tuning the Tree with Pipelines1 lesson · 12 min
- Creating a Pipeline & Hyperparameter Tuning11:58
04Self Assessment3 lessons · 5 min
- Random Forest Approach0:57
- Gradient Boosting0:39
- Compare results between 4 approaches3:08
05XGBoost model using PySpark3 lessons · 12 min
- The problem with XGBoost for PySpark4:30
- Install XGBoost in PySpark2:28
- Run XGBoost in PySpark5:09
What you need
- Basics of Python
- Foundational knowledge of Data Science
- High school maths
Who this course is for
- Data Science Aspirants
- Data Science Professionals
- Software/Data engineers interested in quantitative analysis
- Professionals working with large datasets
- Data analysts, economists, researchers
Your instructor
Prabhakaran S
Principal Data Scientist
My name is Selva, and I am super excited to mentor you on this course!
I head the Data Science team for a global Fortune 500 company and over the last 10 years of my data science experience I've deployed 20+ global products. I'm also the Founder and Chief Author of Machine Learning Plus, which has over 4M annual readers.
I specialize in covering the in-depth intuition and maths of any concept or algorithm. And based on my existing student requests, I've put up the series of courses and projects with detailed explanations, just like on-the-job experience. Hope you love it!
- from 3,965 ratings
- 4.92
- learners
- 57K+
- courses
- 85
Questions
How do I get this course?
It is part of All-Access: one membership that opens all 85 courses. The plans and prices are on our course site. You can watch the free preview lessons before you join.
Can I get a refund?
Yes. If the membership is not right for you, ask for a refund within 30 days of your first payment.
Can I learn on my phone?
Yes. Lessons play in the browser on any phone, tablet or computer, and your place is saved for the next device.
I am already a member. Where are my courses?
On our course site, edu.machinelearningplus.com. Sign in there with the email you joined with. The courses move to this site later, with your access, and you will not pay again.
