Overview

This course serves as an appropriate entry point to learn Apache Spark Programming with Databricks.

Below, we describe each of the four, four-hour modules included in this course.

Introduction to Apache Spark

This course offers essential knowledge of Apache Spark, with a focus on its distributed architecture and practical applications for large-scale data processing. Participants will explore programming frameworks, learn the Spark DataFrame API, and develop skills for reading, writing, and transforming data using Python-based Spark workflows.

Developing Applications with Apache Spark

Master scalable data processing with Apache Spark in this hands-on course. Learn to build efficient ETL pipelines, perform advanced analytics, and optimize distributed data transformations using Spark’s DataFrame API. Explore grouping, aggregation, joins, set operations, and window functions. Work with complex data types like arrays, maps, and structs while applying best practices for performance optimization.

Stream Processing and Analysis with Apache Spark

Learn the essentials of stream processing and analysis with Apache Spark in this course. Gain a solid understanding of stream processing fundamentals and develop applications using the Spark Structured Streaming API. Explore advanced techniques such as stream aggregation and window analysis to process real-time data efficiently. This course equips you with the skills to create scalable and fault-tolerant streaming applications for dynamic data environments.

Monitoring and Optimizing Apache Spark Workloads on Databricks

This course explores the Lakehouse architecture and Medallion design for scalable data workflows, focusing on Unity Catalog for secure data governance, access control, and lineage tracking. The curriculum includes building reliable, ACID-compliant pipelines with Delta Lake. You’ll examine Spark optimization techniques, such as partitioning, caching, and query tuning, and learn performance monitoring, troubleshooting, and best practices for efficient data engineering and analytics to address real-world challenges.

Why Choose KORNERSTONE

  • Accredited, practitioner-led, expert certificate trainers to provide high quality training
  • Official training material included
  • Practical, scenario-based learning – focus on application, not just theory
  • Excellent Passing Rate

Skills Covered

  • Introduction to Apache Spark
  • Developing Applications with Apache Spark
  • Stream Processing and Analysis with Apache Spark
  • Monitoring and Optimizing Apache Spark Workloads on Databricks

Prerequisites

  • Basic programming knowledge
  • Familiarity with Python
  • Basic understanding of SQL queries (SELECT, JOIN, GROUP BY)
  • Familiarity with data processing concepts
  • No prior Spark or Databricks experience required

Course Curriculum

Module 1: Introduction to Apache Spark

  • Spark Runtime Architecture
  • Exploring Apache Spark Architecture in Databricks
  • Introduction to Spark DataFrames and SQL
  • Reading and Writing Data with DataFrames
  • Distributed System Programming Fundamentals
  • Basic ETL with the DataFrame API
  • Flight Data ETL with the DataFrame API
  • Analyzing Transaction Data with DataFrames

Module 2: Developing Applications with Apache Spark

  • DataFrame API Basics
  • Demo: (Optional) Basic ETL with the DataFrame API
  • Grouping and Aggregating Data
  • Demo: Grouping and Aggregating Data
  • Lab: Grouping and Aggregating E-Commerce Data
  • Relational Operations
  • Demo: Data Relational Operations in Apache Spark
  • Working with Complex Data
  • Demo: Working with Complex Data Types in Apache Spark
  • Lab: Working with Complex Data Types in E-Commerce Data

Module 3: Stream Processing and Analysis with Apache Spark

  • Introduction to Stream Processing
  • Spark Structured Streaming
  • Demo: Introduction to Spark Structured Streaming
  • Lab: Introduction to Spark Structured Streaming
  • Advanced Stream Processing and Analysis
  • Demo: Window Aggregation in Spark Structured Streaming
  • Lab: Window Aggregation in Spark Structured Streaming

Module 4: Monitoring and Optimizing Apache Spark Workloads on Databricks

  • Apache Spark and Databricks
  • Using Apache Spark with Delta Lake
  • Demo: Introduction to Delta Lake
  • Lab: Introduction to Delta Lake
  • Optimizing Apache Spark
  • Demo: Optimizing Apache Spark
  • Lab: Optimizing Apache Spark

Let's make it work for you

Can’t find a date that fits? Need to train your whole team? Looking for a discount?
Speak to one of our training consultant today.

Dates & Locations

x Clear all selections

Exam & Certification

Databricks Certified Associate Developer for Apache Spark.

The Databricks Certified Associate Developer for Apache Spark certification exam assesses the understanding of the Apache Spark Architecture and Components and the ability to apply the Spark DataFrame API to complete basic data manipulation tasks within a Spark session. These tasks include selecting, renaming and manipulating columns; filtering, dropping, sorting, and aggregating rows; handling missing data; combining, reading, writing and partitioning DataFrames with schemas; and working with UDFs and Spark SQL functions.

In addition, the exam will assess the basics of the Spark architecture like execution/deployment modes, the execution hierarchy, fault tolerance, garbage collection, lazy evaluation, Shuffling and usage of Actions and broadcasting, Structured Streaming, Spark Connect, and common troubleshooting and tuning techniques. Individuals who pass this certification exam can be expected to complete basic Spark DataFrame tasks using Python.

This exam covers:

  1. Apache Spark Architecture and Components – 20%

  2. Using Spark SQL – 20%

  3. Developing Apache Spark™ DataFrame/DataSet API Applications – 30%

  4. Troubleshooting and Tuning Apache Spark DataFrame API Applications – 10%

  5. Structured Streaming – 10%

  6. Using Spark Connect to deploy applications – 5%

  7. Using Pandas API on Apache Spark – 5%

Contact Us

Get in touch with our team via the form or WhatsApp »

Your preferences: