Delta Lake with Apache Spark using Scala

Course details

You will Learn Delta Lake with Apache Spark using Scala on DataBricks Platform

Delta Lake is an open-source storage layer that brings reliability to data lakes. Delta Lake provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. Delta Lake runs on top of your existing data lake and is fully compatible with Apache Spark APIs.

Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph processing, and Spark Streaming.

Topics Included in the Courses

• Introduction to Delta Lake

• Introduction to Data Lake

• Key Features of Delta Lake

• Introduction to Spark

• Free Account creation in Databricks

• Provisioning a Spark Cluster

• Basics about notebooks

• Dataframes

• Create a table

• Write a table

• Read a table

• Schema validation

• Update table schema

• Table Metadata

• Delete from a table

• Update a Table

• Vacuum

• History

• Concurrency Control

• Optimistic concurrency control

• Migrate Workloads to Delta Lake

• Optimize Performance with File Management

• Auto Optimize

• Optimize Performance with Caching

• Delta and Apache Spark caching

• Cache a subset of the data

• Isolation Levels

• Best Practices

• Frequently Asked Question in Interview

About Databricks:

Databricks lets you start writing Spark code instantly so you can focus on your data problems.