What you’ll learn
-
You will learn how to Architect, Design and build a real-world enterprise level data platform solution including multiple services.
-
You will learn design solution using ADF, Azure Function, Databricks, pyspark, Azure Data lake storage Gen 2 (ADLS), Azure SQL Server
-
You will learn how to build a real-world data pipeline in Azure Data Factory (ADF). This course has been taught using 2 real world use case scenarios.
-
You will learn how to transform data using Databricks Notebook Activity in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2
-
You will learn how to build production ready pipelines and good practices and naming standards
-
You will learn how to integrate Databricks with ADF and send the response back from Databricks to ADF
-
You will learn how to develop the triggered based Azure Function to validate files.
-
You will learn how to create Azure Key vault and use it to store secret credentials and SAS token
-
You will learn how to connect the Azure SQL Database and Databricks cluster using the Key Vault
-
You will learn how to mount he Azure Storage Account in the Databricks to access the files and preform transformation on it.
-
You will learn how to transform the data in the Azure Databricks using the pyspark.
