Zenler Player
Your course is loading. Hang tight.
Azure Data Factory and Fabric Data Factory: Data Integration for Data Professionals
Back to curriculum
0% Complete
0% Complete
Who is this course for?
Who is Greg?
What will I learn in this course?
Configuring your lab environment
Module introduction
Role of Integration in modern analytics
Common loading patterns - ETL vs ELT
On-premises tooling vs cloud-based tooling
What are pipelines?
What is Azure Data Factory (ADF)?
What is Fabric Data Factory (FDF)?
Core Data Factory components
Differences between ADF and FDF
Pros and cons of each Data Factory platform
Quiz 1
Module introduction
Creating an Azure Resource Group
Creating an Azure Data Factory
Using ADF with and without source control
Connecting source control
Using ADF Studio
Creating a Fabric Workspace and Data Factory
Using Fabric Studio
Lab 2
Quiz 2
Module introduction
Types of Azure storage
Azure Storage Explorer
SQL Server Management Studio (SSMS)
Copy Data Tool in ADF
Storage in Fabric
Copy Data Job in FDF
Lab 3
Quiz 3
Module introduction
Pipeline core components
Pipeline design interface elements
Connections
Linked services
Datasets
Folders
Lab 4
Quiz 4
Module introduction
Pipeline workflow
Common activity settings
Wait activity
Common activity actions
Variables
Set Variable activity
Expression language
Append Variable activity
If Condition activity
Implementing For Each loops
Implementing While Until loops
Filter activity
Switch activity
Logical AND vs logical OR
Lab 5
Quiz 5
Module introduction
Pipeline parameters
Using built-in templates
Creating templates
Lab 6
Quiz 6
Module introduction
Scheduling in ADF
Manual triggers
Schedule triggers
Tumbling window triggers
Storage Event triggers
Custom event triggers
Scheduling in FDF
Lab 7
Quiz 7
Module introduction
Delete activity
Copy activity
Stored Procedure activity
Lookup activity
Script activity
Get Metadata activity
Execute Pipeline activity
Web activity
Fail activity
Notebook activity
Lab 8
Quiz 8
Module introduction
Integration runtimes
Azure integration runtimes
Self-hosted integration runtimes
Linking and sharing integration runtimes
SQL Server Integration Services
Executing SSIS packages in ADF
Azure SSIS integration runtimes
SSIS IR cost implications
Lab 9
Quiz 9
Module introduction
ADF mapping data flows
ADF data flow performance and cost implications
ADF Power Queries
Comparing Power Queries to data flows
ADF flowlets
FDF gen 2 data flows
Comparing ADF and FDF data flows
Lab 10
Quiz 10
Module introduction
Debugging pipelines
Annotations
User properties
Defining dynamic connections by using expressions
Monitoring ADF pipelines
Sending alerts on failure
Copying JSON to create pipelines
Useful 3rd party deployment utilities and tools
Git Integration in Fabric
Monitoring FDF pipelines
Lab 11
Quiz 11
Module introduction
Identity concepts in Azure
RBAC based security
System-assigned managed identities
User-assigned managed identities
Workspace identities in FDF
Using managed identities in connections
Azure Key Vault
Integrating ADF with Azure Key Vault
Integrating FDF with Azure Key Vault
Using key vault secrets in connections
Connecting via private endpoints
Lab 12
Quiz 12
Module introduction
Problem to solve
REST based calls required
Configuring security
Making refresh request
Checking latest refreshes
Filter to current refresh
Retry until complete
Throw error on failure
Module introduction
Problem to solve
Options considered
Overall solution design
File in storage account
Testing requirements
First attempt - add dataset reference
Correct multipart form data
Attachment file contents
Returned preview data
Multipart form data format
Web request content type and body
Configure the web activity for upload
HTTP request timeout
Line endings
Requirement achieved
Further thoughts
Module introduction
Validation activity
Azure Data Explorer command
Azure Function activity
Databricks-related activities
WebHook activity
HDInsight-related activities
Data Lake Analytics-related activities
Machine Learning-related activities
Summary and further steps
Module 0: Getting started
Who is this course for?
Preview
Who is Greg?
Preview
What will I learn in this course?
Preview
Configuring your lab environment
Module 1: Core concepts
Module introduction
Role of Integration in modern analytics
Common loading patterns - ETL vs ELT
On-premises tooling vs cloud-based tooling
What are pipelines?
What is Azure Data Factory (ADF)?
What is Fabric Data Factory (FDF)?
Core Data Factory components
Differences between ADF and FDF
Pros and cons of each Data Factory platform
Quiz 1
Module 2: Provisioning data factories
Module introduction
Creating an Azure Resource Group
Creating an Azure Data Factory
Using ADF with and without source control
Connecting source control
Using ADF Studio
Creating a Fabric Workspace and Data Factory
Using Fabric Studio
Lab 2
Quiz 2
Module 3: Wizard-based copying of data
Module introduction
Types of Azure storage
Azure Storage Explorer
SQL Server Management Studio (SSMS)
Copy Data Tool in ADF
Storage in Fabric
Copy Data Job in FDF
Lab 3
Quiz 3
Module 4: Concepts common to both ADF and FDF
Module introduction
Pipeline core components
Pipeline design interface elements
Connections
Linked services
Datasets
Folders
Lab 4
Quiz 4
Module 5: Implementing workflow logic in pipelines
Module introduction
Pipeline workflow
Common activity settings
Wait activity
Common activity actions
Variables
Set Variable activity
Expression language
Append Variable activity
If Condition activity
Implementing For Each loops
Implementing While Until loops
Filter activity
Switch activity
Logical AND vs logical OR
Lab 5
Quiz 5
Module 6: Working with parameters and templates
Module introduction
Pipeline parameters
Using built-in templates
Creating templates
Lab 6
Quiz 6
Module 7: Triggering pipeline execution
Module introduction
Scheduling in ADF
Manual triggers
Schedule triggers
Tumbling window triggers
Storage Event triggers
Custom event triggers
Scheduling in FDF
Lab 7
Quiz 7
Module 8: Working with common activities
Module introduction
Delete activity
Copy activity
Stored Procedure activity
Lookup activity
Script activity
Get Metadata activity
Execute Pipeline activity
Web activity
Fail activity
Notebook activity
Lab 8
Quiz 8
Module 9: Managing integration runtimes
Module introduction
Integration runtimes
Azure integration runtimes
Self-hosted integration runtimes
Linking and sharing integration runtimes
SQL Server Integration Services
Executing SSIS packages in ADF
Azure SSIS integration runtimes
SSIS IR cost implications
Lab 9
Quiz 9
Module 10: Working with data flows
Module introduction
ADF mapping data flows
ADF data flow performance and cost implications
ADF Power Queries
Comparing Power Queries to data flows
ADF flowlets
FDF gen 2 data flows
Comparing ADF and FDF data flows
Lab 10
Quiz 10
Module 11: Managing pipelines
Module introduction
Debugging pipelines
Annotations
User properties
Defining dynamic connections by using expressions
Monitoring ADF pipelines
Sending alerts on failure
Copying JSON to create pipelines
Useful 3rd party deployment utilities and tools
Git Integration in Fabric
Monitoring FDF pipelines
Lab 11
Quiz 11
Module 12: Managing Data Factory security
Module introduction
Identity concepts in Azure
RBAC based security
System-assigned managed identities
User-assigned managed identities
Workspace identities in FDF
Using managed identities in connections
Azure Key Vault
Integrating ADF with Azure Key Vault
Integrating FDF with Azure Key Vault
Using key vault secrets in connections
Connecting via private endpoints
Lab 12
Quiz 12
Module 13: Medium complexity worked example
Module introduction
Problem to solve
REST based calls required
Configuring security
Making refresh request
Checking latest refreshes
Filter to current refresh
Retry until complete
Throw error on failure
Module 14: Complex worked example
Module introduction
Problem to solve
Options considered
Overall solution design
File in storage account
Testing requirements
First attempt - add dataset reference
Correct multipart form data
Attachment file contents
Returned preview data
Multipart form data format
Web request content type and body
Configure the web activity for upload
HTTP request timeout
Line endings
Requirement achieved
Further thoughts
Module 15: Advanced ADF and FDF concepts
Module introduction
Validation activity
Azure Data Explorer command
Azure Function activity
Databricks-related activities
WebHook activity
HDInsight-related activities
Data Lake Analytics-related activities
Machine Learning-related activities
Module 16: Next steps
Summary and further steps
×
This is an unpublished lesson. This lesson will not be shown for students unless you set it as Public.
Back to Dashboard
No contents are available in this lesson!
No lessons available !
Back to Dashboard
Lesson contents locked
Enroll to unlock this lesson.
Enroll to unlock
Next Lesson