Established a metadata-driven approach with Azure Synapse Data Pipelines?

2022

TL; DR

Established a metadata-driven approach with Synapse

Session Details

Can we build our Azure Synapse Data Pipelines all with parameters and can we use a Meta Driven Approach? Yes, you can, and I will show you how to. During this session I will show you how, you can use a Meta Driven Approach to achieve the following requirement: • Load Incremental or Full datasets from your source database to Azure Data Lake. • Track history from extracted tables by using the Delta Lake functionality in Azure DataBricks, Synapse Spark or DataFlows. • Make this data available in Azure Synapse Analytics Serverless Pools. • Create logging of above processes. As you can see a lot to talk and to demo about during this session.

3 things you'll get out of this session

Erwin de Kreuk's other proposed sessions for 2026

Build Your AI Foundation with Fabric Metadata-driven Framework - 2026

Empowering Data Products and Data Agents through Data Mesh in Microsoft Fabric - 2026

Fabric Automation All the Way: A Deep Dive into Automating Your Fabric Data Platform - 2026

From Insight to Oversight: Securing Power BI with Purview - 2026

Why Data Quality Matters for AI-Driven Insights - 2026

Erwin de Kreuk's previous sessions

Managing access to data sources in your data estate with Microsoft Purview

One of the new applications within Microsoft Purview Governance Portal are Access Policies. Access policies in Microsoft Purview allow you to manage access to different data systems across your entire data estate. The big advantage of these policies is that you do not have to apply RBAC roles and you have an overview of all applied policies. Are you a data consumer? Then the self-service access policy is an easy way to request access to data while browsing or searching for data. Are you a data producer? Then access policies helps you to easily create and publish access to data sources In this session, I will explain and show you how to create and publish policies and how to set up a self-service access workflow.

Data Governance with Azure Purview - Ask the Experts

Join us for an hour to examine data governance for business and how it can benefit you.

Lake Database with Database Template and Mapping Data with Azure Synapse Analytics

Learn how to easily build your lake database without having to start with a blank canvas on Azure Synapse.

Azure Key Vault, Azure Dev Ops and Data Factory

Azure Key Vault, Azure Dev Ops and Data Factory how do these Azure Services work perfectly together!