22-25 April 2026

Delta and Databricks vs SQL Server

2024

TL; DR

Are you a DBA trying to learn more about Lakehouse’s? Or a Data Engineer wanting to better understand traditional Data Warehousing? Whether you are from a DBA or Data Engineering background, understanding the differences between traditional Warehousing systems and Lakehouse’s is invaluable. This allows us to work with both old and new, whether that be managing both side by side, ingesting from a traditional system into a Lakehouse, or moving completely away from a traditional SQL Warehouse to a Data Lake In this session we will do direct comparisons between features/functionality , illustrating how these different tools are ultimately the same and very different at the same time. Finally, we will talk about how these two technologies can be part of the same data platform solution!

Session Details

Once upon a time, well 1989, we had the Data Warehouse in SQL Server and life was good in the land. It did have its challenges, particularly around loading/storing complex data types as well as the Budget! As data grew larger and more varied, the warehouse became too rigid and opinionated.

In 2012 analytics use cases were growing and Microsoft launched Column Store Index but were very limited however there was talk of a new land with new ideas. In 2013 databricks started a venture which brought a new approach to data warehouses with the separation of storage and compute. As we no longer needed the controls of a transaction database, this was lost in the changes along with many features like ACID.

Data lakes grew with cheap cloud storage from cloud provider and databricks became our compute. Things were great in this new land but the same protections were not in place, as ACID had been left behind in the old land. This was the frontiers where life was harsh without consistency or durability where mistakes could cost you.

Times were changing in the old lands of SQL Server, as cluster column indexes became usable, and the query engine was becoming better at adapting the to the diverse types of queries and making many of the Enterprise features free in 2016.

Things were much more liberal in the new lands as in 2022 Databricks open-sourced the entire code-base, including lots of advanced features that were previously Databricks-only. SQL Server also found its new offering in the cloud which changed its place in our data platform.

This is all great, but how do those who have been using traditional Warehousing tools, in particular SQL Server, make the leap to Delta and Databricks? In this session we will explore this question. We will do direct comparisons between features/functionality, illustrating how these different tools are ultimately the same and very different at the same time. Finally, we will talk about how these two technologies can be part of the same data platform solution! You could be a DBA, BI Developer or Data Engineer, whatever type of Data Professional you are, this session will compare the differences and help you understand them.


The key areas we will explore are:

Optimization
Storage
Compute
Security

3 things you'll get out of this session

Speakers

Anna-Maria Wykes

linkedin.com/in/anna-maria-wykes-31939454/recent-activity/posts

Anna-Maria Wykes's previous sessions

How to Run Code Clubs for Neurodiverse Children
Code Clubs offer an amazing opportunity to introduce our next generation to coding, with simple brightly colored drag-and-drop tooling to get them started, we are successfully inspiring many to join the tech industry. In this session I want to talk you through my journey setting up a Code Club for neurodiverse children, what I found worked, and what doesn’t. I hope that from this session you will be inspired to follow the same path I have, using your amazing tech experience to empower some of the most vulnerable children, enabling them to become inspired not just by coding, but the tech industry itself.
 
Introduction to the wonders of Azure DevOps
Azure DevOps is the leading deployment tool for build and release solutions end to end. It helps you plan your Agile project, manages Git code, and deploys solutions using Continuous Integration (CI) and Continuous Deployment (CD) pipelines. In this session we will cover some of the core components of Azure DevOps and show you how to implement a secure deployment pipeline, using unit tests and gating with your CI builds and CD releases.
 
Automate the deployment of Databricks components using Terraform
Introduction into Terraform, Databricks provider and steps required to build an automated solution to provision Databricks workspace and resources into Azure cloud platform using Terraform.
 
So you want to be a Data Engineer?
An introduction to becoming a Data Engineer, Anna, Mikey and Ust will introduce the technology stack, tools and development skills needed for data engineering and show you how and where to go to learn them. We'll also show you how the skills you already have can kickstart your journey to becoming a Data Engineer.
 
Scala for Big Data the Big Picture
An opportunity to explore Scala, and why it is truly a “Data Engineers language”. Using Azure Functions, Data Factory, Azure Data Lake Gen2 and Databricks the basics will be explored, followed by real world examples