Dumping data into Azure Data Lake with Data Factory
2016TL; DR
An introduction with worked examples of the new Data Lake storage platform in Azure.
Session Details
This session is all about putting big data to work. To do that we need to be clear on objectives and to transform a proof of concept into a reliable, resilient and repeatable production process. To make that work we'll show you how to hook up Data Lake to a variety of sources using Data Factory using Visual Studio so that we also get source control. The trick with all of this is to use the scalability of the cloud to minimise the costs involved by turning on and off the services we need. Inevitably things will go wrong with our demo and that is part of the plan as we want to show you how to diagnose problems and fix them.
3 things you'll get out of this session
Speakers
Allan Mitchell's previous sessions
Getting Started with Azure Search
Need to search your data efficiently whilst in the cloud? This session introduces you to MSFT's highly scalable search solution in Azure
A Look at 3 Big Data Tools
This session will look at SQOOP, PIG and HIVE. Three tools which are essential in your Big Data Toolbox. We'll look at what they do, how they do it and why you would want to use them
Incremental Loading of Data using CDC in SQL Server 2012
More data, less time, storing everything. These ingredients mean it is becoming imperative we do not move the kitchen sink everytime we do our ETL Come to this session to find out how.
StreamInsight-Solving Business problems (Deep Dive)
Event Stream Processing opens up a mass of opportunities for reacting to patterns in your data in realtime. This session shows you how.
Data Quality Services-Making Data Suitable for Business
Data Quality Services is a new product coming in the SQL Server Denali release of Microsoft SQL Server. Come to this session to understand how it can take your data cleansing to the next level.
StreamInsight Deep Dive
Look deep under the covers at what is involved when building a StreamInsight application.