An Engineers Guide To Realtime Data Handling And Analytics

The velocity of data is getting faster across many industries, fuelled by the business demand to gain insights and value from sources in near real-time. This necessity is then allowing decision makers to pivot and ultimately stay ahead of the competition. Furthermore, the growth of the internet of things and ‘smart’ devices now means the volume of that high velocity data has exploded. Meeting this demand requires new concepts and new designs for data/solution architects, with high throughput ingestion endpoints and query stream tools that can perform aggregations ‘on the fly’.

In this course, we will address the above head on. Discussing and designing architectures that can scale and burst for high throughput events. Querying using both SQL and KQL to blend stream and batch data feeds for downstream reporting.

As a platform, we’ll use Azure Event Hub and Azure Stream Analytics to ingest and handle that initial data stream. Before applying the same patterns to other resources in Microsoft Fabric and Azure Data Explorer. Understanding the patterns to apply as an architect vs the tooling available for delivery.