Data engineering is about 70% of any data pipeline today, and without having the experience to implement a data engineering pipeline well, there is no value to be accumulated from your data.
In this session from Microsoft Ignite we discuss the best practices and demonstrate how a data engineer can develop and orchestrate the big data pipeline, including: data ingestion and orchestration using Azure Data Factory; data curation, cleansing and transformation using Azure Databricks; data loading into Azure SQL Data Warehouse for serving your BI tools.
Watch and learn how to effectively do the ETL/ELT process combined with advanced capabilities such as monitoring the jobs, getting alerts, jobs retrial, set permissions, and much more.

On the podcast Andy Leonard and I create, we love experimenting around here and examining the resulting data. After all, we are Data Driven: not just in name but also in spirit.In this webinar Andy recorded, he also streamed it live on our Facebook page.

We thought it was good enough to share with our larger audience here.Let us know what you think. Both Frank and Andy have been recording/streaming their live events and we’re curious to hear what you have to say about this innovation in how we podcast.

Press the play button below to listen here or visit the show page at DataDriven.tv

 

Frank and Andy talked about doing a Deep Dive show where they take a deep look into a particular data science technology, term, or methodology.  And now, they deliver!

In this very first Deep Dive, Frank and Andy discuss the differences between Data Science and Data Engineering, where they overlap, where they differ, and why so many C-level execs can’t seem to figure out the deltas.