Data Engineering

Azure Databricks

Supercharge Data and AI Innovation with Azure Databricks

 Imagine you’re given a toolbox—a really good one. It’s packed with every tool you could possibly need to build and manage a vast, complex machinery. Now, your challenge is not only to learn how to use each tool effectively but to integrate them so seamlessly that the machinery runs more efficiently than ever before. That, […]

Read More
Data Databricks

How to Simplify Streaming Pipelines with Lakeflow

Imagine you’re behind the steering wheel of a futuristic car, hurtling through city streets at speeds only seen in sci-fi films. Every red light, stop sign, and curve in the road demands instant, flawless navigation. Now, replace that thrilling drive with the world of data streaming, where every millisecond of processing speed can lead to […]

Read More
AI Data

AI’s Silent Killer: Technical Debt & the Hidden Costs of Bad Data

Recently, I was a guest on a podcast on AI and the importance of Data Engineering. Delving into the technological endeavors of AI is like embarking on a grand voyage. It’s thrilling but fraught with hidden icebergs and unforeseen currents. The push towards integrating AI into business operations stirs up a mix of excitement and […]

Read More
Data Data Driven

When Your Data Goes Off the Rails: A Conversation with Barr Moses

In the latest episode of Data Driven, we had the pleasure of chatting with Barr Moses, CEO and co-founder of Monte Carlo—not the casino, not the car, but the company making sure your data doesn’t drive your business off a cliff. If you’ve ever stared at a dashboard and thought, “Wait, that can’t be right,” […]

Read More
Data Visualization PowerBI

Maximizing Power BI Efficiency: Alternatives to Measure Tables

Creating and managing measures is a fundamental aspect of data analytics. Measures, which are numerical metrics based on column data, are essential for performing calculations and generating insights from datasets. Traditionally, many data analysts have used measures tables as a convenient way to store and organize their measures. These tables collate all measures in one […]

Read More
Azure Azure Synapse Microsoft

What’s New in Fabric? January 2025

One of the standout features of Microsoft Fabric is its adaptability. As technology continues to evolve at an unprecedented pace, it is essential for businesses to stay ahead of the curve. Microsoft Fabric is designed to do just that. With its robust framework and flexible architecture, it can be customized to meet the unique needs […]

Read More
AI Career

There’s No Such Thing as an ML Job

The journey into the world of machine learning (ML) is often marked by countless nights spent studying gradient descent, fine-tuning Transformers, and building LLMS. The excitement of landing your first job as an ML engineer is often fueled by the prospect of working with cutting-edge algorithms and training state-of-the-art neural networks. However, a month into […]

Read More
Agentic AI

Andrew Ng Explores The Rise Of AI Agents And Agentic Reasoning | BUILD 2024 Keynote

Artificial Intelligence (AI) is a transformative technology that, like electricity, is increasingly becoming indispensable in our daily lives. This transformative power of AI is due to its ability to automate complex tasks, generate insights from vast amounts of data, and enable new applications that were not possible before. Today, I would like to discuss the […]

Read More
Career Data

Should You Still Learn SSIS in 2025?

As the holiday season brings a mix of nostalgia and anticipation for the future, it’s a fitting time to talk about SQL Server Integration Services (SSIS) — a technology as classic as Christmas carols but often met with mixed reactions, much like fruitcake. Despite being around for nearly two decades, SSIS remains a cornerstone in […]

Read More
AI Big Data Career Data Data Driven

Tis the Season for SSIS: Celebrating a Data Engineering Classic

The holiday season is a time for reflection and celebration, and here at Data Driven, we’re diving into a topic as classic as Christmas carols and as warmly debated as fruitcake: SQL Server Integration Services (SSIS). In this festive podcast episode, Frank and Andy unwrap the enduring significance of SSIS, a stalwart in the data […]

Read More