Gaurav Malhotra

Azure Data

Enable unified data governance with Azure Purview

The recently announced Azure Purview is a unified data governance service that helps you manage and govern your on-premises, multi-cloud, and software-as-a-service (SaaS) data. Gaurav Malhotra joins Scott Hanselman to show how easy it is to create a holistic, up-to-date map of your data landscape with automated data discovery, sensitive data classification, and end-to-end data […]

Read More
Azure Big Data Data

Data Prep and Wrangling Data in Azure Data Factory

Gaurav Malhotra joins Scott Hanselman to show how wrangling data flows in Azure Data Factory. This provides  a code-free, serverless environment that simplifies data preparation in the cloud and scales to any data size with no infrastructure management required. It uses the industry-leading Power Query data preparation technology (also used in Power Platform dataflows, Excel, […]

Read More
Azure Machine Learning

How to Execute Azure Machine Learning service pipelines in Azure Data Factory

Gaurav Malhotra joins Scott Hanselman to show how you can run your Azure Machine Learning (AML) service pipelines as a step in your Azure Data Factory (ADF) pipelines. This enables you to run your machine learning models with data from multiple sources (85+ data connectors supported in ADF). This seamless integration enables batch prediction scenarios […]

Read More
Azure Data

Create Dependent Pipelines in your Azure Data Factory

Data integration is complex and has many moving parts that spans across hybrid data environments. Typically, data integration projects have dependencies upstream and downstream making dependencies an important aspect to consider in any job scheduling. Gaurav Malhotra joins Scott Hanselman to show how you can create dependent pipelines in Azure Data Factory by creating dependencies […]

Read More
Azure Big Data Data

Code-free modern data warehouse using Azure SQL DW and Data Factory

Gaurav Malhotra joins Scott Hanselman to show how to build a modern data warehouse solution from ingress of structured, unstructured, semi-structured data to code-free data transformation at scale and finally to extracting business insights into your Azure SQL Data Warehouse.      Related resources: Azure Data Factory overview Azure SQL Data Warehouse overview Azure Data Factory docs […]

Read More
Azure Big Data Data

Enhanced Productivity with Azure Data Factory Visual Tools

Gaurav Malhotra joins Scott Hanselman to discuss the Azure Data Factory visual tools, which enable you to iteratively create, configure, test, deploy, and monitor data integration pipelines. We took into account your feedback to enable functional, performance, and security improvements to the visual tools. For more information: Enhance productivity using Azure Data Factory Visual Tools […]

Read More
Azure Big Data Data

Monitor Azure Data Factory Pipelines with OMS

Gaurav Malhotra joins Scott Hanselman to discuss Azure Data Factory (ADF) integration with Azure Monitor, which enables you to route your data factory metrics to Operations and Management Suite (OMS). Get the Azure Data Factory Analytics OMS service pack from the Azure marketplace. For more information: Monitor Azure Data Factory with Operations Management Suite (OMS) (docs) […]

Read More
Azure Big Data Data Science Databricks

Execute Jars and Python scripts on Azure Databricks using Data Factory

Gaurav Malhotra discusses how you can operationalize Jars and Python scripts running on Azure Databricks as an activity step in a Data Factory pipeline. For more information: Transform data by running a Jar activity in Azure Databricks docs Transform data by running a Python activity in Azure Databricks docs Azure Databricks overview Azure Data Factory […]

Read More
Azure Big Data Databricks

Using Azure Databricks and Data Factory

Today’s business managers depend heavily on reliable data integration systems that run complex ETL/ELT workflows (extract, transform/load and load/transform data). Gaurav Malhotra joins Scott Hanselman to discuss how you can iteratively build, debug, deploy, and monitor your data integration workflows (including analytics workloads in Azure Databricks) using Azure Data Factory pipelines. For more information: Ingest, […]

Read More
Azure Big Data

Visually Building Pipelines for Azure Data Factory V2

Gaurav Malhotra shows Donovan Brown how you can now visually build pipelines for Azure Data Factory V2 and be more productive by getting pipelines up & running quickly without writing any code. For more information, see: ADF v2: Visual Tools enabled in public preview (blog) Azure Data Factory V2 Preview Documentation (docs) Follow @donovanbrown Follow […]

Read More