Big Data

Azure Big Data Data

Where Should You Put Your Data in Azure?

A frequent question asked is: Where goes what or where should I put my data? With Amy Boyd, Frank (not me) invited different product teams to share what type of data and goes in their service. Let’s meet with Synapse Analytics, Cosmo DB, Azure Data Lake, and Azure Data Explorer product manager. Each one will […]

Read More
Big Data Data Warehouse Databricks

Delta Lake Roadmap 2021 H2: Features Overview

Get a look at the feature roadmap for Delta Lake.

Read More
Big Data

Scaling Zeus to Petabytes of Shuffle Data at Uber

Zeus is an efficient, highly scalable, and distributed shuffle as a service that is powering all Data processing (Spark and Hive) at Uber. Uber runs one of the largest Spark and Hive clusters on top of YARN in the industry which leads to many issues such as hardware failures (Burn out Disks), reliability, and scalability […]

Read More
Big Data Containers Databricks

Scaling your Data Pipelines with Apache Spark on Kubernetes

There is no doubt Kubernetes has emerged as the next generation of cloud native infrastructure to support a wide variety of distributed workloads. Apache Spark has evolved to run both Machine Learning and large scale analytics workloads. There is growing interest in running Apache Spark natively on Kubernetes. By combining the flexibility of Kubernetes and […]

Read More
Big Data Data

Portable UDFs : Write Once, Run Anywhere

While most query engines come with a rich set of functions, it does not cover all the needs of users. In such cases, user defined functions (UDFs) allow users to express their business logic and use it in their queries. It is common for users to use more than one compute engine for solving their […]

Read More
Big Data Databricks

Observability for Data Pipelines With OpenLineage

Data is increasingly becoming core to many products and services. Whether to provide recommendations for users, getting insights on how they use the product, or using machine learning to improve the experience. This creates a critical need for reliable data operations and understanding how data is flowing through our systems. Data pipelines must be auditable, […]

Read More
Big Data Databricks

Clean Your Data Swamp by Migrating Off of Hadoop

In this session, learn how to quickly supplement your on-premises Hadoop environment with a simple, open, and collaborative cloud architecture that enables you to generate greater value with scaled application of analytics and AI on all your data. You will also learn five critical steps for a successful migration to the Databricks Lakehouse Platform along […]

Read More
Big Data

Dealing With Big Data – Computerphile

Big Data sounds may be a buzz word, and is hard to quantify, but the problems with large data sets are very real. Dr Isaac Triguero explains some of the challenges.

Read More
Big Data Data

You Can Do It in SQL

Learn how this one weird trick (Jinja templating) will supercharge your analytics workflows and help you do more, better, faster with SQL. Databricks,

Read More
Big Data

What Is Big Data Analytics & Why It Is Important?

This Edureka ‘What is Big Data Analytics & Why it is Important’ video helps you to understand Big Data in detail. This tutorial will be discussing about evolution of Big Data, factors associated with Big Data and how it transforms the world of Data Analytics.

Read More