Spark

Azure Synapse Spark

Get started with SPARK in Azure Synapse Analytics

Using Spark in Azure Synapse Analytics opens up a lot of possibilities to work with your data. Venk joins the Guy in a Cube gang to get you started with setting it up and quickly using data.

Read More
Python Spark

Learn PySpark in 60 Minutes

This Edureka video on PySpark Tutorial will provide you with a detailed and comprehensive knowledge of Pyspark, how it works, the reason why python works best with Apache Spark. You will also learn about RDDs, dataframes and mllib. Time stamps: 00:00 Introduction 00:15 Agenda 00:35 PySpark 01:45 Spark Ecosystem 05:25 Advantages of PySpark 06:10 PySpark […]

Read More
Spark

Comprehensive View on Intervals in Apache Spark 3.2

Here’s an overview of intervals in Apache Spark before version 3.2, and the changes that are coming in the future releases.

Read More
Natural Language Processing Spark

Jeeves Grows Up: An AI Chatbot for Performance and Quality

Jeeves is a chatbot created to simplify data operations management for enterprise Spark clusters. Powered by advanced AI algorithms and an intuitive conversational interface answers to get users in and out of problems quickly. Instead of being stuck to screens displaying logs and metrics, users can now have a more refreshing experience via a two-way […]

Read More
Privacy Spark

Scaling Privacy in a Spark Ecosystem

Privacy has become one of the most important topics in data today. It has more than how do we ingest and consume data but the important factors about how you protect your customer’s rights while balancing the business need. In this video, Privacera CTO, Don Bosco Durai together with Northwestern Mutual to detail an important […]

Read More
Big Data Microsoft Spark

What’s New in .NET for Apache Spark v1.1.1?

.NET for Apache Spark empowers .NET developers to participate in the world of big data analytics. In this episode, Jeremy chats with Michael Rys to discuss some of the new features and capabilities available in this release Related Links .NET for Apache Spark™ .NET for Apache Spark™ tutorial .NET for Apache Spark™ documentation

Read More
Databricks Spark

Advancing Spark – Runtime 8 2 and Advanced Schema Evolution

Another week, another new Databricks Runtime. Runtime 8.2 brings some nice functionality around operational metrics, but the big star of the week is the new Schema Inference & Evolution functionality available through Autoloader. In this video, Simon takes a look through simple schema inference, applying schema hints and watching the schema metadata evolve through the […]

Read More
Spark

Unboxing Spark Standalone Architecture

Big Data Engineering closely examines  Spark Standalone Architecture. Apache Spark has a well-defined layered architecture where all the spark components and layers are loosely coupled. This architecture is further integrated with various extensions and libraries. Apache Spark Architecture is based on two main abstractions: Resilient Distributed Dataset (RDD) Directed Acyclic Graph (DAG)

Read More