Spark
Spark Executor & Driver Memory Calculation
-
Frank
- January 23, 2021
- apache hadoop
- Apache Spark
- apache spark architecture
- apache spark rdd
- Big Data
- big data analysis
- big data engineering
- big data in tamil
- big data use cases
- create spark RDD
- driver memory
- executor memory
- hadoop in tamil
- num cores
- num executors
- spark architecture
- spark big data
- spark data frames
- spark executor memory
- spark framework overview
- spark memory
- spark memory calculation
- spark rdd
- spark standalone cluster
- spark submit
- what is big data
Big Data Engineering takes a closer look at Spark.
Read More
Spark
Unboxing Spark Standalone Architecture
-
Frank
- January 22, 2021
- apache hadoop
- apache hive
- Apache Spark
- apache spark architecture
- Big Data
- big data in tamil
- big data training
- Data Engineering
- Hadoop
- hadoop ecosystem
- hadoop framework
- hadoop in tamil
- hadoop overview
- hadoop training
- scala wordcount
- spark architecture
- spark big data
- spark data frames
- spark in tamil
- spark rdd
- spark standalone cluster
- Spark Streaming
- spark vs hadoop
- spark wordcount
- what is big data
- what is hadoop
Big Data Engineering closely examines Spark Standalone Architecture. Apache Spark has a well-defined layered architecture where all the spark components and layers are loosely coupled. This architecture is further integrated with various extensions and libraries. Apache Spark Architecture is based on two main abstractions: Resilient Distributed Dataset (RDD) Directed Acyclic Graph (DAG)
Read More