Apache-spark Archives - GeeksforGeeks

Similar Topics

Web Technologies32.1K+ articles

DSA20.8K+ articles

Python19.6K+ articles

Experiences15.8K+ articles

Interview Experiences14.5K+ articles

School Learning11.1K+ articles

JavaScript9.9K+ articles

Scala1.2K+ articles

Installation Guide1.2K+ articles

Python-Pyspark180+ articles

Apache-spark

11 posts

Wide and Narrow Dependencies in Apache Spark

Last Updated: 23 July 2025

Apache Spark, a powerful distributed computing framework, is designed to process large-scale datasets efficiently across a cluster of machines. However, Dependencies play ...read more

article_image

Apache Flink vs. Apache Spark

Last Updated: 06 October 2026

Apache Flink and Apache Spark are popular open-source distributed processing frameworks used for handling large-scale data. Both support batch and stream processing, but t...read more

Big Data Frameworks

Last Updated: 08 October 2026

Big Data frameworks help organizations process analyze and manage large volumes of data efficiently. They support distributed computing real-time data processing data stre...read more

Spark vs Impala

Last Updated: 01 December 2023

Spark and Impala are the two most common tools used for big data analytics. This article focuses on discussing the pros, cons, and differences between the two tools.What i...read more

Geeks Premier League 2023

Identify corrupted records in a dataset using pyspark

Last Updated: 23 July 2025

There can be datasets that may contain corrupt records. Those records don't follow data-specific rules that are followed by correct records e.g., a corrupt record may have...read more

article_image

Spark - Split array to separate column

Last Updated: 23 July 2025

Apache Spark is a potent big data processing system that can analyze enormous amounts of data concurrently over distributed computer clusters. PySpark is a Python-based in...read more

article_image

PySpark lag() Function

Last Updated: 23 July 2025

The function that allows the user to query on more than one row of a table returning the previous row in the table is known as lag in Python. Apart from returning the offs...read more

article_image

Technical Scripter

Python Programs

Technical Scripter 2022

Drop a column with same name using column index in PySpark

Last Updated: 23 July 2025

In this article, we are going to learn how to drop a column with the same name using column index using Pyspark in Python.Pyspark offers you the essential function 'drop' ...read more

article_image

Technical Scripter

Python Programs

Technical Scripter 2022

Python Framework

How to add a column to a nested struct in a pyspark

Last Updated: 23 July 2025

In this article, we are going to learn how to add a column to a nested struct using Pyspark in Python.Have you ever worked in a Pyspark data frame? If yes, then might sure...read more

article_image

Technical Scripter

Python Programs

Technical Scripter 2022

Python Framework

How to Create Multiple Lags in PySpark

Last Updated: 23 July 2025

In this article, we are going to learn how to create multiple lags using pyspark in Python.What is lag in Pyspark? The lag lets our query on more than one row of a table a...read more

article_image

Technical Scripter

Python Programs

Technical Scripter 2022

Python Framework

Install Apache Spark in a Standalone Mode on Windows

Last Updated: 23 July 2025

Apache Spark is a lightning-fast unified analytics engine used for cluster computing for large data sets like BigData and Hadoop with the aim to run programs parallel acro...read more

article_image

Installation Guide