Apache Spark 면접 질문과 답변

Question: What is the Broadcast variable in Spark and when is it used?

Answer: A Broadcast variable is a read-only variable cached on each worker node. It is used to efficiently distribute large read-only data structures, such as lookup tables, to all tasks in a Spark job.

Example:

val broadcastVar = sc.broadcast(Array(1, 2, 3))