PySpark 面试题与答案

Question: Explain the concept of 'broadcast' variables in PySpark.

Answer: 'Broadcast' variables are read-only variables cached on each node of a cluster to efficiently distribute large read-only data structures.

Example:

from pyspark.sql.functions import broadcast

result = df1.join(broadcast(df2), 'key')