PySpark 面试题与答案

Question: Explain the use of the 'broadcast' hint in PySpark.

Answer: The 'broadcast' hint is used to explicitly instruct PySpark to use a broadcast join strategy for better performance, especially when one DataFrame is significantly smaller than the other.

Example:

from pyspark.sql.functions import broadcast

result = df1.join(broadcast(df2), 'key')