Sensitive data written to disk unencrypted in Spark
Apache Spark
Affected versions
| Product | Affected | Fixed in |
|---|---|---|
| org.apache.spark:spark-core_2.11 Maven |
|
|
| pyspark PyPI |
|
|
Apache Spark
| Product | Affected | Fixed in |
|---|---|---|
| org.apache.spark:spark-core_2.11 Maven |
|
|
| pyspark PyPI |
|
|
Prior to Spark 2.3.3, in certain situations Spark would write user data to local disk unencrypted, even if spark.io.encryption.enabled=true. This includes cached blocks that are fetched to disk (controlled by spark.maxRemoteBlockSizeFetchToMem); in SparkR, using parallelize; in Pyspark, using broadcast and parallelize; and use of python udfs.
From the CVE record · apache
Before Spark 2.3.3, certain operations can write user data to local disk unencrypted even when spark.io.encryption.enabled=true. This can expose that data to someone with access to the local disk; the input does not describe a network attack or required user interaction.
Teams running Apache Spark, Maven org.apache.spark:spark-core_2.11, or PyPI pyspark versions earlier than 2.3.3 should check their deployments.