Prepare with Databricks : Associate-Developer-Apache-Spark-3.5 exam braindumps as your best preparation materials

A data engineer is building an Apache Spark™ Structured Streaming application to process a stream of JSON events in real time. The engineer wants the application to be fault-tolerant and resume processing from the last successfully processed record in case of a failure. To achieve this, the data engineer decides to implement checkpoints.
Which code snippet should the data engineer use?

• A. query = streaming_df.writeStream \
  .format("console") \
  .option("checkpoint", "/path/to/checkpoint") \
  .outputMode("append") \
  .start()
• B. query = streaming_df.writeStream \
  .format("console") \
  .outputMode("complete") \
  .start()
• C. query = streaming_df.writeStream \
  .format("console") \
  .outputMode("append") \
  .start()
• D. query = streaming_df.writeStream \
  .format("console") \
  .outputMode("append") \
  .option("checkpointLocation", "/path/to/checkpoint") \
  .start()

An engineer has a large ORC file located at /file/test_data.orc and wants to read only specific columns to reduce memory usage.
Which code fragment will select the columns, i.e., col1, col2, during the reading process?

• A. spark.read.format("orc").load("/file/test_data.orc").select("col1", "col2")
• B. spark.read.orc("/file/test_data.orc").selected("col1", "col2")
• C. spark.read.orc("/file/test_data.orc").filter("col1 = 'value' ").select("col2")
• D. spark.read.format("orc").select("col1", "col2").load("/file/test_data.orc")

A data engineer is asked to build an ingestion pipeline for a set of Parquet files delivered by an upstream team on a nightly basis. The data is stored in a directory structure with a base path of "/path/events/data". The upstream team drops daily data into the underlying subdirectories following the convention year/month/day.
A few examples of the directory structure are:

Which of the following code snippets will read all the data within the directory structure?

• A. df = spark.read.option("inferSchema", "true").parquet("/path/events/data/")
• B. df = spark.read.parquet("/path/events/data/*")
• C. df = spark.read.parquet("/path/events/data/")
• D. df = spark.read.option("recursiveFileLookup", "true").parquet("/path/events/data/")

9 of 55.
Given the code fragment:
import pyspark.pandas as ps
pdf = ps.DataFrame(data)
Which method is used to convert a Pandas API on Spark DataFrame (pyspark.pandas.DataFrame) into a standard PySpark DataFrame (pyspark.sql.DataFrame)?

• A. pdf.to_dataframe()
• B. pdf.to_pandas()
• C. pdf.spark()
• D. pdf.to_spark()

A data engineer is building a Structured Streaming pipeline and wants the pipeline to recover from failures or intentional shutdowns by continuing where the pipeline left off.
How can this be achieved?

• A. By configuring the option recoveryLocation during the SparkSession initialization
• B. By configuring the option recoveryLocation during writeStream
• C. By configuring the option checkpointLocation during readStream
• D. By configuring the option checkpointLocation during writeStream