Databricks Certified Data Engineer Professional Sample Questions:

1. A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non-overlapping five-minute interval. Incremental state information should be maintained for 10 minutes for late-arriving data.
Streaming DataFrame df has the following schema:
"device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT"
Code block:

Choose the response that correctly fills in the blank within the code block to complete this task.

A) slidingWindow("event_time", "10 minutes")
B) await("event_time + `10 minutes'")
C) withWatermark("event_time", "10 minutes")
D) delayWrite("event_time", "10 minutes")
E) awaitArrival("event_time", "10 minutes")

2. A workspace admin has created a new catalog called finance_data and wants to delegate permission management to a finance team lead without giving them full admin rights. Which privilege should be granted to the finance team lead?

A) GRANT OPTION privilege on the finance_data catalog.
B) ALL PRIVILEGES on the finance_data catalog.
C) MANAGE privilege on the finance_data catalog.
D) Make the finance team lead a metastore admin.

3. The data engineering team has configured a job to process customer requests to be forgotten (have their data deleted). All user data that needs to be deleted is stored in Delta Lake tables using default table settings.
The team has decided to process all deletions from the previous week as a batch job at 1am each Sunday. The total duration of this job is less than one hour. Every Monday at 3am, a batch job executes a series of VACUUM commands on all Delta Lake tables throughout the organization.
The compliance officer has recently learned about Delta Lake's time travel functionality. They are concerned that this might allow continued access to deleted data.
Assuming all delete logic is correctly implemented, which statement correctly addresses this concern?

A) Because the default data retention threshold is 24 hours, data files containing deleted records will be retained until the vacuum job is run the following day.
B) Because the vacuum command permanently deletes all files containing deleted records, deleted records may be accessible with time travel for around 24 hours.
C) Because the default data retention threshold is 7 days, data files containing deleted records will be retained until the vacuum job is run 8 days later.
D) Because Delta Lake time travel provides full access to the entire history of a table, deleted records can always be recreated by users with full admin privileges.
E) Because Delta Lake's delete statements have ACID guarantees, deleted records will be permanently purged from all storage systems as soon as a delete job completes.

4. Each configuration below is identical to the extent that each cluster has 400 GB total of RAM 160 total cores and only one Executor per VM.
Given an extremely long-running job for which completion must be guaranteed, which cluster configuration will be able to guarantee completion of the job in light of one or more VM failures?

A) - Total VMs: 1
- 400 GB per Executor
- 160 Cores/Executor
B) - Total VMs: 16
- 25 GB per Executor
- 10 Cores / Executor
C) - Total VMs: 4
- 100 GB per Executor
- 40 Cores / Executor
D) - Total VMs: 8
- 50 GB per Executor
- 20 Cores / Executor
E) - Total VMs: 2
- 200 GB per Executor
- 80 Cores / Executor

5. The Databricks CLI is use to trigger a run of an existing job by passing the job_id parameter. The response that the job run request has been submitted successfully includes a filed run_id.
Which statement describes what the number alongside this field represents?

A) The globally unique ID of the newly triggered run.
B) The number of times the job definition has been run in the workspace.
C) The total number of jobs that have been run in the workspace.
D) The job_id is returned in this field.
E) The job_id and number of times the job has been are concatenated and returned.

Solutions: