Get Prepared for Your Databricks-Certified-Data-Engineer-Professional Exam With Actual Databricks Study Guide!
Pass Your Next Databricks-Certified-Data-Engineer-Professional Certification Exam Easily & Hassle Free
NEW QUESTION # 10
Each configuration below is identical to the extent that each cluster has 400 GB total of RAM 160 total cores and only one Executor per VM.
Given an extremely long-running job for which completion must be guaranteed, which cluster configuration will be able to guarantee completion of the job in light of one or more VM failures?
- A. - Total VMs: 2
- 200 GB per Executor
- 80 Cores / Executor - B. - Total VMs: 4
- 100 GB per Executor
- 40 Cores / Executor - C. - Total VMs: 16
- 25 GB per Executor
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from
- 10 Cores / Executor - D. - Total VMs: 8
- 50 GB per Executor
- 20 Cores / Executor - E. - Total VMs: 1
- 400 GB per Executor
- 160 Cores/Executor
Answer: C
Explanation:
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from
NEW QUESTION # 11
Which statement describes Delta Lake Auto Compaction?
- A. Data is queued in a messaging bus instead of committing data directly to memory; all data is committed from the messaging bus in one batch once the job is complete.
- B. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 1 GB.
- C. Before a Jobs cluster terminates, optimize is executed on all tables modified during the most recent job.
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from - D. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 128 MB.
- E. Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written.
Answer: D
Explanation:
This is the correct answer because it describes the behavior of Delta Lake Auto Compaction, which is a feature that automatically optimizes the layout of Delta Lake tables by coalescing small files into larger ones. Auto Compaction runs as an asynchronous job after a write to a table has succeeded and checks if files within a partition can be further compacted. If yes, it runs an optimize job with a default target file size of 128 MB. Auto Compaction only compacts files that have not been compacted previously.
NEW QUESTION # 12
A production cluster has 3 executor nodes and uses the same virtual machine type for the driver and executor.
When evaluating the Ganglia Metrics for this cluster, which indicator would signal a bottleneck caused by code executing on the driver?
- A. Bytes Received never exceeds 80 million bytes per second
- B. Network I/O never spikes
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from - C. Total Disk Space remains constant
- D. The five Minute Load Average remains consistent/flat
- E. Overall cluster CPU utilization is around 25%
Answer: E
Explanation:
This is the correct answer because it indicates a bottleneck caused by code executing on the driver. A bottleneck is a situation where the performance or capacity of a system is limited by a single component or resource. A bottleneck can cause slow execution, high latency, or low throughput. A production cluster has 3 executor nodes and uses the same virtual machine type for the driver and executor. When evaluating the Ganglia Metrics for this cluster, one can look for indicators that show how the cluster resources are being utilized, such as CPU, memory, disk, or network. If the overall cluster CPU utilization is around 25%, it means that only one out of the four nodes (driver + 3 executors) is using its full CPU capacity, while the other three nodes are idle or underutilized. This suggests that the code executing on the driver is taking too long or consuming too much CPU resources, preventing the executors from receiving tasks or data to process. This can happen when the code has driver-side operations that are not parallelized or distributed, such as collecting large amounts of data to the driver, performing complex calculations on the driver, or using non-Spark libraries on the driver.
NEW QUESTION # 13
Which statement characterizes the general programming model used by Spark Structured Streaming?
- A. Structured Streaming uses specialized hardware and I/O streams to achieve sub-second latency for data transfer.
- B. Structured Streaming is implemented as a messaging bus and is derived from Apache Kafka.
- C. Structured Streaming leverages the parallel processing of GPUs to achieve highly parallel data throughput.
- D. Structured Streaming models new data arriving in a data stream as new rows appended to an unbounded table.
- E. Structured Streaming relies on a distributed network of nodes that hold incremental state values for cached stages.
Answer: D
Explanation:
The key idea in Structured Streaming is to treat a live data stream as a table that is being continuously appended. This leads to a new stream processing model that is very similar to a batch processing model. You will express your streaming computation as standard batch-like query as on a static table, and Spark runs it as an incremental query on the unbounded input table. Let's understand this model in more detail.
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html
NEW QUESTION # 14
A data engineer is testing a collection of mathematical functions, one of which calculates the area under a curve as described by another function.
Which kind of the test does the above line exemplify?
- A. Integration
- B. End-to-end
- C. Manual
- D. Unit
- E. functional
Answer: D
Explanation:
A unit test is designed to verify the correctness of a small, isolated piece of code, typically a single function. Testing a mathematical function that calculates the area under a curve is an example of a unit test because it is testing a specific, individual function to ensure it operates as expected.
NEW QUESTION # 15
Although the Databricks Utilities Secrets module provides tools to store sensitive credentials and avoid accidentally displaying them in plain text users should still be careful with which credentials are stored here and which users have access to using these secrets.
Which statement describes a limitation of Databricks Secrets?
- A. Account administrators can see all secrets in plain text by logging on to the Databricks Accounts console.
- B. The Databricks REST API can be used to list secrets in plain text if the personal access token has proper credentials.
- C. Because the SHA256 hash is used to obfuscate stored secrets, reversing this hash will display Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from the value in plain text.
- D. Secrets are stored in an administrators-only table within the Hive Metastore; database administrators have permission to query this table by default.
- E. Iterating through a stored secret and printing each character will display secret contents in plain text.
Answer: B
Explanation:
This is the correct answer because it describes a limitation of Databricks Secrets. Databricks Secrets is a module that provides tools to store sensitive credentials and avoid accidentally displaying them in plain text. Databricks Secrets allows creating secret scopes, which are collections of secrets that can be accessed by users or groups. Databricks Secrets also allows creating and managing secrets using the Databricks CLI or the Databricks REST API. However, a limitation of Databricks Secrets is that the Databricks REST API can be used to list secrets in plain text if the personal access token has proper credentials. Therefore, users should still be careful with which credentials are stored in Databricks Secrets and which users have access to using these secrets.
NEW QUESTION # 16
A data architect has designed a system in which two Structured Streaming jobs will concurrently write to a single bronze Delta table. Each job is subscribing to a different topic from an Apache Kafka source, but they will write data with the same schema. To keep the directory structure simple, a data engineer has decided to nest a checkpoint directory to be shared by both streams.
The proposed directory structure is displayed below:
Which statement describes whether this checkpoint directory structure is valid for the given scenario and why?
- A. No; each of the streams needs to have its own checkpoint directory.
- B. No; only one stream can write to a Delta Lake table.
- C. No; Delta Lake manages streaming checkpoints in the transaction log.
- D. Yes; both of the streams can share a single checkpoint directory.
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from - E. Yes; Delta Lake supports infinite concurrent writers.
Answer: A
Explanation:
This is the correct answer because checkpointing is a critical feature of Structured Streaming that provides fault tolerance and recovery in case of failures. Checkpointing stores the current state and progress of a streaming query in a reliable storage system, such as DBFS or S3. Each streaming query must have its own checkpoint directory that is unique and exclusive to that query. If two streaming queries share the same checkpoint directory, they will interfere with each other and cause unexpected errors or data loss.
NEW QUESTION # 17
The data architect has mandated that all tables in the Lakehouse should be configured as external (also known as "unmanaged") Delta Lake tables.
Which approach will ensure that this requirement is met?
- A. When configuring an external data warehouse for all table storage, leverage Databricks for all ELT.
- B. When a database is being created, make sure that the LOCATION keyword is used.
- C. When tables are created, make sure that the EXTERNAL keyword is used in the CREATE TABLE statement.
- D. When the workspace is being configured, make sure that external cloud object storage has been mounted.
- E. When data is saved to a table, make sure that a full file path is specified alongside the Delta format.
Answer: C
Explanation:
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from To create an external or unmanaged Delta Lake table, you need to use the EXTERNAL keyword in the CREATE TABLE statement. This indicates that the table is not managed by the catalog and the data files are not deleted when the table is dropped. You also need to provide a LOCATION clause to specify the path where the data files are stored.
For example:
CREATE EXTERNAL TABLE events ( date DATE, eventId STRING, eventType STRING, data STRING) USING DELTA LOCATION `/mnt/delta/events'; This creates an external Delta Lake table named events that references the data files in the
`/mnt/delta/events' path. If you drop this table, the data files will remain intact and you can recreate the table with the same statement.
NEW QUESTION # 18
Which configuration parameter directly affects the size of a spark-partition upon ingestion of data into Spark?
- A. spark.sql.adaptive.advisoryPartitionSizeInBytes
- B. spark.sql.adaptive.coalescePartitions.minPartitionNum
- C. spark.sql.files.openCostInBytes
- D. spark.sql.autoBroadcastJoinThreshold
- E. spark.sql.files.maxPartitionBytes
Answer: E
Explanation:
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from This is the correct answer because spark.sql.files.maxPartitionBytes is a configuration parameter that directly affects the size of a spark-partition upon ingestion of data into Spark. This parameter configures the maximum number of bytes to pack into a single partition when reading files from file- based sources such as Parquet, JSON and ORC. The default value is 128 MB, which means each partition will be roughly 128 MB in size, unless there are too many small files or only one large file.
NEW QUESTION # 19
The Databricks workspace administrator has configured interactive clusters for each of the data engineering groups. To control costs, clusters are set to terminate after 30 minutes of inactivity.
Each user should be able to execute workloads against their assigned clusters at any time of the day.
Assuming users have been added to a workspace but not granted any permissions, which of the following describes the minimal permissions a user would need to start and attach to an already configured cluster.
- A. Cluster creation allowed. "Can Restart" privileges on the required cluster
- B. "Can Manage" privileges on the required cluster
- C. Cluster creation allowed. "Can Attach To" privileges on the required cluster
- D. "Can Restart" privileges on the required cluster
- E. Workspace Admin privileges, cluster creation allowed. "Can Attach To" privileges on the required cluster
Answer: D
Explanation:
https://learn.microsoft.com/en-us/azure/databricks/security/auth-authz/access-control/cluster-acl
https://docs.databricks.com/en/security/auth-authz/access-control/cluster-acl.html Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from
NEW QUESTION # 20
A user wants to use DLT expectations to validate that a derived table report contains all records from the source, included in the table validation_copy.
The user attempts and fails to accomplish this by adding an expectation to the report table definition.
Which approach would allow using DLT expectations to validate all expected records are present in this table?
- A. Define a temporary table that perform a left outer join on validation_copy and report, and define an expectation that no report key values are null
- B. Define a function that performs a left outer join on validation_copy and report and report, and check against the result in a DLT expectation for the report table
- C. Define a view that performs a left outer join on validation_copy and report, and reference this view in DLT expectations for the report table
- D. Define a SQL UDF that performs a left outer join on two tables, and check if this returns null values for report key values in a DLT expectation for the report table.
Answer: C
Explanation:
To validate that all records from the source are included in the derived table, creating a view that performs a left outer join between the validation_copy table and the report table is effective. The view can highlight any discrepancies, such as null values in the report table's key columns, indicating missing records. This view can then be referenced in DLT (Delta Live Tables) expectations for the report table to ensure data integrity. This approach allows for a comprehensive comparison between the source and the derived table.
NEW QUESTION # 21
The data engineering team has configured a job to process customer requests to be forgotten (have their data deleted). All user data that needs to be deleted is stored in Delta Lake tables using default table settings.
The team has decided to process all deletions from the previous week as a batch job at 1am each Sunday. The total duration of this job is less than one hour. Every Monday at 3am, a batch job executes a series of VACUUM commands on all Delta Lake tables throughout the organization.
The compliance officer has recently learned about Delta Lake's time travel functionality. They are concerned that this might allow continued access to deleted data.
Assuming all delete logic is correctly implemented, which statement correctly addresses this concern?
- A. Because Delta Lake time travel provides full access to the entire history of a table, deleted records can always be recreated by users with full admin privileges.
- B. Because the default data retention threshold is 7 days, data files containing deleted records will be retained until the vacuum job is run 8 days later.Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from
- C. Because the vacuum command permanently deletes all files containing deleted records, deleted records may be accessible with time travel for around 24 hours.
- D. Because the default data retention threshold is 24 hours, data files containing deleted records will be retained until the vacuum job is run the following day.
- E. Because Delta Lake's delete statements have ACID guarantees, deleted records will be permanently purged from all storage systems as soon as a delete job completes.
Answer: B
Explanation:
https://learn.microsoft.com/en-us/azure/databricks/delta/vacuum
NEW QUESTION # 22
In order to prevent accidental commits to production data, a senior data engineer has instituted a policy that all development work will reference clones of Delta Lake tables. After testing both deep and shallow clone, development tables are created using shallow clone. A few weeks after initial table creation, the cloned versions of several tables implemented as Type 1 Slowly Changing Dimension (SCD) stop working. The transaction logs for the source tables show that vacuum was run the day before.
Why are the cloned tables no longer working?
- A. The metadata created by the clone operation is referencing data files that were purged as invalid by the vacuum command
- B. The data files compacted by vacuum are not tracked by the cloned metadata; running refresh on the cloned table will pull in recent changes.
- C. Because Type 1 changes overwrite existing records, Delta Lake cannot guarantee data consistency for cloned tables.
- D. Tables created with SHALLOW CLONE are automatically deleted after their default retention threshold of 7 days.
- E. Running vacuum automatically invalidates any shallow clones of a table; deep clone should always be used when a cloned table will be repeatedly queried.
Answer: A
Explanation:
In Delta Lake, a shallow clone creates a new table by copying the metadata of the source table without duplicating the data files. When the vacuum command is run on the source table, it removes old data files that are no longer needed to maintain the transactional log's integrity, potentially including files referenced by the shallow clone's metadata. If these files are purged, the shallow cloned tables will reference non-existent data files, causing them to stop working properly. This highlights the dependency of shallow clones on the source table's data files and the impact of data management operations like vacuum on these clones.
NEW QUESTION # 23
A nightly job ingests data into a Delta Lake table using the following code:
The next step in the pipeline requires a function that returns an object that can be used to manipulate new records that have not yet been processed to the next table in the pipeline.
Which code snippet completes this function definition?
- A. return spark.read.option("readChangeFeed", "true").table ("bronze")
- B. def new_records():
- C. return spark.readStream.load("bronze")
- D. return spark.readStream.table("bronze")
- E.
Answer: E
Explanation:
https://docs.databricks.com/en/delta/delta-change-data-feed.html
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from
NEW QUESTION # 24
A nightly batch job is configured to ingest all data files from a cloud object storage container where records are stored in a nested directory structure YYYY/MM/DD. The data for each date represents all records that were processed by the source system on that date, noting that some records may be delayed as they await moderator approval. Each entry represents a user review of a product and has the following schema:
user_id STRING, review_id BIGINT, product_id BIGINT, review_timestamp TIMESTAMP, review_text STRING The ingestion job is configured to append all data for the previous date to a target table reviews_raw with an identical schema to the source system. The next step in the pipeline is a batch write to propagate all new records inserted into reviews_raw to a table where data is fully deduplicated, validated, and enriched.
Which solution minimizes the compute costs to propagate this batch of data?
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from
- A. Filter all records in the reviews_raw table based on the review_timestamp; batch append those records produced in the last 48 hours.
- B. Reprocess all records in reviews_raw and overwrite the next table in the pipeline.
- C. Use Delta Lake version history to get the difference between the latest version of reviews_raw and one version prior, then write these records to the next table.
- D. Perform a batch read on the reviews_raw table and perform an insert-only merge using the natural composite key user_id, review_id, product_id, review_timestamp.
- E. Configure a Structured Streaming read against the reviews_raw table using the trigger once execution mode to process new records as a batch job.
Answer: E
Explanation:
https://www.databricks.com/blog/2017/05/22/running-streaming-jobs-day-10x-cost-savings.html
NEW QUESTION # 25
A junior data engineer is migrating a workload from a relational database system to the Databricks Lakehouse. The source system uses a star schema, leveraging foreign key constrains and multi-table inserts to validate records on write.
Which consideration will impact the decisions made by the engineer while migrating this workload?
- A. All Delta Lake transactions are ACID compliance against a single table, and Databricks does not enforce foreign key constraints.
- B. Databricks only allows foreign key constraints on hashed identifiers, which avoid collisions in highly-parallel writes.
- C. Foreign keys must reference a primary key field; multi-table inserts must leverage Delta Lake's upsert functionality.
- D. Databricks supports Spark SQL and JDBC; all logic can be directly migrated from the source system without refactoring.
- E. Committing to multiple tables simultaneously requires taking out multiple table locks and can lead to a state of deadlock.
Answer: A
NEW QUESTION # 26
A data engineer wants to join a stream of advertisement impressions (when an ad was shown) with another stream of user clicks on advertisements to correlate when impression led to monitizable clicks.
Which solution would improve the performance?
- A.
- B.
- C.
- D.
Answer: B
Explanation:
When joining a stream of advertisement impressions with a stream of user clicks, you want to Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from minimize the state that you need to maintain for the join. Option A suggests using a left outer join with the condition that clickTime == impressionTime, which is suitable for correlating events that occur at the exact same time. However, in a real-world scenario, you would likely need some leeway to account for the delay between an impression and a possible click. It's important to design the join condition and the window of time considered to optimize performance while still capturing the relevant user interactions. In this case, having the watermark can help with state management and avoid state growing unbounded by discarding old state data that's unlikely to match with new data.
NEW QUESTION # 27
The data engineering team has configured a Databricks SQL query and alert to monitor the values in a Delta Lake table. The recent_sensor_recordings table contains an identifying sensor_id alongside the timestamp and temperature for the most recent 5 minutes of recordings.
The below query is used to create the alert:
The query is set to refresh each minute and always completes in less than 10 seconds. The alert is set to trigger when mean (temperature) > 120. Notifications are triggered to be sent at most Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from every 1 minute.
If this alert raises notifications for 3 consecutive minutes and then stops, which statement must be true?
- A. The average temperature recordings for at least one sensor exceeded 120 on three consecutive executions of the query
- B. The source query failed to update properly for three consecutive minutes and then restarted
- C. The total average temperature across all sensors exceeded 120 on three consecutive executions of the query
- D. The recent_sensor_recordingstable was unresponsive for three consecutive runs of the query
- E. The maximum temperature recording for at least one sensor exceeded 120 on three consecutive executions of the query
Answer: A
Explanation:
This is the correct answer because the query is using a GROUP BY clause on the sensor_id column, which means it will calculate the mean temperature for each sensor separately. The alert will trigger when the mean temperature for any sensor is greater than 120, which means at least one sensor had an average temperature above 120 for three consecutive minutes. The alert will stop when the mean temperature for all sensors drops below 120.
NEW QUESTION # 28
The business intelligence team has a dashboard configured to track various summary metrics for retail stories. This includes total sales for the previous day alongside totals and averages for a variety of time periods. The fields required to populate this dashboard have the following schema:
For Demand forecasting, the Lakehouse contains a validated table of all itemized sales updated incrementally in near real-time. This table named products_per_order, includes the following fields:
Because reporting on long-term sales trends is less volatile, analysts using the new dashboard only require data to be refreshed once daily. Because the dashboard will be queried interactively by many users throughout a normal business day, it should return results quickly and reduce total compute associated with each materialization.
Which solution meets the expectations of the end users while controlling and limiting possible costs?
- A. Use the Delta Cache to persists the products_per_order table in memory to quickly the dashboard with each query.
- B. Configure a webhook to execute an incremental read against products_per_order each time the dashboard is refreshed.
- C. Populate the dashboard by configuring a nightly batch job to save the required to quickly update the dashboard with each query.
- D. Use Structure Streaming to configure a live dashboard against the products_per_order table within a Databricks notebook.
- E. Define a view against the products_per_order table and define the dashboard against this view.
Answer: C
NEW QUESTION # 29
A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non-overlapping five-minute interval. Events are recorded once per minute per device.
Streaming DataFrame df has the following schema:
"device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT"
Code block:
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from
Choose the response that correctly fills in the blank within the code block to complete this task.
- A. window("event_time", "5 minutes").alias("time")
- B. to_interval("event_time", "5 minutes").alias("time")
- C. "event_time"
- D. window("event_time", "10 minutes").alias("time")
- E. lag("event_time", "10 minutes").alias("time")
Answer: A
Explanation:
This is the correct answer because the window function is used to group streaming data by time intervals. The window function takes two arguments: a time column and a window duration. The window duration specifies how long each window is, and must be a multiple of 1 second. In this case, the window duration is "5 minutes", which means each window will cover a non-overlapping five- minute interval. The window function also returns a struct column with two fields: start and end, which represent the start and end time of each window. The alias function is used to rename the struct column as "time".
NEW QUESTION # 30
A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non-overlapping five-minute interval. Incremental state information should be maintained for 10 minutes for late-arriving data.
Streaming DataFrame df has the following schema:
"device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT"
Code block:
Choose the response that correctly fills in the blank within the code block to complete this task.
- A. withWatermark("event_time", "10 minutes")
- B. awaitArrival("event_time", "10 minutes")
- C. delayWrite("event_time", "10 minutes")
- D. slidingWindow("event_time", "10 minutes")
- E. await("event_time + `10 minutes'")
Answer: A
Explanation:
This is because the question asks for incremental state information to be maintained for 10 minutes for late-arriving data. The withWatermark method is used to define the watermark for late data. The watermark is a timestamp column and a threshold that tells the system how long to wait for late data. In this case, the watermark is set to 10 minutes. The other options are incorrect because they are not valid methods or syntax for watermarking in Structured Streaming.
NEW QUESTION # 31
A Delta table of weather records is partitioned by date and has the below schema:
date DATE, device_id INT, temp FLOAT, latitude FLOAT, longitude FLOAT
To find all the records from within the Arctic Circle, you execute a query with the below filter:
latitude > 66.3
Which statement describes how the Delta engine identifies which files to load?
- A. The Parquet file footers are scanned for min and max statistics for the latitude column
- B. The Hive metastore is scanned for min and max statistics for the latitude column
- C. All records are cached to attached storage and then the filter is applied Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from
- D. The Delta log is scanned for min and max statistics for the latitude column
- E. All records are cached to an operational database and then the filter is applied
Answer: D
Explanation:
This is the correct answer because Delta Lake uses a transaction log to store metadata about each table, including min and max statistics for each column in each data file. The Delta engine can use this information to quickly identify which files to load based on a filter condition, without scanning the entire table or the file footers. This is called data skipping and it can improve query performance significantly. Verified Reference: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; [Databricks Documentation], under "Optimizations - Data Skipping" section.
In the Transaction log, Delta Lake captures statistics for each data file of the table. These statistics indicate per file:
- Total number of records
- Minimum value in each column of the first 32 columns of the table
- Maximum value in each column of the first 32 columns of the table
- Null value counts for in each column of the first 32 columns of the table When a query with a selective filter is executed against the table, the query optimizer uses these statistics to generate the query result. it leverages them to identify data files that may contain records matching the conditional filter.
For the SELECT query in the question, The transaction log is scanned for min and max statistics for the price column.
NEW QUESTION # 32
The data governance team is reviewing user for deleting records for compliance with GDPR. The following logic has been implemented to propagate deleted requests from the user_lookup table to the user aggregate table.
Assuming that user_id is a unique identifying key and that all users have requested deletion have been removed from the user_lookup table, which statement describes whether successfully executing the above logic guarantees that the records to be deleted from the user_aggregates table are no longer accessible and why?
- A. No; the Delta Lake DELETE command only provides ACID guarantees when combined with the MERGE INTO command
- B. Yes; Delta Lake ACID guarantees provide assurance that the DELETE command successed fully and permanently purged these records.
- C. No; files containing deleted records may still be accessible with time travel until a BACUM command is used to remove invalidated data files.
- D. No; the change data feed only tracks inserts and updates not deleted records.
- E. Yes; the change data feed uses foreign keys to ensure delete consistency throughout the Lakehouse.
Answer: C
Explanation:
Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from Explanation:
The DELETE operation in Delta Lake is ACID compliant, which means that once the operation is successful, the records are logically removed from the table. However, the underlying files that contained these records may still exist and be accessible via time travel to older versions of the table. To ensure that these records are physically removed and compliance with GDPR is maintained, a VACUUM command should be used to clean up these data files after a certain retention period. The VACUUM command will remove the files from the storage layer, and after this, the records will no longer be accessible.
NEW QUESTION # 33
......
Ace Databricks-Certified-Data-Engineer-Professional Certification with 127 Actual Questions: https://www.braindumpquiz.com/Databricks-Certified-Data-Engineer-Professional-exam-material.html