Databricks Certified Data Engineer Professional202 questions across 21 pages are complete and free, with no account or card.
202 questions in this bank
Free access · progress saved on this device
Which statement describes the default execution mode for Databricks Auto Loader?
Answer: D
"Auto Loader uses directory listing mode by default. In directory listing mode, Auto Loader identifies new files by listing the input directory." https://learn.microsoft.com/en-us/azure/databricks/ingestion/auto-loader/directory-listing-mode
Source: Adapted from community discussion — vctrhugo
A Delta Lake table representing metadata about content posts from users has the following schema: user_id LONG, post_text STRING, post_id STRING, longitude FLOAT, latitude FLOAT, post_time TIMESTAMP, date DATE Based on the above schema, which column is a good candidate for partitioning the Delta Table?
Answer: E
Partitioning a Delta Lake table on the date column is a common practice. This is because partitioning by date can significantly improve query performance when dealing with time-series data. It allows for efficient filtering of data based on time periods, which is a common requirement in many analytics workloads. Partitioning by date also helps manage the size of your partitions, as each partition will contain only the data for a specific date. This can lead to more efficient reads and writes, and can also make it easier to manage and maintain your data.
Source: Adapted from community discussion — vctrhugo
The data engineering team has configured a job to process customer requests to be forgotten (have their data deleted). All user data that needs to be deleted is stored in Delta Lake tables using default table settings. The team has decided to process all deletions from the previous week as a batch job at 1am each Sunday. The total duration of this job is less than one hour. Every Monday at 3am, a batch job executes a series of VACUUM commands on all Delta Lake tables throughout the organization. The compliance officer has recently learned about Delta Lake's time travel functionality. They are concerned that this might allow continued access to deleted data. Assuming all delete logic is correctly implemented, which statement correctly addresses this concern?
Answer: E
Delta Lake's time travel feature allows you to query an older snapshot of a table. By default, Delta Lake retains a 7-day history for the table to support operations like time travel. When data is deleted from a Delta table, the actual data files are not immediately removed from the storage layer; they are just marked for deletion. The VACUUM command is used to clean up these files that are no longer in the state of the table, but it will not remove any files that fall within the retention period unless it is run with an override option to reduce the retention period. Thus, if the deletions are processed on Sunday and the VACUUM command is run on Monday without overriding the default retention period, the deleted records would still be accessible via time travel for approximately 8 days (until the next run of the VACUUM command after the data has aged past the 7-day retention period).
Source: Adapted from community discussion — aragorn_brego
A large company seeks to implement a near real-time solution involving hundreds of pipelines with parallel updates of many tables with extremely high volume and high velocity data. Which of the following solutions would you implement to achieve this requirement?
Answer: A
High Concurrency clusters in Databricks are designed for multiple concurrent users and workloads. They provide fine-grained sharing of cluster resources and are optimized for operations such as running multiple parallel queries and updates. This would be suitable for a solution that involves many pipelines with parallel updates, especially with high volume and high velocity data.
Source: Adapted from community discussion — aragorn_brego
Which describes a method of installing a Python package scoped at the notebook level to all nodes in the currently active cluster?
Answer: C
In Databricks notebooks, you can use the %pip install command in a notebook cell to install a Python package. This will install the package on all nodes in the currently active cluster at the notebook level. It is a feature provided by Databricks to facilitate the installation of Python libraries for the notebook environment specifically.
Source: Adapted from community discussion — aragorn_brego
Each configuration below is identical to the extent that each cluster has 400 GB total of RAM 160 total cores and only one Executor per VM. Given an extremely long-running job for which completion must be guaranteed, which cluster configuration will be able to guarantee completion of the job in light of one or more VM failures?
Answer: B
If VM is down, performance is degraded, so opting for vm's which has distributed memory per executor and optimal cores per executor.
Source: Adapted from community discussion — ChayV
A Delta Lake table in the Lakehouse named customer_churn_params is used in churn prediction by the machine learning team. The table contains information about customers derived from a number of upstream sources. Currently, the data engineering team populates this table nightly by overwriting the table with the current valid values derived from upstream data sources. Immediately after each update succeeds, the data engineering team would like to determine the difference between the new version and the previous version of the table. Given the current implementation, which method can be used?
Answer: A
A data team’s Structured Streaming job is configured to calculate running aggregates for item sales to update a downstream marketing dashboard. The marketing team has introduced a new promotion, and they would like to add a new field to track the number of times this promotion code is used for each item. A junior data engineer suggests updating the existing query as follows. Note that proposed changes are in bold. Original query:
Proposed query:
Which step must also be completed to put the proposed query into production?
Answer: A
This checkpoint location preserves all of the essential information that identifies a query. Each query must have a different checkpoint location. Multiple queries should never have the same location. For more information, see the Structured Streaming Programming Guide. https://docs.databricks.com/en/structured-streaming/query-recovery.html
Source: Adapted from community discussion — MDWPartners
When using CLI or REST API to get results from jobs with multiple tasks, which statement correctly describes the response structure?
Answer: E
The correct answer is E. https://docs.databricks.com/api/workspace/jobs/getrun You can visit the link and observe the response sample of the API { "job_id": 11223344, "run_id": 455644833, job_id is the unique id of the job. run_id is the unique id of the run Then each task will have its unique run id: "tasks": [ { ... "run_id": 2112892, ...
Source: Adapted from community discussion — cales
The data engineering team is configuring environments for development, testing, and production before beginning migration on a new data pipeline. The team requires extensive testing on both the code and data resulting from code execution, and the team wants to develop and test against data as similar to production data as possible. A junior data engineer suggests that production data can be mounted to the development and testing environments, allowing pre-production code to execute against production data. Because all users have admin privileges in the development environment, the junior data engineer has offered to configure permissions and mount this data for the team. Which statement captures best practices for this situation?
Answer: B
Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page
DE-PRO practice, question 11. 202 questions available.
Version 1.0.0. How we source questions