Databricks Certified Data Engineer Professional202 questions across 21 pages are complete and free, with no account or card.
202 questions in this bank
Free access · progress saved on this device
Which REST API call can be used to review the notebooks configured to run as tasks in a multi-task job?
Answer: D
The answer is right, the explanation wrong. Both single & multi task jobs definitions are returned by /jobs/get. The /jobs/runs/get gets the executed/run jobs bases on the definitions you can get from /jobs/get.
Source: Adapted from community discussion — arekm
A Data Engineer wants to run unit tests using common Python testing frameworks on Python functions defined across several Databricks notebooks currently used in production. How can the data engineer run unit tests against functions that work with data in production?
Answer: B
Databricks Repos is the recommended way to organize and manage code, including functions and unit tests, in a scalable and maintainable way. By defining your functions and unit tests in Files in Repos, you can: Modularize Your Code: Functions can be organized into separate Python files or modules, making them reusable and easier to test. Use Standard Testing Frameworks: Frameworks like pytest or unittest can be used to write and execute unit tests against these functions. Version Control Integration: Files in Repos can be version-controlled using Git, ensuring traceability and collaboration. Production Data Testing: With proper safeguards, you can design unit tests to test production-like data while maintaining modularity and separation from production pipelines.
Source: Adapted from community discussion — Thameur01
A data engineer wants to refactor the following DLT code, which includes multiple table definitions with very similar code.
In an attempt to programmatically create these tables using a parameterized table definition, the data engineer writes the following code.
The pipeline runs an update with this refactored code, but generates a different DAG showing incorrect configuration values for these tables. How can the data engineer fix this?
Answer: C
Problem seems to be the fact that the new_table function has no parameter and the t variable won't be recognized as a variable.. However i have not tested it as DLT is not available in free membership. :(
Source: Adapted from community discussion — benni_ale
Each configuration below is identical to the extent that each cluster has 400 GB total of RAM, 160 total cores and only one Executor per VM. Given a job with at least one wide transformation, which of the following cluster configurations will result in maximum performance?
Answer: B
Option A, question is about maximum performance. Wide transformation will result in often expensive shuffle. With one executor this problem will be resolved. https://docs.databricks.com/en/clusters/cluster-config-best-practices.html#complex-batch-etl
Source: Adapted from community discussion — robson90
A junior data engineer on your team has implemented the following code block.
The view new_events contains a batch of records with the same schema as the events Delta table. The event_id field serves as a unique key for this table. When this query is executed, what will happen with new records that have the same event_id as an existing record?
Answer: B
The answer is correct. "If none of the WHEN MATCHED conditions evaluate to true for a source and target row pair that matches the merge_condition, then the target row is left unchanged." https://docs.databricks.com/en/sql/language-manual/delta-merge-into.html#:~:text=If%20none%20of%20the%20WHEN%20MATCHED%20conditions%20evaluate%20to%20true%20for%20a%20source%20and%20target%20row%20pair%20that%20matches%20the%20merge_condition%2C%20then%20the%20target%20row%20is%20left%20unchanged.
Source: Adapted from community discussion — rairaix
A junior data engineer seeks to leverage Delta Lake's Change Data Feed functionality to create a Type 1 table representing all of the values that have ever been valid for all rows in a bronze table created with the property delta.enableChangeDataFeed = true. They plan to execute the following code as a daily job:
Which statement describes the execution and results of running the above query multiple times?
Answer: B
it's B: Reading table’s changes, captured by CDF, using spark.read means that you are reading them as a static source. So, each time you run the query, all table’s changes (starting from the specified startingVersion) will be read.
Source: Adapted from community discussion — taif12340
A new data engineer notices that a critical field was omitted from an application that writes its Kafka source to Delta Lake. This happened even though the critical field was in the Kafka source. That field was further missing from data written to dependent, long-term storage. The retention threshold on the Kafka service is seven days. The pipeline has been in production for three months. Which describes how Delta Lake can help to avoid data loss of this nature in the future?
Answer: E
When we design pipeline, we will have to make sure data from source will be present there in the raw layer/bronze layer and the transformation we make should be done in refine and enterprise layer so by this way we can tackle this kind of situation where the necessary column was not replicated in previous runs of pipeline and we can create new column based on raw data we have.
Source: Adapted from community discussion — Anithec0der
The data engineering team maintains the following code:
Assuming that this code produces logically correct results and the data in the source tables has been de-duplicated and validated, which statement describes what will occur when this code is executed?
Answer: B
The data engineering team is migrating an enterprise system with thousands of tables and views into the Lakehouse. They plan to implement the target architecture using a series of bronze, silver, and gold tables. Bronze tables will almost exclusively be used by production data engineering workloads, while silver tables will be used to support both data engineering and machine learning workloads. Gold tables will largely serve business intelligence and reporting purposes. While personal identifying information (PII) exists in all tiers of data, pseudonymization and anonymization rules are in place for all data at the silver and gold levels. The organization is interested in reducing security concerns while maximizing the ability to collaborate across diverse teams. Which statement exemplifies best practices for implementing this system?
Answer: A
A - most logical B - it is a logical construct, but under default settings tables are stored where the database is so there is a security component to it C - never a good idea to store everything in one db since db allows to group tables with similar area of interest and allows to manage permissions (like groups in Entra and assigning permissions to groups) D - not default database does not mean we cannot use managed tables and you can specify your location still; I do not think that storing anything on DBFS is a good idea - even Databricks suggests to use workspaces for your code, not to mention the data. E - thousand databases - nonsense; you can specify the location of individual tables.
Source: Adapted from community discussion — arekm
To reduce storage and compute costs, the data engineering team has been tasked with curating a series of aggregate tables leveraged by business intelligence dashboards, customer-facing applications, production machine learning models, and ad hoc analytical queries. The data engineering team has been made aware of new requirements from a customer-facing application, which is the only downstream workload they manage entirely. As a result, an aggregate table used by numerous teams across the organization will need to have a number of fields renamed, and additional fields will also be added. Which of the solutions addresses the situation while minimally interrupting other teams in the organization without increasing the number of tables that need to be managed?
Answer: B
B makes way more sense, the number of tables managed do not increase since the old table won't be used anymore, then the view on top of this table is not another table to manage, just maintains the "original API" of the table to avoid breaking changes in downstream applications
Source: Adapted from community discussion — guillesd
Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page
DE-PRO practice, question 101. 202 questions available.
Version 1.0.0. How we source questions