Databricks Certified Data Engineer Associate practice questions

1. Question 1 of 25

   A data engineer has left the organization. The data team needs to transfer ownership of the data engineer’s Delta tables to a new data engineer. The new data engineer is the lead engineer on the data team. Assuming the original data engineer no longer has access, which of the following individuals must be the one to transfer ownership of the Delta tables in Data Explorer?

   • ADatabricks account representative
   • BThis transfer is not possible
   • CWorkspace administrator
   • DNew lead data engineer
   • EOriginal data engineer

   Show answer

   Correct answer: C

   How others answered

   • C100%

   Explanation

   The ownership of Delta tables in Data Explorer can be transferred by the current owner, a metastore admin, or the owner of the container1. In this case, since the original data engineer no longer has access, the Workspace Administrator (Option C) would be the most appropriate individual to transfer the ownership of the Delta tables to the new data engineer. This is because the Workspace Administrator typically has the necessary permissions to manage such resources2. Please note that the exact process may vary depending on the specific configurations and permissions set up in your workspace. It’s always a good idea to consult with your organization’s IT or data governance team to ensure the correct procedures are followed.

   Adapted from community discussion by Garyn · 2023-12-29 · 4 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

   Supporting references

   • Databricks documentation

   View community discussion4

   Candidate context

   Question 1 discussion

   These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

   1. G

      Garyn

      2023-12-29

      Selected C4 upvotes

      The ownership of Delta tables in Data Explorer can be transferred by the current owner, a metastore admin, or the owner of the container1. In this case, since the original data engineer no longer has access, the Workspace Administrator (Option C) would be the most appropriate individual to transfer the ownership of the Delta tables to the new data engineer. This is because the Workspace Administrator typically has the necessary permissions to manage such resources2. Please note that the exact process may vary depending on the specific configurations and permissions set up in your workspace. It’s always a good idea to consult with your organization’s IT or data governance team to ensure the correct procedures are followed.

   2. X

      XiltroX

      2023-04-01

      Selected C2 upvotes

      Wrong answer. The correct answer should be C https://www.databricks.com/blog/2022/08/26/databricks-workspace-administration-best-practices-for-account-workspace-and-metastore-admins.html

   3. V

      Varma_Saraswathula

      2023-04-21

      3 upvotes

      Option C - https://docs.databricks.com/sql/admin/transfer-ownership.html

   4. M

      Majjjj

      2023-05-04

      Selected C2 upvotes

      The Workspace administrator must be the one to transfer ownership of the Delta tables in Data Explorer in this scenario.

2. Question 2 of 25

   A data engineer wants to create a new table containing the names of customers that live in France. They have written the following command:

   

   A senior data engineer mentions that it is organization policy to include a table property indicating that the new table includes personally identifiable information (PII). Which of the following lines of code fills in the above blank to successfully complete the task?

   • AThere is no way to indicate whether a table contains PII.
   • B"COMMENT PII"
   • CTBLPROPERTIES PII
   • DCOMMENT "Contains PII"
   • EPII

   Show answer

   Correct answer: D

   How others answered

   • D100%

   Explanation

   The correct answer is D. COMMENT "Contains PII". Context matters. Yes, you can use Table Property to add additional metadata. But you cannot view that property when you describe the table. With the Comment "this is ..." anyone who describe the table will see the comment.

   Adapted from community discussion by Huroye · 2023-11-15 · 10 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

   Supporting references

   • Databricks documentation
   • Microsoft Learn

   View community discussion4

   Candidate context

   Question 2 discussion

   These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

   1. H

      Huroye

      2023-11-15

      10 upvotes

      The correct answer is D. COMMENT "Contains PII". Context matters. Yes, you can use Table Property to add additional metadata. But you cannot view that property when you describe the table. With the Comment "this is ..." anyone who describe the table will see the comment.

   2. G

      Gems1

      2023-07-29

      7 upvotes

      D Ref:https://www.databricks.com/discover/pages/data-quality-management CREATE TABLE my_table (id INT COMMENT 'Unique Identification Number', name STRING COMMENT 'PII', age INT COMMENT 'PII') TBLPROPERTIES ('contains_pii'=True) COMMENT 'Contains PII';

   3. A

      Atnafu

      2023-07-07

      3 upvotes

      D The COMMENT keyword is used to add a comment to a table. The comment can be used to provide additional information about the table, such as its purpose or the data that it contains. In this case, the data engineer wants to add a comment to the customersInFrance table indicating that the table contains PII. The following line of code will do this: Code snippet COMMENT "Contains PII" Use code with caution. Learn more This will add the comment "Contains PII" to the customersInFrance table. The other options are not valid ways to indicate that a table contains PII. The TBLPROPERTIES keyword is used to set the table properties, but there is no property for indicating whether a table contains PII. The PII keyword is not a valid keyword in SQL. Therefore, the only valid way to indicate that a table contains PII is to use the COMMENT keyword.

   4. S

      softthinkers

      2023-05-04

      2 upvotes

      Correct answer should be C asommand creates a new table called "customersInFrance" with the properties of Personally Identifiable Information (PII) and selects the columns ID, FIRSTNAME, LASTNAME, ADDRESS, and PHONE_NUMBER from the existing "customers" table where the country is France.

3. Question 3 of 25

   Identify the impact of ON VIOLATION DROP ROW and ON VIOLATION FAIL UPDATE for a constraint violation. A data engineer has created an ETL pipeline using Delta Live table to manage their company travel reimbursement detail, they want to ensure that the if the location details has not been provided by the employee, the pipeline needs to be terminated. How can the scenario be implemented?

   • ACONSTRAINT valid_location EXPECT (location = NULL)
   • BCONSTRAINT valid_location EXPECT (location != NULL) ON VIOLATION FAIL UPDATE
   • CCONSTRAINT valid_location EXPECT (location != NULL) ON DROP ROW
   • DCONSTRAINT valid_location EXPECT (location != NULL) ON VIOLATION FAIL

   Show answer

   Correct answer: B

   How others answered

   • B71%
   • D29%

   Only 71% agreed on this answer — treat it as contested rather than settled.

   Explanation

   B is correct https://docs.databricks.com/en/delta-live-tables/sql-ref.html ON VIOLATION Optional action to take for failed rows: FAIL UPDATE: Immediately stop pipeline execution. DROP ROW: Drop the record and continue processing.

   Adapted from community discussion by Worldmaster · 2024-12-01 · 2 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

   Supporting references

   • Microsoft Learn
   • Databricks documentation

   View community discussion4

   Candidate context

   Question 3 discussion

   These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

   1. C

      CaoMengde09

      2024-12-30

      1 upvote

      There is no such a thing as ON VIOLATION FAIL. We have only 3 behaviour to trigger when a constraint is being violated, and they fall under those syntaxes : --> ON VIOLATION FAIL UPDATE : The pipeline fails once it detects the first violation --> ON VIOLATION DROP ROW : The pipeline won't fail but the failing rows will be flagged and stored in event log --> If you put nothing after the expectation : The rows that violates the expectation will be loaded to the sink (Since Databricks doesn't enforce classical database constraints) and the rows will be flagged in the event log. B is the best answer, if you're not convinced try to run D in a notebook and you'll get a syntax error

   2. W

      Worldmaster

      2024-12-01

      Selected B2 upvotes

      B is correct https://docs.databricks.com/en/delta-live-tables/sql-ref.html ON VIOLATION Optional action to take for failed rows: FAIL UPDATE: Immediately stop pipeline execution. DROP ROW: Drop the record and continue processing.

   3. K

      knightkkd

      2024-12-04

      Selected B1 upvote

      FAIL UPDATE: Immediately stop pipeline execution. https://learn.microsoft.com/en-us/azure/databricks/delta-live-tables/expectations#fail

   4. S

      san089

      2024-12-18

      Selected B1 upvote

      Correct Answer: B From Databricks doc CONSTRAINT valid_count EXPECT (count > 0) ON VIOLATION FAIL UPDATE

4. Question 4 of 25

   Select 2

   The Delta transaction log for the ‘students’ tables is shown using the ‘DESCRIBE HISTORY students’ command. A Data Engineer needs to query the table as it existed before the UPDATE operation listed in the log. Which command should the Data Engineer use to achieve this? (Choose two.)

   

   • ASELECT * FROM students@v4
   • BSELECT * FROM students TIMESTAMP AS OF ‘2024-04-22T 14:32:47.000+00:00’
   • CSELECT * FROM students FROM HISTORY VERSION AS OF 3
   • DSELECT * FROM students VERSION AS OF 5
   • ESELECT * FROM students TIMESTAMP AS OF ‘2024-04-22T 14:32:58.000+00:00’

   Show answer

   Correct answer: A, B

   How others answered

   • AB83%
   • BD17%

   Explanation

   SELECT * FROM people10m VERSION AS OF 123; Is identical to SELECT * FROM people10m@v123; so . SELECT * FROM students@v4 is the same as running . SELECT * FROM students@v4 VERSION AS OF 5 ["A", "B"]

   Adapted from community discussion by CaoMengde09 · 2024-12-30 · 2 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

   Supporting references

   • Databricks documentation

   View community discussion4

   Candidate context

   Question 4 discussion

   These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

   1. C

      CaoMengde09

      2024-12-30

      Selected A, B2 upvotes

      SELECT * FROM people10m VERSION AS OF 123; Is identical to SELECT * FROM people10m@v123; so . SELECT * FROM students@v4 is the same as running . SELECT * FROM students@v4 VERSION AS OF 5 ["A", "B"]

   2. D

      DipeshGandhi131

      2024-12-21

      Selected A, B2 upvotes

      https://docs.databricks.com/en/delta/history.html#delta-time-travel-syntax

   3. W

      Worldmaster

      2024-12-01

      Selected A, B1 upvote

      AB correct https://docs.databricks.com/en/delta/history.html

   4. B

      Billybob0604

      2025-03-24

      1 upvote

      No , SELECT * FROM students@v4 is the same as running SELECT * FROM students VERSION AS OF 4

5. Question 5 of 25

   Which of the following commands can be used to write data into a Delta table while avoiding the writing of duplicate records?

   • ADROP
   • BIGNORE
   • CMERGE
   • DAPPEND
   • EINSERT

   Show answer

   Correct answer: C

   How others answered

   • C94%

   Explanation

   C. MERGE To write data into a Delta table while avoiding the writing of duplicate records, you can use the MERGE command. The MERGE command in Delta Lake allows you to combine the ability to insert new records and update existing records in a single atomic operation. The MERGE command compares the data being written with the existing data in the Delta table based on specified matching criteria, typically using a primary key or unique identifier. It then performs conditional actions, such as inserting new records or updating existing records, depending on the comparison results. By using the MERGE command, you can handle the prevention of duplicate records in a more controlled and efficient manner. It allows you to synchronize and reconcile data from different sources while avoiding duplication and ensuring data integrity. Therefore, option C, MERGE, is the correct command to use when writing data into a Delta table while avoiding the writing of duplicate records.

   Adapted from community discussion by Atnafu · 2023-07-07 · 2 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

   Supporting references

   • Databricks documentation

   View community discussion4

   Candidate context

   Question 5 discussion

   These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

   1. A

      Atnafu

      2023-07-07

      2 upvotes

      C. MERGE To write data into a Delta table while avoiding the writing of duplicate records, you can use the MERGE command. The MERGE command in Delta Lake allows you to combine the ability to insert new records and update existing records in a single atomic operation. The MERGE command compares the data being written with the existing data in the Delta table based on specified matching criteria, typically using a primary key or unique identifier. It then performs conditional actions, such as inserting new records or updating existing records, depending on the comparison results. By using the MERGE command, you can handle the prevention of duplicate records in a more controlled and efficient manner. It allows you to synchronize and reconcile data from different sources while avoiding duplication and ensuring data integrity. Therefore, option C, MERGE, is the correct command to use when writing data into a Delta table while avoiding the writing of duplicate records.

   2. V

      vctrhugo

      2023-09-03

      Selected C3 upvotes

      C. MERGE The MERGE command is used to write data into a Delta table while avoiding the writing of duplicate records. It allows you to perform an "upsert" operation, which means that it will insert new records and update existing records in the Delta table based on a specified condition. This helps maintain data integrity and avoid duplicates when adding new data to the table.

   3. 8

      80370eb

      2024-08-08

      Selected C3 upvotes

      C. MERGE The MERGE command allows you to perform upserts (update and insert) into a Delta table, effectively avoiding duplicates by updating existing records and inserting new ones as needed.

   4. O

      Oleskie

      2023-04-05

      4 upvotes

      'C' is a correct answer. https://docs.databricks.com/sql/language-manual/delta-merge-into.html

6. Question 6 of 25

   A data engineer runs a statement every day to copy the previous day’s sales into the table transactions. Each day’s sales are in their own file in the location "/transactions/raw". Today, the data engineer runs the following command to complete this task:

   

   After running the command today, the data engineer notices that the number of records in table transactions has not changed. Which of the following describes why the statement might not have copied any new records into the table?

   • AThe format of the files to be copied were not included with the FORMAT_OPTIONS keyword.
   • BThe names of the files to be copied were not included with the FILES keyword.
   • CThe previous day’s file has already been copied into the table.
   • DThe PARQUET file format does not support COPY INTO.
   • EThe COPY INTO statement requires the table to be refreshed to view the copied rows.

   Show answer

   Correct answer: C

   How others answered

   • C79%
   • E18%

   Explanation

   C. The previous day’s file has already been copied into the table. The COPY INTO statement is generally used to copy data from files or a location into a table. If the data engineer runs this statement daily to copy the previous day’s sales into the "transactions" table and the number of records hasn't changed after today's execution, it's possible that the data from today's file might not have differed from the data already present in the table. If the files in the "/transactions/raw" location are expected to contain distinct data for each day and the number of records in the table remains the same, it implies that the data engineer might have already copied today's data previously, or today's data was identical to the data already present in the table. Options A, B, D, and E don't accurately explain why the statement might not have copied new records into the table based on the provided scenario.

   Adapted from community discussion by Garyn · 2023-12-29 · 3 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

   Supporting references

   • Databricks documentation
   • Databricks documentation

   View community discussion4

   Candidate context

   Question 6 discussion

   These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

   1. G

      Garyn

      2023-12-29

      Selected C3 upvotes

      C. The previous day’s file has already been copied into the table. The COPY INTO statement is generally used to copy data from files or a location into a table. If the data engineer runs this statement daily to copy the previous day’s sales into the "transactions" table and the number of records hasn't changed after today's execution, it's possible that the data from today's file might not have differed from the data already present in the table. If the files in the "/transactions/raw" location are expected to contain distinct data for each day and the number of records in the table remains the same, it implies that the data engineer might have already copied today's data previously, or today's data was identical to the data already present in the table. Options A, B, D, and E don't accurately explain why the statement might not have copied new records into the table based on the provided scenario.

   2. K

      knivesz

      2023-04-04

      Selected C2 upvotes

      Respuesta C, por descarte, A) No es necesario B) No se coloca FILES D) PARQUET si es soportado E) No es necesario refrescar la vista, ya que se esta copiando un archivo

   3. A

      AndreFR

      2023-08-18

      Selected C1 upvote

      https://docs.databricks.com/en/ingestion/copy-into/index.html The COPY INTO SQL command lets you load data from a file location into a Delta table. This is a re-triable and idempotent operation; files in the source location that have already been loaded are skipped. if there are no new records, the only consistent choice is C no new files were loaded because already loaded files were skipped.

   4. [

      [Removed]

      2023-05-23

      2 upvotes

      The correct answer is letter C. The use of specific files names with keyword "FILES" is optional as the syntax of COPY INTO declares: [ FILES = ( file_name [, ...] ) | PATTERN = glob_pattern ] When keyword FILES is not used in the statement all files of the directory is used once (because this operation is idempotent).

7. Question 7 of 25

   Which of the following describes a scenario in which a data team will want to utilize cluster pools?

   • AAn automated report needs to be refreshed as quickly as possible.
   • BAn automated report needs to be made reproducible.
   • CAn automated report needs to be tested to identify errors.
   • DAn automated report needs to be version-controlled across multiple collaborators.
   • EAn automated report needs to be runnable by all stakeholders.

   Show answer

   Correct answer: A

   How others answered

   • A98%

   Explanation

   Using cluster pools reduces the cluster startup time. So in this case, the reports can be refreshed quickly and not having to wait long for the cluster to start

   Adapted from community discussion by Data_4ever · 2023-04-07 · 19 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

   Supporting references

   • Databricks documentation

   View community discussion4

   Candidate context

   Question 7 discussion

   These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

   1. D

      Data_4ever

      2023-04-07

      Selected A19 upvotes

      Using cluster pools reduces the cluster startup time. So in this case, the reports can be refreshed quickly and not having to wait long for the cluster to start

   2. 8

      806e7d2

      2024-11-14

      Selected A6 upvotes

      Cluster pools are used in Databricks to reduce the time needed to create and scale clusters by maintaining a set of pre-configured, ready-to-use instances. When an automated report needs to be refreshed quickly, cluster pools help by minimizing cluster startup time, allowing the report generation process to start almost immediately. This is especially beneficial in scenarios where low latency is required to ensure data is updated in near real-time. The other options (B, C, D, and E) do not directly benefit from the use of cluster pools, as they involve aspects like reproducibility, testing, version control, and stakeholder access, which are not specifically addressed by the primary function of cluster pools.

   3. V

      vctrhugo

      2024-09-24

      Selected A3 upvotes

      A. An automated report needs to be refreshed as quickly as possible. Cluster pools are typically used in distributed computing environments, such as cloud-based data platforms like Databricks. They allow you to pre-allocate a set of compute resources (a cluster) for specific tasks or workloads. In this case, if an automated report needs to be refreshed as quickly as possible, you can allocate a cluster pool with sufficient resources to ensure fast data processing and report generation. This helps ensure that the report is generated with minimal latency and can be delivered to stakeholders in a timely manner. Cluster pools allow you to optimize resource allocation for high-demand, time-sensitive tasks like real-time report generation.

   4. M

      Majjjj

      2024-09-24

      Selected A4 upvotes

      Cluster pools in Databricks are used to ensure that a set of pre-warmed clusters is readily available to run workloads. This means that when a job is submitted, it can be executed more quickly because there is no need to wait for a cluster to spin up. Therefore, if a data team needs to refresh an automated report as quickly as possible, they will want to utilize cluster pools to ensure that the job can be executed as quickly as possible.

8. Question 8 of 25

   A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE. The table is configured to run in Production mode using the Continuous Pipeline Mode. Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after clicking Start to update the pipeline?

   • AAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.
   • BAll datasets will be updated once and the pipeline will persist without any processing. The compute resources will persist but go unused.
   • CAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.
   • DAll datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.
   • EAll datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.

   Show answer

   Correct answer: C

   How others answered

   • C100%

   Explanation

   C. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped. Explanation: Continuous Pipeline Mode in Production mode implies that the pipeline continuously processes incoming data updates at set intervals, ensuring the datasets are kept up-to-date as new data arrives. Since the pipeline is set to Continuous Pipeline Mode, it will keep running and updating the datasets until it is manually shut down. The compute resources are allocated dynamically to process and update the datasets as needed, and they will be terminated when the pipeline is stopped or shut down. This mode allows for real-time or near-real-time updates to the datasets from the streaming/live tables, ensuring that the data remains current and reflects the changes occurring in the data sources.

   Adapted from community discussion by Garyn · 2023-12-30 · 10 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

   Supporting references

   • Databricks documentation

   View community discussion4

   Candidate context

   Question 8 discussion

   These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

   1. G

      Garyn

      2023-12-30

      Selected C10 upvotes

      C. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped. Explanation: Continuous Pipeline Mode in Production mode implies that the pipeline continuously processes incoming data updates at set intervals, ensuring the datasets are kept up-to-date as new data arrives. Since the pipeline is set to Continuous Pipeline Mode, it will keep running and updating the datasets until it is manually shut down. The compute resources are allocated dynamically to process and update the datasets as needed, and they will be terminated when the pipeline is stopped or shut down. This mode allows for real-time or near-real-time updates to the datasets from the streaming/live tables, ensuring that the data remains current and reflects the changes occurring in the data sources.

   2. V

      vctrhugo

      2023-09-03

      Selected C4 upvotes

      C. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped. In the scenario described: The Delta Live Table pipeline is configured in Production mode, which means it will continuously process data using the Continuous Pipeline Mode. There are both STREAMING LIVE TABLE datasets and LIVE TABLE datasets defined. When you click Start to update the pipeline in Continuous Pipeline Mode: All datasets, including both STREAMING LIVE TABLE and LIVE TABLE datasets, will be updated at set intervals. Compute resources will be deployed for the update, ensuring that the pipeline processes data. The compute resources will be terminated when the pipeline is stopped or shut down. This setup allows for continuous data processing while efficiently managing compute resources, and the pipeline can be stopped when no longer needed.

   3. A

      Atnafu

      2023-07-08

      2 upvotes

      C. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped. In a Delta Live Table pipeline running in Continuous Pipeline Mode, when you click Start to update the pipeline, the following outcome is expected: All datasets defined using STREAMING LIVE TABLE and LIVE TABLE against Delta Lake table sources will be updated at set intervals. The compute resources will be deployed for the update process and will be active during the execution of the pipeline. The compute resources will be terminated when the pipeline is stopped or shut down. This mode allows for continuous and periodic updates to the datasets as new data arrives or changes in the underlying Delta Lake tables occur. The compute resources are provisioned and utilized during the update intervals to process the data and perform the necessary operations.

   4. S

      say88

      2023-07-31

      4 upvotes

      No answer is correct. Prod Continuous mode processes data at set intervals until pipe is shutdown. However, compute must be always-on and will not terminate. https://docs.databricks.com/delta-live-tables/updates.html#continuous-triggered

9. Question 9 of 25

   In order for Structured Streaming to reliably track the exact progress of the processing so that it can handle any kind of failure by restarting and/or reprocessing, which of the following two approaches is used by Spark to record the offset range of the data being processed in each trigger?

   • ACheckpointing and Write-ahead Logs
   • BStructured Streaming cannot record the offset range of the data being processed in each trigger.
   • CReplayable Sources and Idempotent Sinks
   • DWrite-ahead Logs and Idempotent Sinks
   • ECheckpointing and Idempotent Sinks

   Show answer

   Correct answer: A

   How others answered

   • A76%
   • E24%

   Explanation

   Why the correct answer is E. Checkpointing and Idempotent Sinks: Checkpointing: Spark Structured Streaming uses checkpointing to track the state of the data being processed. Checkpoints allow the system to restart processing from where it left off in case of failure, ensuring reliability. Idempotent Sinks: Idempotent sinks ensure that reprocessing the same data multiple times (in case of a failure or restart) doesn’t lead to duplicate results. The sink can handle repeated writes of the same data without issues. Why A. Checkpointing and Write-ahead Logs is incorrect: Spark Structured Streaming does not use Write-ahead Logs (WAL) for tracking offsets or ensuring fault tolerance. While WALs are used in some systems for durability, Spark Structured Streaming relies on checkpointing and the concept of idempotent operations to ensure consistency and fault tolerance.

   Adapted from community discussion by Colje · 2024-09-27 · 3 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

   Supporting references

   • Microsoft Learn

   View community discussion4

   Candidate context

   Question 9 discussion

   These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

   1. Z

      ZSun

      2023-06-13

      6 upvotes

      spark handle streaming failure through: 1. track the progress/offset(This is option A) 2. fix failure(This is option E) But the question is "two approaches ... record the offset range" Therefore, A

   2. C

      Colje

      2024-09-27

      3 upvotes

      Why the correct answer is E. Checkpointing and Idempotent Sinks: Checkpointing: Spark Structured Streaming uses checkpointing to track the state of the data being processed. Checkpoints allow the system to restart processing from where it left off in case of failure, ensuring reliability. Idempotent Sinks: Idempotent sinks ensure that reprocessing the same data multiple times (in case of a failure or restart) doesn’t lead to duplicate results. The sink can handle repeated writes of the same data without issues. Why A. Checkpointing and Write-ahead Logs is incorrect: Spark Structured Streaming does not use Write-ahead Logs (WAL) for tracking offsets or ensuring fault tolerance. While WALs are used in some systems for durability, Spark Structured Streaming relies on checkpointing and the concept of idempotent operations to ensure consistency and fault tolerance.

   3. C

      chays

      2023-06-01

      Selected A3 upvotes

      Answer is A: The engine uses checkpointing and write-ahead logs to record the offset range of the data being processed in each trigger. The streaming sinks are designed to be idempotent for handling reprocessing. Together, using replayable sources and idempotent sinks, Structured Streaming can ensure end-to-end exactly-once semantics under any failure. https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#:~:text=The%20engine%20uses%20checkpointing%20and,being%20processed%20in%20each%20trigger.

   4. X

      XiltroX

      2023-04-01

      Selected A4 upvotes

      E is a partial answer. The two correct answers are A and E. Structured streaming is important because it uses these two methods to make sure there is fault tolerance and Exactly-once guarantee of data

10. Question 10 of 25

    Which of the following is hosted completely in the control plane of the classic Databricks architecture?

    • AWorker node
    • BJDBC data source
    • CDatabricks web application
    • DDatabricks Filesystem
    • EDriver node

    Show answer

    Correct answer: C

    How others answered

    • C100%

    Explanation

    C. Databricks web application In the classic Databricks architecture, the control plane includes components like the Databricks web application, the Databricks REST API, and the Databricks Workspace. These components are responsible for managing and controlling the Databricks environment, including cluster provisioning, notebook management, access control, and job scheduling. The other options, such as worker nodes, JDBC data sources, Databricks Filesystem (DBFS), and driver nodes, are typically part of the data plane or the execution environment, which is separate from the control plane. Worker nodes are responsible for executing tasks and computations, JDBC data sources are used to connect to external databases, DBFS is a distributed file system for data storage, and driver nodes are responsible for coordinating the execution of Spark jobs.

    Adapted from community discussion by vctrhugo · 2024-09-24 · 15 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation
    • Microsoft Learn

    View community discussion4

    Candidate context

    Question 10 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. V

       vctrhugo

       2024-09-24

       Selected C15 upvotes

       C. Databricks web application In the classic Databricks architecture, the control plane includes components like the Databricks web application, the Databricks REST API, and the Databricks Workspace. These components are responsible for managing and controlling the Databricks environment, including cluster provisioning, notebook management, access control, and job scheduling. The other options, such as worker nodes, JDBC data sources, Databricks Filesystem (DBFS), and driver nodes, are typically part of the data plane or the execution environment, which is separate from the control plane. Worker nodes are responsible for executing tasks and computations, JDBC data sources are used to connect to external databases, DBFS is a distributed file system for data storage, and driver nodes are responsible for coordinating the execution of Spark jobs.

    2. H

       h79

       2023-03-29

       15 upvotes

       I disagree with this answer. I think its the databricks web app that is always in the control plane

    3. 8

       806e7d2

       2024-11-14

       Selected C2 upvotes

       In the classic Databricks architecture, the Databricks web application (which includes the Databricks user interface, job scheduling, and management components) is hosted entirely in the control plane. This control plane is managed by Databricks and contains the user-facing services and APIs that allow users to interact with the Databricks environment. Here’s why the other options don’t fit: A. Worker node and E. Driver node: Both the driver and worker nodes are part of the data plane, where actual data processing occurs. B. JDBC data source: This is external to Databricks, typically hosted wherever the source database resides, and does not exist within the control plane. D. Databricks Filesystem (DBFS): While DBFS metadata might be managed by the control plane, the actual data is stored in the data plane (often within the customer’s cloud account, such as in AWS S3 or Azure Data Lake Storage).

    4. M

       Majjjj

       2024-09-24

       Selected C3 upvotes

       The control plane in the classic Databricks architecture is responsible for managing the Databricks workspace, user and group management, and cluster management, among other things. The Databricks web application is a part of the control plane that enables users to interact with the workspace, create and manage clusters, and work with notebooks, jobs, and data. Worker nodes and driver nodes are part of the data plane, which is responsible for executing data processing tasks. JDBC data sources and the Databricks Filesystem are services that are used by both the control plane and the data plane.

11. Question 11 of 25

    Which of the following tools is used by Auto Loader process data incrementally?

    • ACheckpointing
    • BSpark Structured Streaming
    • CData Explorer
    • DUnity Catalog
    • EDatabricks SQL

    Show answer

    Correct answer: B

    How others answered

    • B88%

    Explanation

    Databricks uses Apache Spark Structured Streaming to back numerous product associated with ingestion workloads, including: - Auto Loader - COPY INTO - Delta Live Tables pipelines - Materialized views and streaming tables in Databricks SQL Source: https://docs.databricks.com/en/ingestion/streaming.html

    Adapted from community discussion by res3 · 2024-12-12 · 3 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation
    • Databricks documentation

    View community discussion4

    Candidate context

    Question 11 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. X

       XiltroX

       2023-04-01

       Selected B10 upvotes

       B is the correct answer. Checkpointing is a method that is part of structured streaming.

    2. A

       Atnafu

       2023-07-08

       3 upvotes

       B Auto Loader uses Spark Structured Streaming to process data incrementally. Spark Structured Streaming is a streaming engine that can be used to process data as it arrives. This makes it ideal for processing data that is being generated in real time. Option A: Checkpointing is a technique used to ensure that data is not lost in case of a failure. It is not used to process data incrementally. Option C: Data Explorer is a data exploration tool that can be used to explore data. It is not used to process data incrementally. Option D: Unity Catalog is a metadata management tool that can be used to store and manage metadata about data assets. It is not used to process data incrementally. Option E: Databricks SQL is a SQL engine that can be used to query data. It is not used to process data incrementally.

    3. V

       vctrhugo

       2023-09-03

       Selected B2 upvotes

       B. Spark Structured Streaming The Auto Loader process in Databricks is typically used in conjunction with Spark Structured Streaming to process data incrementally. Spark Structured Streaming is a real-time data processing framework that allows you to process data streams incrementally as new data arrives. The Auto Loader is a feature in Databricks that works with Structured Streaming to automatically detect and process new data files as they are added to a specified data source location. It allows for incremental data processing without the need for manual intervention.

    4. R

       res3

       2024-12-12

       Selected B3 upvotes

       Databricks uses Apache Spark Structured Streaming to back numerous product associated with ingestion workloads, including: - Auto Loader - COPY INTO - Delta Live Tables pipelines - Materialized views and streaming tables in Databricks SQL Source: https://docs.databricks.com/en/ingestion/streaming.html

12. Question 12 of 25

    A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table. The cade block used by the data engineer is below:

    

    If the data engineer only wants the query to execute a micro-batch to process data every 5 seconds, which of the following lines of code should the data engineer use to fill in the blank?

    • Atrigger("5 seconds")
    • Btrigger()
    • Ctrigger(once="5 seconds")
    • Dtrigger(processingTime="5 seconds")
    • Etrigger(continuous="5 seconds")

    Show answer

    Correct answer: D

    How others answered

    • D100%

    Explanation

    The correct line of code to fill in the blank to execute a micro-batch to process data every 5 seconds is: D. trigger(processingTime="5 seconds") Option A ("trigger("5 seconds")") would not work because it does not specify that the trigger should be a processing time trigger, which is necessary to trigger a micro-batch processing at regular intervals. Option B ("trigger()") would not work because it would use the default trigger, which is not a processing time trigger. Option C ("trigger(once="5 seconds")") would not work because it would only trigger the query once, not at regular intervals. Option E ("trigger(continuous="5 seconds")") would not work because it would trigger the query to run continuously, without any pauses in between, which is not what the data engineer wants.

    Adapted from community discussion by 4be8126 · 2023-04-05 · 5 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Microsoft Learn

    View community discussion4

    Candidate context

    Question 12 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. 4

       4be8126

       2023-04-05

       Selected D5 upvotes

       The correct line of code to fill in the blank to execute a micro-batch to process data every 5 seconds is: D. trigger(processingTime="5 seconds") Option A ("trigger("5 seconds")") would not work because it does not specify that the trigger should be a processing time trigger, which is necessary to trigger a micro-batch processing at regular intervals. Option B ("trigger()") would not work because it would use the default trigger, which is not a processing time trigger. Option C ("trigger(once="5 seconds")") would not work because it would only trigger the query once, not at regular intervals. Option E ("trigger(continuous="5 seconds")") would not work because it would trigger the query to run continuously, without any pauses in between, which is not what the data engineer wants.

    2. V

       vctrhugo

       2023-09-03

       Selected D2 upvotes

       # ProcessingTime trigger with two-seconds micro-batch interval df.writeStream \ .format("console") \ .trigger(processingTime='2 seconds') \ .start() https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#triggers

    3. A

       AndreFR

       2023-08-19

       Selected D1 upvote

       https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#triggers

    4. A

       Atnafu

       2023-07-08

       1 upvote

       D val query = sourceTable .writeStream .format("delta") .outputMode("append") .trigger(Trigger.ProcessingTime("5 seconds")) .start(destinationTable)

13. Question 13 of 25

    An engineering manager wants to monitor the performance of a recent project using a Databricks SQL query. For the first week following the project’s release, the manager wants the query results to be updated every minute. However, the manager is concerned that the compute resources used for the query will be left running and cost the organization a lot of money beyond the first week of the project’s release. Which of the following approaches can the engineering team use to ensure the query does not cost the organization any money beyond the first week of the project’s release?

    • AThey can set a limit to the number of DBUs that are consumed by the SQL Endpoint.
    • BThey can set the query’s refresh schedule to end after a certain number of refreshes.
    • CThey cannot ensure the query does not cost the organization money beyond the first week of the project’s release.
    • DThey can set a limit to the number of individuals that are able to manage the query’s refresh schedule.
    • EThey can set the query’s refresh schedule to end on a certain date in the query scheduler.

    Show answer

    Correct answer: E

    How others answered

    • E77%
    • C20%

    Explanation

    The query scheduler only gives the option on what the interval is to run the query. It does not provide a way to stop after x iterations or at a point in time. The question is confusing. From what i found the only option is to limit users access to the query (and therefore query scheduler). https://docs.databricks.com/security/auth-authz/access-control/query-acl.html Not convinced how this would be helping the organization save money if no-one is manually stopping the schedule. Answer C seems most correct Answer D can be achieved using acl however how is this helpful in the use case described?

    Adapted from community discussion by BigDaddyAus · 2023-04-25 · 10 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation
    • Databricks documentation
    • Microsoft Learn
    • Databricks documentation

    View community discussion4

    Candidate context

    Question 13 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. N

       Nika12

       2024-01-27

       Selected E16 upvotes

       Just got 100% on the test. E was correct. C was not in the available options.

    2. B

       BigDaddyAus

       2023-04-25

       10 upvotes

       The query scheduler only gives the option on what the interval is to run the query. It does not provide a way to stop after x iterations or at a point in time. The question is confusing. From what i found the only option is to limit users access to the query (and therefore query scheduler). https://docs.databricks.com/security/auth-authz/access-control/query-acl.html Not convinced how this would be helping the organization save money if no-one is manually stopping the schedule. Answer C seems most correct Answer D can be achieved using acl however how is this helpful in the use case described?

    3. 4

       4be8126

       2023-04-05

       Selected E4 upvotes

       The correct answer is E. They can set the query's refresh schedule to end on a certain date in the query scheduler. Databricks SQL supports a query scheduler that enables users to schedule SQL queries to run at defined intervals. By default, scheduled queries run indefinitely. However, users can configure the scheduler to stop running queries at a specific time or after a specific number of runs. In this scenario, the engineering team can set the query's refresh schedule to end on a certain date, ensuring that the query does not run beyond the first week of the project's release and potentially cost the organization more money.

    4. G

       Garyn

       2023-12-30

       Selected E3 upvotes

       E. They can set the query’s refresh schedule to end on a certain date in the query scheduler. Explanation: Query Scheduler: Databricks offers a Query Scheduler that allows users to schedule the execution of SQL queries at specific intervals or for specific durations. Setting a Specific End Date: The team can configure the query's refresh schedule to conclude or end on a certain date. By specifying an end date within the first week of the project's release, the query will automatically stop refreshing after that date. This action ensures that compute resources aren't continuously utilized beyond the specified timeframe, preventing unnecessary costs. This approach allows the team to control and limit the execution of the query to the required duration without incurring additional costs beyond the first week of the project's release.

14. Question 14 of 25

    A data analysis team has noticed that their Databricks SQL queries are running too slowly when connected to their always-on SQL endpoint. They claim that this issue is present when many members of the team are running small queries simultaneously. They ask the data engineering team for help. The data engineering team notices that each of the team’s queries uses the same SQL endpoint. Which of the following approaches can the data engineering team use to improve the latency of the team’s queries?

    • AThey can increase the cluster size of the SQL endpoint.
    • BThey can increase the maximum bound of the SQL endpoint’s scaling range.
    • CThey can turn on the Auto Stop feature for the SQL endpoint.
    • DThey can turn on the Serverless feature for the SQL endpoint.
    • EThey can turn on the Serverless feature for the SQL endpoint and change the Spot Instance Policy to “Reliability Optimized.”

    Show answer

    Correct answer: B

    How others answered

    • B57%
    • A36%

    Only 57% agreed on this answer — treat it as contested rather than settled.

    Explanation

    Answer B is correct For those who's selected the same answer as the question 40 in the Databricks exam training, be careful becaue it's quite different: - Here the question is about simultaneously runs -> Scale Out clusters (involves adding more clusters) - In the Databricks exam training, the question is about "sequentially run queries" -> Scale Up (increasing the size of the nodes) Pleas refer to the this accepted answer https://community.databricks.com/t5/data-engineering/sequential-vs-concurrency-optimization-questions-from-query/td-p/36696

    Adapted from community discussion by mokrani · 2023-11-07 · 16 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation

    View community discussion4

    Candidate context

    Question 14 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. D

       damaldon

       2023-09-06

       31 upvotes

       Answer is B. According to databricks documentation: -Sequentially -> Increase cluster size -Concurrent --> Scale out cluster

    2. M

       mokrani

       2023-11-07

       16 upvotes

       Answer B is correct For those who's selected the same answer as the question 40 in the Databricks exam training, be careful becaue it's quite different: - Here the question is about simultaneously runs -> Scale Out clusters (involves adding more clusters) - In the Databricks exam training, the question is about "sequentially run queries" -> Scale Up (increasing the size of the nodes) Pleas refer to the this accepted answer https://community.databricks.com/t5/data-engineering/sequential-vs-concurrency-optimization-questions-from-query/td-p/36696

    3. M

       Majjjj

       2023-05-04

       Selected B4 upvotes

       Option B is the correct answer. The engineering team can set the query’s refresh schedule to end after a certain number of refreshes to ensure that it does not run and cost the organization any money beyond the first week of the project’s release. By setting a limit on the number of refreshes, the query will stop running automatically once the limit is reached. This approach allows the team to monitor the performance of the project for the first week with frequent updates, but also ensures that the query does not consume resources unnecessarily after that period. Options A, C, D, and E are incorrect as they do not provide a solution to the problem of controlling the query's runtime cost.

    4. T

       t30730

       2023-04-03

       5 upvotes

       "The question is looking to test your ability to know how to scale a SQL Endpoint(SQL Warehouse) and you have to look for cue words or need to understand if the queries are running sequentially or concurrently. if the queries are running sequentially then scale up(Size of the cluster from 2X-Small to 4X-Large) if the queries are running concurrently or with more users then scale out(add more clusters)."

15. Question 15 of 25

    Which of the following benefits of using the Databricks Lakehouse Platform is provided by Delta Lake?

    • AThe ability to manipulate the same data using a variety of languages
    • BThe ability to collaborate in real time on a single notebook
    • CThe ability to set up alerts for query failures
    • DThe ability to support batch and streaming workloads
    • EThe ability to distribute complex data operations

    Show answer

    Correct answer: D

    How others answered

    • D100%

    Explanation

    D. The ability to support batch and streaming workloads Delta Lake is a key component of the Databricks Lakehouse Platform that provides several benefits, and one of the most significant benefits is its ability to support both batch and streaming workloads seamlessly. Delta Lake allows you to process and analyze data in real-time (streaming) as well as in batch, making it a versatile choice for various data processing needs. While the other options may be benefits or capabilities of Databricks or the Lakehouse Platform in general, they are not specifically associated with Delta Lake.

    Adapted from community discussion by vctrhugo · 2023-09-03 · 13 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Microsoft Learn

    View community discussion4

    Candidate context

    Question 15 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. V

       vctrhugo

       2023-09-03

       Selected D13 upvotes

       D. The ability to support batch and streaming workloads Delta Lake is a key component of the Databricks Lakehouse Platform that provides several benefits, and one of the most significant benefits is its ability to support both batch and streaming workloads seamlessly. Delta Lake allows you to process and analyze data in real-time (streaming) as well as in batch, making it a versatile choice for various data processing needs. While the other options may be benefits or capabilities of Databricks or the Lakehouse Platform in general, they are not specifically associated with Delta Lake.

    2. X

       XiltroX

       2023-04-01

       3 upvotes

       D is the right answer https://learn.microsoft.com/en-us/azure/databricks/delta/

    3. B

       Basha1996

       2025-02-28

       Selected D1 upvote

       D. The ability to support batch and streaming workloads Adding features such as ACID Properties are allowed which eliminate the drawbacks on DW and Data lake.

    4. T

       Tedet

       2025-01-10

       Selected D1 upvote

       The ability to support batch and streaming workloads - Key feature of lakehouse

16. Question 16 of 25

    A data engineer wants to schedule their Databricks SQL dashboard to refresh once per day, but they only want the associated SQL endpoint to be running when it is necessary. Which of the following approaches can the data engineer use to minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard?

    • AThey can ensure the dashboard’s SQL endpoint matches each of the queries’ SQL endpoints.
    • BThey can set up the dashboard’s SQL endpoint to be serverless.
    • CThey can turn on the Auto Stop feature for the SQL endpoint.
    • DThey can reduce the cluster size of the SQL endpoint.
    • EThey can ensure the dashboard’s SQL endpoint is not one of the included query’s SQL endpoint.

    Show answer

    Correct answer: C

    How others answered

    • C100%

    Explanation

    The data engineer can use the Auto Stop feature to minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard. The Auto Stop feature allows the SQL endpoint to automatically shut down when there are no active connections, which will minimize the total running time of the SQL endpoint. By scheduling the dashboard to refresh once per day, the SQL endpoint will only be running for a short period of time each day, which will minimize the total running time and reduce costs.

    Adapted from community discussion by 4be8126 · 2023-04-05 · 11 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation

    View community discussion4

    Candidate context

    Question 16 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. 4

       4be8126

       2023-04-05

       Selected C11 upvotes

       The data engineer can use the Auto Stop feature to minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard. The Auto Stop feature allows the SQL endpoint to automatically shut down when there are no active connections, which will minimize the total running time of the SQL endpoint. By scheduling the dashboard to refresh once per day, the SQL endpoint will only be running for a short period of time each day, which will minimize the total running time and reduce costs.

    2. V

       vctrhugo

       2023-09-03

       Selected C2 upvotes

       C. They can turn on the Auto Stop feature for the SQL endpoint. To minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard while ensuring that it only runs when necessary, the data engineer can turn on the Auto Stop feature for the SQL endpoint. This feature will automatically stop the SQL endpoint when it is idle for a specified period, reducing costs by avoiding unnecessary running time. Option C allows you to efficiently manage the SQL endpoint's lifecycle, ensuring it's active only when needed, which aligns with the goal of minimizing running time and associated costs. Option B (setting the dashboard's SQL endpoint to be serverless) can also be a valid approach, as it allows the SQL endpoint to be provisioned on-demand and incurs costs only when queries are executed. However, it depends on the specific requirements of your dashboard and queries. Options A, D, and E do not directly address the goal of minimizing the SQL endpoint's running time while ensuring it runs when necessary.

    3. M

       mokrani

       2023-11-30

       5 upvotes

       Why it can't be B ? . They can set up the dashboard’s SQL endpoint to be serverless. ? they can use a serverless endpoint and it will only be active when required.

    4. 8

       80370eb

       2024-08-09

       Selected C2 upvotes

       The Auto Stop feature ensures that the SQL endpoint will automatically shut down when not in use, which helps in reducing unnecessary running time and associated costs. The endpoint will only be running when it's needed for refreshing the dashboard.

17. Question 17 of 25

    A data engineer has been using a Databricks SQL dashboard to monitor the cleanliness of the input data to an ELT job. The ELT job has its Databricks SQL query that returns the number of input records containing unexpected NULL values. The data engineer wants their entire team to be notified via a messaging webhook whenever this value reaches 100. Which of the following approaches can the data engineer use to notify their entire team via a messaging webhook whenever the number of NULL values reaches 100?

    • AThey can set up an Alert with a custom template.
    • BThey can set up an Alert with a new email alert destination.
    • CThey can set up an Alert with a new webhook alert destination.
    • DThey can set up an Alert with one-time notifications.
    • EThey can set up an Alert without notifications.

    Show answer

    Correct answer: C

    How others answered

    • C100%

    Explanation

    https://docs.databricks.com/en/lakehouse-monitoring/monitor-alerts.html Monitor alerts are created and used the same way as other Databricks SQL alerts. You create a Databricks SQL query on the monitor profile metrics table or drift metrics table. You then create a Databricks SQL alert for this query. You can configure the alert to evaluate the query at a desired frequency, and send a notification if the alert is triggered. By default, email notification is sent. You can also set up a webhook or send notifications to other applications such as Slack or Pagerduty.

    Adapted from community discussion by AndreFR · 2024-06-20 · 4 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation

    View community discussion4

    Candidate context

    Question 17 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. 4

       4be8126

       2023-10-05

       Selected C4 upvotes

       The approach the data engineer can use to notify their entire team via a messaging webhook whenever the number of NULL values reaches 100 is: C. They can set up an Alert with a new webhook alert destination. Explanation: To achieve this, the data engineer can set up an Alert in the Databricks workspace that triggers when the query results exceed the threshold of 100 NULL values. They can create a new webhook alert destination in the Alert's configuration settings and provide the necessary messaging webhook URL to receive notifications. When the Alert is triggered, it will send a message to the configured webhook URL, which will then notify the entire team of the issue.

    2. A

       AndreFR

       2024-06-20

       Selected C4 upvotes

       https://docs.databricks.com/en/lakehouse-monitoring/monitor-alerts.html Monitor alerts are created and used the same way as other Databricks SQL alerts. You create a Databricks SQL query on the monitor profile metrics table or drift metrics table. You then create a Databricks SQL alert for this query. You can configure the alert to evaluate the query at a desired frequency, and send a notification if the alert is triggered. By default, email notification is sent. You can also set up a webhook or send notifications to other applications such as Slack or Pagerduty.

    3. A

       Atnafu

       2024-01-08

       3 upvotes

       C Alerts allow you to be notified when something goes wrong in your Databricks environment. You can set up alerts to be notified by email, webhook, or Slack. Webhooks are a way to send data from one application to another. You can use a webhook to send data from Databricks to a messaging service, such as Slack or PagerDuty. One-time notifications allow you to be notified only once when an alert is triggered. This is useful if you only want to be notified about a specific event. Custom templates allow you to customize the email or webhook notification that is sent when an alert is triggered. This is useful if you want to include additional information in the notification, such as the name of the alert or the value of the metric that triggered the alert.

    4. V

       vctrhugo

       2024-03-04

       Selected C2 upvotes

       C. They can set up an Alert with a new webhook alert destination. To notify their entire team via a messaging webhook whenever the number of NULL values reaches 100, the data engineer can set up an Alert in Databricks with a new webhook alert destination. This allows them to configure the alert to trigger when the specified condition (reaching 100 NULL values) is met, and the notification can be sent to the team's messaging webhook. Option C provides the specific approach to achieve the desired outcome of notifying the team via a messaging webhook when the condition is met.

18. Question 18 of 25

    A single Job runs two notebooks as two separate tasks. A data engineer has noticed that one of the notebooks is running slowly in the Job’s current run. The data engineer asks a tech lead for help in identifying why this might be the case. Which of the following approaches can the tech lead use to identify why the notebook is running slowly as part of the Job?

    • AThey can navigate to the Runs tab in the Jobs UI to immediately review the processing notebook.
    • BThey can navigate to the Tasks tab in the Jobs UI and click on the active run to review the processing notebook.
    • CThey can navigate to the Runs tab in the Jobs UI and click on the active run to review the processing notebook.
    • DThere is no way to determine why a Job task is running slowly.
    • EThey can navigate to the Tasks tab in the Jobs UI to immediately review the processing notebook.

    Show answer

    Correct answer: C

    How others answered

    • C88%

    Explanation

    The job run details page contains job output and links to logs, including information about the success or failure of each task in the job run. You can access job run details from the Runs tab for the job. To view job run details from the Runs tab, click the link for the run in the Start time column in the runs list view. To return to the Runs tab for the job, click the Job ID value. If the job contains multiple tasks, click a task to view task run details, including: the cluster that ran the task the Spark UI for the task logs for the task metrics for the task https://docs.databricks.com/en/workflows/jobs/monitor-job-runs.html#job-run-details

    Adapted from community discussion by AndreFR · 2023-08-20 · 4 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation
    • Databricks documentation

    View community discussion4

    Candidate context

    Question 18 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. A

       AndreFR

       2023-08-20

       Selected C4 upvotes

       The job run details page contains job output and links to logs, including information about the success or failure of each task in the job run. You can access job run details from the Runs tab for the job. To view job run details from the Runs tab, click the link for the run in the Start time column in the runs list view. To return to the Runs tab for the job, click the Job ID value. If the job contains multiple tasks, click a task to view task run details, including: the cluster that ran the task the Spark UI for the task logs for the task metrics for the task https://docs.databricks.com/en/workflows/jobs/monitor-job-runs.html#job-run-details

    2. G

       Garyn

       2023-12-30

       Selected C2 upvotes

       The tech lead can navigate to the Runs tab in the Jobs UI and click on the active run to review the processing notebook (Option C). This will allow them to inspect the details of the job run, including the duration of each task, which can help identify potential performance issues. There could be several reasons why a notebook is running slowly as part of a job. For instance, there might be a delay when the job cluster has to be spun up, or the table gets delta cached in memory and copies of files will be stored on local node’s storage. Even certain operations like pandas UDFs can be slow. Please note that the exact process may vary depending on the specific configurations and permissions set up in your workspace. It’s always a good idea to consult with your organization’s IT or data governance team to ensure the correct procedures are followed.

    3. 8

       806e7d2

       2024-11-18

       Selected C1 upvote

       In Databricks, Jobs allow users to monitor the performance of tasks and troubleshoot issues with specific runs. When a notebook is running slowly as part of a Job, the tech lead can use the Runs tab in the Jobs UI to examine the task's performance. Runs tab in the Jobs UI: This tab shows a list of all runs associated with the Job. The tech lead can identify the specific run where the notebook is performing poorly and click on that run to access detailed information about its performance. Once the tech lead selects the active run, they can inspect the logs, metrics, and other performance details associated with that task, which will help them identify the cause of the slowdown, such as resource contention, inefficient code, or insufficient compute.

    4. A

       Atnafu

       2023-07-08

       3 upvotes

       C In the Databricks Jobs UI, the Runs tab provides detailed information about the execution of each run in a Job. By clicking on the active run associated with the notebook running slowly, you can access the specific run details, including the notebook execution logs, execution duration, resource utilization, and any error messages or warnings.

19. Question 19 of 25

    A data engineer has a Job with multiple tasks that runs nightly. Each of the tasks runs slowly because the clusters take a long time to start. Which of the following actions can the data engineer perform to improve the start up time for the clusters used for the Job?

    • AThey can use endpoints available in Databricks SQL
    • BThey can use jobs clusters instead of all-purpose clusters
    • CThey can configure the clusters to be single-node
    • DThey can use clusters that are from a cluster pool
    • EThey can configure the clusters to autoscale for larger data sizes

    Show answer

    Correct answer: D

    How others answered

    • D92%

    Explanation

    D Cluster pools are a way to pre-provision clusters that are ready to use. This can reduce the start up time for clusters, as they do not have to be created from scratch. All-purpose clusters are not pre-provisioned, so they will take longer to start up. Jobs clusters are a type of cluster pool, but they are not the best option for this use case. Jobs clusters are designed for long-running jobs, and they can be more expensive than other types of cluster pools. Single-node clusters are the smallest type of cluster, and they will start up the fastest. However, they may not be powerful enough to run the Job's tasks. Autoscaling clusters can scale up or down based on demand. This can help to improve the start up time for clusters, as they will only be created when they are needed. However, autoscaling clusters can also be more expensive than other types of cluster pool

    Adapted from community discussion by Atnafu · 2023-07-08 · 8 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation

    View community discussion4

    Candidate context

    Question 19 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. A

       Atnafu

       2023-07-08

       8 upvotes

       D Cluster pools are a way to pre-provision clusters that are ready to use. This can reduce the start up time for clusters, as they do not have to be created from scratch. All-purpose clusters are not pre-provisioned, so they will take longer to start up. Jobs clusters are a type of cluster pool, but they are not the best option for this use case. Jobs clusters are designed for long-running jobs, and they can be more expensive than other types of cluster pools. Single-node clusters are the smallest type of cluster, and they will start up the fastest. However, they may not be powerful enough to run the Job's tasks. Autoscaling clusters can scale up or down based on demand. This can help to improve the start up time for clusters, as they will only be created when they are needed. However, autoscaling clusters can also be more expensive than other types of cluster pool

    2. G

       Garyn

       2023-12-30

       Selected D3 upvotes

       D. They can use clusters that are from a cluster pool. Explanation: Cluster Pools: Cluster pools in Databricks allow for the pre-creation and management of clusters in a pool that are readily available for use. With cluster pools, clusters are pre-initialized and kept in a ready state, minimizing the startup time when tasks need to run. This reduces the overhead of cluster initialization as the clusters are already provisioned and waiting for the tasks to be assigned. Using clusters from a pool ensures that there is no wait time for cluster initialization when the tasks start running in the nightly Job. This approach significantly reduces the time taken for clusters to start, thereby improving the overall performance and efficiency of the tasks by minimizing the overhead of cluster startup delays.

    3. V

       vctrhugo

       2023-09-03

       Selected D3 upvotes

       D. They can use clusters that are from a cluster pool. To improve startup time for the clusters used for the Job, the data engineer can configure the clusters to be sourced from a cluster pool. Cluster pools are pre-allocated clusters that are kept in a running state, ready for use. This eliminates the need to start new clusters from scratch each time a Job runs, significantly reducing startup times. Cluster pools are designed to optimize cluster reuse, making them an efficient choice for recurring jobs like the one described in the scenario. Option D provides a practical solution to address the slow cluster startup time issue.

    4. T

       TC007

       2023-04-11

       Selected D4 upvotes

       D: use clusters that are from a cluster pool. Using clusters from a cluster pool can improve the start-up time for the clusters used in the Job because the pool contains preconfigured and pre-started clusters that can be used immediately. This can save time and resources compared to starting new clusters for each task.

20. Question 20 of 25

    A new data engineering team has been assigned to work on a project. The team will need access to database customers in order to see what tables already exist. The team has its own group team. Which of the following commands can be used to grant the necessary permission on the entire database to the new team?

    • AGRANT VIEW ON CATALOG customers TO team;
    • BGRANT CREATE ON DATABASE customers TO team;
    • CGRANT USAGE ON CATALOG team TO customers;
    • DGRANT CREATE ON DATABASE team TO customers;
    • EGRANT USAGE ON DATABASE customers TO team;

    Show answer

    Correct answer: E

    How others answered

    • E77%
    • A18%

    Explanation

    The previous question 44 was all options were about permissions for DATABASE. How come all of a sudden there is no such thing as DATABASE?? E is the right answer as USAGE is a permission type required for performing any actions on the securable item, in this case a database named Customers. Options A, B, D are incorrect as there are no privileges named VIEW & CREATE exists. CREATE CATALOG, CREATE SCHEMA, CREATE TABLE etc., exists but not just CREATE. Option C has the users and database name misplaced and hence it is also incorrect.

    Adapted from community discussion by Data_4ever · 2023-04-07 · 5 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation
    • Microsoft Learn
    • Databricks documentation
    • Microsoft Learn

    View community discussion4

    Candidate context

    Question 20 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. N

       nedlo

       2023-12-12

       Selected E7 upvotes

       GRANT USAGE, answer E is correct. I dont see such privilage as GRANT VIEW https://docs.databricks.com/en/sql/language-manual/sql-ref-privileges.html#privilege-types

    2. D

       Data_4ever

       2023-04-07

       5 upvotes

       The previous question 44 was all options were about permissions for DATABASE. How come all of a sudden there is no such thing as DATABASE?? E is the right answer as USAGE is a permission type required for performing any actions on the securable item, in this case a database named Customers. Options A, B, D are incorrect as there are no privileges named VIEW & CREATE exists. CREATE CATALOG, CREATE SCHEMA, CREATE TABLE etc., exists but not just CREATE. Option C has the users and database name misplaced and hence it is also incorrect.

    3. K

       kishanu

       2023-10-19

       Selected E4 upvotes

       Customers is a Database and not a catalog. The three-space naming convention has Catalog at the top level(catalog.database(schema).table). So granting a usage on catalog will expose other objects.

    4. O

       Office2022

       2023-07-25

       2 upvotes

       The correct answer is E. GRANT ALL PRIVILEGES ON DATABASE customers TO team; The GRANT statement is used to grant privileges on a database, table, or view to a user or role. The ALL PRIVILEGES option grants all possible privileges on the specified object, such as CREATE, SELECT, MODIFY, and USAGE. The syntax of the GRANT statement is: GRANT privilege_type ON object TO user_or_role; Therefore, to grant full permissions on the database customers to the new data engineering team, the command should be: GRANT ALL PRIVILEGES ON DATABASE customers TO team; Option A is incorrect because it only grants the USAGE privilege, which allows the team to access the database but not to create or modify any tables or views in it.

21. Question 21 of 25

    A data engineer has realized that the data files associated with a Delta table are incredibly small. They want to compact the small files to form larger files to improve performance. Which of the following keywords can be used to compact the small files?

    • AREDUCE
    • BOPTIMIZE
    • CCOMPACTION
    • DREPARTITION
    • EVACUUM

    Show answer

    Correct answer: B

    How others answered

    • B100%

    Explanation

    OPTIMIZE is the correct answer. Compacting small files using the OPTIMIZE command improves table performance such as by combining multiple small files into larger ones.

    Adapted from community discussion by UGOTCOOKIES · 2024-01-26 · 2 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation

    View community discussion4

    Candidate context

    Question 21 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. K

       kishanu

       2023-10-20

       Selected B5 upvotes

       OPTIMIZE can be used to club small files into 1 and improve performance.

    2. U

       UGOTCOOKIES

       2024-01-26

       Selected B2 upvotes

       OPTIMIZE is the correct answer. Compacting small files using the OPTIMIZE command improves table performance such as by combining multiple small files into larger ones.

    3. M

       meow_akk

       2023-10-22

       3 upvotes

       Ans B : optimize is used to compact small files which in turn improves perf

    4. 8

       80370eb

       2024-08-09

       Selected B1 upvote

       The OPTIMIZE command in Delta Lake merges small files into larger files, which can help improve query performance and manage storage more efficiently.

22. Question 22 of 25

    A data engineer has created a new database using the following command: CREATE DATABASE IF NOT EXISTS customer360; In which of the following locations will the customer360 database be located?

    • Adbfs:/user/hive/database/customer360
    • Bdbfs:/user/hive/warehouse
    • Cdbfs:/user/hive/customer360
    • DMore information is needed to determine the correct response
    • Edbfs:/user/hive/database

    Show answer

    Correct answer: B

    How others answered

    • B72%
    • D28%

    Only 72% agreed on this answer — treat it as contested rather than settled.

    Explanation

    B. dbfs:/user/hive/warehouse Explanation: In Databricks, the default location for databases created in the Hive Metastore is often under the warehouse directory. The CREATE DATABASE command usually creates the metadata entry for the database in the Hive Metastore, but it doesn’t directly create the physical database directory within DBFS (Databricks File System). The exact path structure may differ based on configuration or settings in the Databricks environment, but generally, the warehouse directory is where Hive databases' metadata resides. The physical data within the database will be stored in DBFS, but the metadata for the customer360 database should be within the warehouse directory in Hive Metastore.

    Adapted from community discussion by Garyn · 2023-12-30 · 2 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation

    View community discussion4

    Candidate context

    Question 22 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. K

       kbaba101

       2023-10-21

       10 upvotes

       B B. dbfs:/user/hive/warehouse Thereby showing "dbfs:/user/hive/warehouse/customer360.db

    2. G

       Garyn

       2023-12-30

       Selected B2 upvotes

       B. dbfs:/user/hive/warehouse Explanation: In Databricks, the default location for databases created in the Hive Metastore is often under the warehouse directory. The CREATE DATABASE command usually creates the metadata entry for the database in the Hive Metastore, but it doesn’t directly create the physical database directory within DBFS (Databricks File System). The exact path structure may differ based on configuration or settings in the Databricks environment, but generally, the warehouse directory is where Hive databases' metadata resides. The physical data within the database will be stored in DBFS, but the metadata for the customer360 database should be within the warehouse directory in Hive Metastore.

    3. K

       kishanu

       2023-10-20

       Selected B3 upvotes

       dbfs:/user/hive/warehouse - which is the default location of any object created

    4. U

       UGOTCOOKIES

       2024-01-26

       Selected B2 upvotes

       Creating tables without using the LOCATION keyword to specify a location will create the table (a managed table) in the default directory which is: dbfs:/user/hive/warehouse https://docs.databricks.com/en/dbfs/root-locations.html

23. Question 23 of 25

    A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE. The table is configured to run in Development mode using the Continuous Pipeline Mode. Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after clicking Start to update the pipeline?

    • AAll datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.
    • BAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist until the pipeline is shut down.
    • CAll datasets will be updated once and the pipeline will persist without any processing. The compute resources will persist but go unused.
    • DAll datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.
    • EAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.

    Show answer

    Correct answer: E

    How others answered

    • E68%
    • B26%

    Only 68% agreed on this answer — treat it as contested rather than settled.

    Explanation

    Ans E : Development and production modes You can optimize pipeline execution by switching between development and production modes. Use the Delta Live Tables Environment Toggle Icon buttons in the Pipelines UI to switch between these two modes. By default, pipelines run in development mode. When you run your pipeline in development mode, the Delta Live Tables system does the following: Reuses a cluster to avoid the overhead of restarts. By default, clusters run for two hours when development mode is enabled. You can change this with the pipelines.clusterShutdown.delay setting in the Configure your compute settings. Disables pipeline retries so you can immediately detect and fix errors. In production mode, the Delta Live Tables system does the following: Restarts the cluster for specific recoverable errors, including memory leaks and stale credentials. Retries execution in the event of specific errors, for example, a failure to start a cluster. https://docs.databricks.com/en/delta-live-tables/updates.html#optimize-execution

    Adapted from community discussion by meow_akk · 2023-10-22 · 11 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation
    • Databricks documentation

    View community discussion4

    Candidate context

    Question 23 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. M

       meow_akk

       2023-10-22

       11 upvotes

       Ans E : Development and production modes You can optimize pipeline execution by switching between development and production modes. Use the Delta Live Tables Environment Toggle Icon buttons in the Pipelines UI to switch between these two modes. By default, pipelines run in development mode. When you run your pipeline in development mode, the Delta Live Tables system does the following: Reuses a cluster to avoid the overhead of restarts. By default, clusters run for two hours when development mode is enabled. You can change this with the pipelines.clusterShutdown.delay setting in the Configure your compute settings. Disables pipeline retries so you can immediately detect and fix errors. In production mode, the Delta Live Tables system does the following: Restarts the cluster for specific recoverable errors, including memory leaks and stale credentials. Retries execution in the event of specific errors, for example, a failure to start a cluster. https://docs.databricks.com/en/delta-live-tables/updates.html#optimize-execution

    2. G

       Garyn

       2023-12-31

       Selected E2 upvotes

       E. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing. Explanation: In Development mode, Delta Live Tables persistently updates datasets at set intervals. The pipeline continuously processes incoming data until manually stopped or shut down. Compute resources, including the cluster used for processing, persist without automatic restarts or retries (as it is the behavior in Development mode). This persistence allows for ongoing processing of data, enabling additional testing or continued data processing until the pipeline is manually shut down. Therefore, option E accurately captures the behavior expected in Development mode, emphasizing the continuous update of datasets and the persistence of compute resources until the pipeline is manually terminated.

    3. 7

       7a22144

       2024-08-16

       1 upvote

       E is correct ! Option B: "All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist until the pipeline is shut down." This option correctly reflects that the pipeline continues running, updating datasets at intervals, and only stops when manually shut down. Compute resources persist throughout this process. Option E: "All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing." While this is very similar, it adds the phrase "to allow for additional testing," which might imply that the resources are persisting just for testing purposes. This can be misleading because the primary reason for resource persistence in Continuous mode is to keep the pipeline active and processing data, not solely for testing.

    4. 5

       55f31c8

       2023-11-30

       Selected E2 upvotes

       https://docs.databricks.com/en/delta-live-tables/updates.html#continuous-vs-triggered-pipeline-execution https://docs.databricks.com/en/delta-live-tables/testing.html#use-development-mode-to-run-pipeline-updates

24. Question 24 of 25

    Which of the following Git operations must be performed outside of Databricks Repos?

    • ACommit
    • BPull
    • CPush
    • DClone
    • EMerge

    Show answer

    Correct answer: E

    How others answered

    • E68%
    • D32%

    Only 68% agreed on this answer — treat it as contested rather than settled.

    Explanation

    wrong answer, Clone can be done in Databricks Repo. Merge not in Repos, need to be in Git. Link here: https://learn.microsoft.com/en-us/azure/databricks/repos/

    Adapted from community discussion by upliftinghut · 2023-04-04 · 3 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation
    • Databricks documentation
    • Databricks documentation
    • Databricks documentation
    • Microsoft Learn

    View community discussion4

    Candidate context

    Question 24 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. Z

       ZSun

       2023-06-13

       24 upvotes

       According to the most recent one, all command is feasible in Repos

    2. N

       NickWerbung

       2023-08-31

       18 upvotes

       Not valid anymore... https://docs.databricks.com/en/repos/ci-cd-techniques-with-repos.html

    3. 9

       9d4d68a

       2024-08-26

       4 upvotes

       D. Clone Cloning a repository creates a local copy of the repository on your machine and must be done using a local Git client or command line. Once the repository is cloned, you can work with it in Databricks Repos, but the initial clone operation itself is outside the Databricks interface. Other operations like commit, pull, push, and merge can be managed within Databricks Repos or through other Git tools.

    4. U

       upliftinghut

       2023-04-04

       Selected E3 upvotes

       wrong answer, Clone can be done in Databricks Repo. Merge not in Repos, need to be in Git. Link here: https://learn.microsoft.com/en-us/azure/databricks/repos/

25. Question 25 of 25

    A dataset has been defined using Delta Live Tables and includes an expectations clause: CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION FAIL UPDATE What is the expected behavior when a batch of data containing data that violates these constraints is processed?

    • ARecords that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
    • BRecords that violate the expectation cause the job to fail.
    • CRecords that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
    • DRecords that violate the expectation are added to the target dataset and recorded as invalid in the event log.
    • ERecords that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.

    Show answer

    Correct answer: B

    How others answered

    • B100%

    Explanation

    Ans B : delta live tables data quality expectations . - https://docs.databricks.com/en/delta-live-tables/expectations.html Action Result warn (default) Invalid records are written to the target; failure is reported as a metric for the dataset. drop Invalid records are dropped before data is written to the target; failure is reported as a metrics for the dataset. fail Invalid records prevent the update from succeeding. Manual intervention is required before re-processing.

    Adapted from community discussion by meow_akk · 2023-10-22 · 4 community upvotes. Third-party contribution; reviewed by our quality filter, not independently verified.

    Supporting references

    • Databricks documentation
    • Databricks documentation

    View community discussion3

    Candidate context

    Question 25 discussion

    These are quality-filtered excerpts from third-party community discussion. They can be useful reasoning signals, but they are not official answers and have not been independently verified by DE-ASSOC or CertExamHQ.

    1. M

       meow_akk

       2023-10-22

       4 upvotes

       Ans B : delta live tables data quality expectations . - https://docs.databricks.com/en/delta-live-tables/expectations.html Action Result warn (default) Invalid records are written to the target; failure is reported as a metric for the dataset. drop Invalid records are dropped before data is written to the target; failure is reported as a metrics for the dataset. fail Invalid records prevent the update from succeeding. Manual intervention is required before re-processing.

    2. 5

       55f31c8

       2023-11-30

       Selected B2 upvotes

       https://docs.databricks.com/en/delta-live-tables/sql-ref.html#sql-properties ON VIOLATION Optional action to take for failed rows: FAIL UPDATE: Immediately stop pipeline execution. DROP ROW: Drop the record and continue processing.

    3. B

       Bakhtiyor

       2023-11-08

       2 upvotes

       ON VIOLATION FAIL UPDATE: Immediately stop pipeline execution. DROP ROW: Drop the record and continue processing.

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

DE-ASSOC practice, question 1 of 25