DE-ASSOC practice questions 21–30

1. Question 21 · 170 in bank

   What is used by Spark to record the offset range of the data being processed in each trigger in order for Structured Streaming to reliably track the exact progress of the processing so that it can handle any kind of failure by restarting and/or reprocessing?

   • ACheckpointing and Write-ahead Logs
   • BReplayable Sources and Idempotent Sinks
   • CWrite-ahead Logs and Idempotent Sinks
   • DCheckpointing and Idempotent Sinks

   Show answer

   Answer: D

   Explanation

   Repeated, Correct The correct answer is A. Checkpointing and Write-ahead Logs. Checkpointing records the progress of streaming queries, while write-ahead logs (WALs) capture the data before it is processed, allowing Spark to recover and process data reliably in case of failures.

   Source: Adapted from community discussion — 9d4d68a

   Next question

2. Question 22 · 170 in bank

   What describes the relationship between Gold tables and Silver tables?

   • AGold tables are more likely to contain aggregations than Silver tables.
   • BGold tables are more likely to contain valuable data than Silver tables.
   • CGold tables are more likely to contain a less refined view of data than Silver tables.
   • DGold tables are more likely to contain truthful data than Silver tables.

   Show answer

   Answer: A

   Next question

3. Question 23 · 170 in bank

   What describes when to use the CREATE STREAMING LIVE TABLE (formerly CREATE INCREMENTAL LIVE TABLE) syntax over the CREATE LIVE TABLE syntax when creating Delta Live Tables (DLT) tables using SQL?

   • ACREATE STREAMING LIVE TABLE should be used when the subsequent step in the DLT pipeline is static.
   • BCREATE STREAMING LIVE TABLE should be used when data needs to be processed incrementally.
   • CCREATE STREAMING LIVE TABLE should be used when data needs to be processed through complicated aggregations.
   • DCREATE STREAMING LIVE TABLE should be used when the previous step in the DLT pipeline is static.

   Show answer

   Answer: B

   Next question

4. Question 24 · 170 in bank

   A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE. The table is configured to run in Production mode using the Continuous Pipeline Mode. What is the expected outcome after clicking Start to update the pipeline assuming previously unprocessed data exists and all definitions are valid?

   • AAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.
   • BAll datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.
   • CAll datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.
   • DAll datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.

   Show answer

   Answer: C

   Explanation

   Continuous Pipeline Mode in Production mode implies that the pipeline continuously processes incoming data updates at set intervals, ensuring the datasets are kept up-to-date as new data arrives. Since the pipeline is set to Continuous Pipeline Mode, it will keep running and updating the datasets until it is manually shut down. The compute resources are allocated dynamically to process and update the datasets as needed, and they will be terminated when the pipeline is stopped or shut down. This mode allows for real-time or near-real-time updates to the datasets from the streaming/live tables, ensuring that the data remains current and reflects the changes occurring in the data sources.

   Source: Adapted from community discussion — CommanderBigMac

   Next question

5. Question 25 · 170 in bank

   Which type of workloads are compatible with Auto Loader?

   • AStreaming workloads
   • BMachine learning workloads
   • CServerless workloads
   • DBatch workloads

   Show answer

   Answer: A

   Next question

6. Question 26 · 170 in bank

   A data engineer has developed a data pipeline to ingest data from a JSON source using Auto Loader, but the engineer has not provided any type inference or schema hints in their pipeline. Upon reviewing the data, the data engineer has noticed that all of the columns in the target table are of the string type despite some of the fields only including float or boolean values. Why has Auto Loader inferred all of the columns to be of the string type?

   • AAuto Loader cannot infer the schema of ingested data
   • BJSON data is a text-based format
   • CAuto Loader only works with string data
   • DAll of the fields had at least one null value

   Show answer

   Answer: B

   Next question

7. Question 27 · 170 in bank

   Which statement regarding the relationship between Silver tables and Bronze tables is always true?

   • ASilver tables contain a less refined, less clean view of data than Bronze data.
   • BSilver tables contain aggregates while Bronze data is unaggregated.
   • CSilver tables contain more data than Bronze tables.
   • DSilver tables contain less data than Bronze tables.

   Show answer

   Answer: D

   Next question

8. Question 28 · 170 in bank

   A dataset has been defined using Delta Live Tables and includes an expectations clause: CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW What is the expected behavior when a batch of data containing data that violates these constraints is processed?

   • ARecords that violate the expectation cause the job to fail.
   • BRecords that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
   • CRecords that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
   • DRecords that violate the expectation are added to the target dataset and recorded as invalid in the event log.

   Show answer

   Answer: C

   Explanation

   C. Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log. When a constraint defined using the EXPECT clause is violated, Delta Live Tables will drop the records that violate the expectation from the target dataset. Additionally, information about the dropped records and the reason for their exclusion will be recorded in the event log for audit and monitoring purposes. This ensures that only valid data meeting the specified constraints is included in the target dataset.

   Source: Adapted from community discussion — d39c1db

   Next question

9. Question 29 · 170 in bank

   A data engineer has a Job with multiple tasks that runs nightly. Each of the tasks runs slowly because the clusters take a long time to start. Which action can the data engineer perform to improve the start up time for the clusters used for the Job?

   • AThey can use endpoints available in Databricks SQL
   • BThey can use jobs clusters instead of all-purpose clusters
   • CThey can configure the clusters to autoscale for larger data sizes
   • DThey can use clusters that are from a cluster pool

   Show answer

   Answer: D

   Next question

10. Question 30 · 170 in bank

    A data engineer has a single-task Job that runs each morning before they begin working. After identifying an upstream data issue, they need to set up another task to run a new notebook prior to the original task. Which approach can the data engineer use to set up the new task?

    • AThey can clone the existing task in the existing Job and update it to run the new notebook.
    • BThey can create a new task in the existing Job and then add it as a dependency of the original task.
    • CThey can create a new task in the existing Job and then add the original task as a dependency of the new task.
    • DThey can create a new job from scratch and add both tasks to run concurrently.

    Show answer

    Answer: B

    Next question

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

DE-ASSOC practice, question 21. 170 questions available.