DE-PRO practice questions 171–180

1. Question 171 · 202 in bank

   A data engineer wants to create a cluster using the Databricks CLI for a big ETL pipeline. The cluster should have five workers and one driver of type i3.xlarge and should use the '14.3.x-scala2.12' runtime. Which command should the data engineer use?

   • Adatabricks compute add 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster-name Data Engineer_cluster
   • Bdatabricks clusters create 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster-name Data Engineer_cluster
   • Cdatabricks compute create 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster-name Data Engineer_cluster
   • Ddatabricks clusters add 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster-name Data Engineer_cluster

   Show answer

   Answer: B

   Next question

2. Question 172 · 202 in bank

   A 'transactions' table has been liquid clustered on the columns 'product_id’, ’user_id' and 'event_date'. Which operation lacks support for cluster on write?

   • ACTAS and RTAS statements
   • Bspark.writeStream.format(’delta').mode(’append’)
   • Cspark.write.format('delta’).mode('append')
   • DINSERT INTO operations

   Show answer

   Answer: B

   Next question

3. Question 173 · 202 in bank

   The data governance team has instituted a requirement that the "user" table containing Personal Identifiable Information (PII) must have the appropriate masking on the SSN column. This means that anyone outside of the HRAdminGroup should see masked social security numbers as ***-**-****. The team created a masking function:

   

   What does the data governance team need to do next to achieve this goal?

   • ACREATE TABLE users - (name STRING); ALTER TABLE users CREATE COLUMN ssn CREATE MASK ssn_mask;
   • BCREATE TABLE users - (name STRING, int STRING); ALTER TABLE users ALTER COLUMN ssn CREATE MASK if is_member('HRAdminGroup');
   • CCREATE TABLE users - (name STRING, ssn INT MASKED ssn_mask);
   • DCREATE TABLE users - (name STRING, ssn STRING); ALTER TABLE users ALTER COLUMN ssn SET MASK ssn_mask;

   Show answer

   Answer: D

   Next question

4. Question 174 · 202 in bank

   A data engineer needs to create an application that will collect information about the latest job run including the repair history. How should the data engineer format the request?

   • ACall/api/2.1/jobs/runs/list with the run_id and include_history parameters
   • BCall/api/2.1/jobs/runs/get with the run_id and include_history parameters
   • CCall/api/2.1/jobs/runs/get with the job_id and include_history parameters
   • DCall/api/2.1/jobs/runs/list with the job_id and include_history parameters

   Show answer

   Answer: B

   Next question

5. Question 175 · 202 in bank

   A data engineer is working in an interactive notebook with many transformations before outputting the result from display(df.collect() ). The notebook includes wide transformations and a cross join. The data engineer is getting the following error: "The spark driver has stopped unexpectedly and is restarting. Your notebook will be automatically reattached." Which action should the data engineer take?

   • ARun the notebook on a single node cluster to keep driver from falling.
   • BRewrite their code to avoid putting memory pressure on the driver node.
   • CCheck into the Spark UI to see how many jobs are assigned to each stage as they are employing fewer executors.
   • DLook at the compute metrics UI to see if the executors have higher than 90% memory utilization.

   Show answer

   Answer: B

   Next question

6. Question 176 · 202 in bank

   An analytics team wants run an experiment in the short term on the customer transaction Delta table (with 20 billions records) created by the data engineering team in Databricks SQL. Which strategy should the data engineering team use to ensure minimal downtime and no impact on the ongoing ETL processes?

   • ADeep clone the table for the analytics team.
   • BCreate a new table for the analytics team using a CTAS statement.
   • CShallow clone the table for the analytics team.
   • DGive access to the table for the analytics team.

   Show answer

   Answer: C

   Next question

7. Question 177 · 202 in bank

   A data team is working to optimize an existing large, fast-growing table 'orders' with high cardinality columns, which experiences significant data skew and requires frequent concurrent writes. The team notice that the columns 'user_id', 'event_timestamp' and 'product_id' are heavily used in analytical queries and filters, although those keys may be subject to change in the future due to different business requirements. Which partitioning strategy should the team choose to optimize the table for immediate data skipping, incremental management over time, and flexibility?

   • APartition the table with: ALTER TABLE orders PARTITION BY user_id, product_id, event_timestamp
   • BUse z-order after partitiing the table: OPTIMIZE orders ZORDER BY (user_id, product_id) WHERE event_timestamp = current date () - 1 DAY
   • CCluster the table with: ALTER TABLE orders CLUSTER BY user_id, product_id, event_timestamp
   • DZ-order the table with OPTIMIZE orders ZORDER BY (user_id, product_id, event_timestamp)

   Show answer

   Answer: C

   Next question

8. Question 178 · 202 in bank

   A data engineer is optimizing a managed table that suffers from data skew and frequently changing query filter columns. The engineer needs to avoid costly data rewrites when query patterns evolve. The table size is under 1TB. How should data engineer meet this requirement?

   • AUse Hive-style partitioning, as it provides efficient data skipping and is easy to change partition columns at any time.
   • BCombine partitioning and Z-ordering to maximize flexibility and minimize maintenance as query patterns change.
   • CEnable liquid clustering, as it efficiently handles data skew, allows clustering keys to be changed without rewriting existing data, and adapts to evolving query patterns.
   • DApply Z-ordering, since it allows flexible reorganization of data layout without rewriting existing and adapts easily to new filter columns.

   Show answer

   Answer: C

   Next question

9. Question 179 · 202 in bank

   Which approach demonstrates a modular and testable way to use DataFrame transform for ETL code in PySpark?

   • A//IMG//
   • B//IMG//
   • C//IMG//
   • D//IMG//

   Show answer

   Answer: C

   Next question

10. Question 180 · 202 in bank

    A data engineering team needs to create a SQL Alert that monitors data quality across multiple columns in their customer table. They want to trigger an alert when both the percentage of customers with missing email addresses exceeds 15% AND the percentage of customers with invalid phone number formats exceeds 10%. Which SQL query pattern is appropriate for implementing this multi-column alert condition?

    • ASELECT COUNT (*) FROM customers WHERE email IS NULL OR phone_format_invalid = true
    • BSELECT email, phone FROM customers WHERE email IS NULL AND phone NOT RLIKE ‘ˆ[0-9-+()\\s]+$’
    • CSELECT email_null_pct, phone_invalid_pct FROM (SELECT (COUNT(CASE WHEN email IS NULL THEN 1 END) * 100.0/COUNT (*)) as email_null_pct, (COUNT(CASE WHEN phone NOT RLIKE ‘ˆ[0-9-+()\\s]+$’ THEN 1 END)* 100.0/COUNT (*)) as phone_invalid_pct FROM customers)
    • DSELECT CASE WHEN email_null_pct >15 AND phone_invalid_pct> 10 THEN 1 ELSE 0 END FROM (SELECT (COUNT (CASE WHEN email IS NULL THEN 1 END) * 100.0 / COUNT (*)) as phone_invalid_pct FROM customers) metrics

    Show answer

    Answer: C

    Next question

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

DE-PRO practice, question 171. 202 questions available.