SPARK-DEV practice questions 211–219

1. Question 211 · 219 in bank

   What happens if a developer registers two DataFrames with the same view name, myView using the createOrReplaceTempView function?

   • ABoth DataFrames will be accessible under the same view name.
   • BThe DataFrames merged into one under the view name myView.
   • CAn Exception is thrown, indicating a duplicate view named myView.
   • DThe previous view will be replaced by the new DataFrame’s view.

   Show answer

   Answer: D

   Next question

2. Question 212 · 219 in bank

   A data engineer needs to control the size of micro-batches when running with the option of Available-now micro-batch. Which trigger should the engineer use?

   • AmaxExecutorsPerTrigger
   • BmaxCoresPerTrigger
   • CmaxBytesPerTrigger
   • DmaxFilesPerTrigger

   Show answer

   Answer: C

   Next question

3. Question 213 · 219 in bank

   How does the Directed Acyclic Graph (DAG) represent the execution hierarchy in Apache Spark™?

   • AIt represents the execution of multiple jobs, where each job runs independently of stages and tasks.
   • BIt represents the execution of a job as a series of interdependent stages, each broken into parallel tasks.
   • CIt represents the tasks executed by Spark as individual jobs, with each job containing multiple applications.
   • DIt represents only the final tasks without considering stages or jobs.

   Show answer

   Answer: B

   Next question

4. Question 214 · 219 in bank

   A data engineer has a PySpark DataFrame named sales_data containing millions of rows. They need to calculate the total sales amount grouped by region using Spark SQL for improved readability and maintainability. Which steps should the data engineer take to achieve this?

   • AWrite the query directly on the DataFrame using PySpark’s ‘groupBy’ and ‘agg’ methods.
   • BExport the DataFrame as a CSV file, read it into a relational database, and write SQL queries in the database
   • CUse the ‘createOrReplaceTempView' method to register ‘sales_data’ as a temporary view and write the query in Spark SQL.
   • DConvert ‘sales _data’ to Pandas DataFrame and use SQL-ike queries with Pandas to achieve the grouping.

   Show answer

   Answer: C

   Next question

5. Question 215 · 219 in bank

   A data engineer has been inspecting one of their scheduled Spark applications which has started running slowly on the most recent executions. Looking at the query plan the engineer notices that Adaptive Query Execution (AQE) is no longer converting one of their join operations from a sort-merge join to a broadcast join, causing extra shuffling in the Spark job. What explains the change in the behaviour of the application?

   • AThe size of the smallest table has grown larger than the configured threshold where AQE will convert the join to a broadcast join.
   • BIt has been too long since ANALYZE TABLE has been run on the tables and the table statistics are stale.
   • CA Spark version upgrade has disabled AQE.
   • DThe distribution of the join keys between the two tables has become more skewed over time.

   Show answer

   Answer: A

   Next question

6. Question 216 · 219 in bank

   Select 2

   A data scientist is working with a massive dataset that exceeds the memory capacity of a single machine. The data scientist is considering using Apache SparkTM instead of processing the data using traditional single-machine programming languages like standard Python scripts. Which two advantages does Apache SparkTM offer over a normal single-machine language in this scenario? (Choose two.)

   • AIt eliminates the need to write any code, automatically handling all data processing.
   • BIt has built-in fault tolerance, allowing it to recover seamlessly from node failures during computation.
   • CIt processes data solely on disk storage, reducing the need for memory resources.
   • DIt can distribute data processing tasks across a cluster of machines, enabling horizontal scalability.
   • EIt requires specialized hardware to run, making it unsuitable for commodity hardware clusters.

   Show answer

   Answer: BD

   Next question

7. Question 217 · 219 in bank

   A data scientist at a large e-commerce company needs to process and analyze 2 TB of daily customer transaction data. The company wants to implement real-time fraud detection and personalized product recommendations. To process their data the company uses a traditional relational database system, which is struggling to handle the increasing data volume and velocity. Which feature of Apache Spark effectively addresses the challenge posed in this scenario?

   • AAbility to process small datasets efficiently
   • BSupport for SQL queries on structured data
   • CIn-memory computation and parallel processing capabilities
   • DBuilt-in machine learning libraries

   Show answer

   Answer: C

   Next question

8. Question 218 · 219 in bank

   A data engineer is investigating a Spark cluster that is experiencing underutilization during scheduled batch jobs. After checking the Spark logs, they noticed that tasks are often getting killed due to timeout errors, and there are several warnings about insufficient resources in the logs. Which action should the engineer take to resolve the underutilization issue?

   • AIncrease the executor memory allocation in the Spark configuration.
   • BSet the spark.network.timeout property to allow tasks more time to complete without being killed.
   • CIncrease the number of executor instances to handle more concurrent tasks.
   • DReduce the size of the data partition to improve task scheduling.

   Show answer

   Answer: C

   Next question

9. Question 219 · 219 in bank

   A data engineer is working on num_df DataFrame: num_df = spark.range(5).toDF(“num”) The engineer is using the Python UDF: def cubefunc(val): return val ** 3 Which code fragment registers and uses this UDF as a Spark SQL function to work with the DataFrame num_df?

   • Aspark.udf.register(“cubeudf”, cubefunc, IntegerType()) num_df.selectExpr(“cubeudf(num)”)
   • Bcubeudf = udf (cubefunc) num_df.select(cubeudf(col(“num”)))
   • Cspark.udf.register(“cubeudf”, cubefunc, DoubleType()) num_df.selectExpr(“cubeudf(num)”)
   • Dcubeudf = udf(cubefunc) num_df.selectExpr(cubeudf(num)”)

   Show answer

   Answer: C

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

SPARK-DEV practice, question 211. 219 questions available.