SPARK-DEV practice questions 31–40

1. Question 31 · 219 in bank

   Which of the following types of processes induces a stage boundary?

   • AShuffle
   • BCaching
   • CExecutor failure
   • DJob delegation
   • EApplication failure

   Show answer

   Answer: A

   Explanation

   A. Shuffle A stage boundary in Spark is induced when data needs to be shuffled across the executors. A shuffle occurs when the current operations require data to be redistributed so that new partitions of data are formed based on which downstream operations are computed.

   Source: Adapted from community discussion — Sowwy1

   Next question

2. Question 32 · 219 in bank

   Which of the following cluster configurations will induce the least network traffic during a shuffle operation?

   

   Note: each configuration has roughly the same compute power using 100GB of RAM and 200 cores.

   • AThis cannot be determined without knowing the number of partitions.
   • BScenario 5
   • CScenario 1
   • DScenario 4
   • EScenario 6

   Show answer

   Answer: C

   Explanation

   Scenario #1 would likely induce the least network traffic during a shuffle operation because all the data would stay within the single node, avoiding any cross-node traffic. Therefore, the answer is: C. Scenario 1

   Source: Adapted from community discussion — Sowwy1

   Next question

3. Question 33 · 219 in bank

   Which of the following describes a partition?

   • AA partition is the amount of data that fits in a single executor.
   • BA partition is an automatically-sized segment of data that is used to create efficient logical plans.
   • CA partition is the amount of data that fits on a single worker node.
   • DA partition is a portion of a Spark application that is made up of similar jobs.
   • EA partition is a collection of rows of data that fit on a single machine in a cluster.

   Show answer

   Answer: E

   Next question

4. Question 34 · 219 in bank

   Which of the following identifies multiple narrow operations that are executed in sequence?

   • ASlot
   • BJob
   • CStage
   • DTask
   • EExecutor

   Show answer

   Answer: C

   Next question

5. Question 35 · 219 in bank

   Which of the following cluster configurations is most likely to experience an out-of-memory error in response to data skew in a single partition?

   

   Note: each configuration has roughly the same compute power using 100 GB of RAM and 200 cores.

   • AScenario #4
   • BScenario #5
   • CScenario #6
   • DMore information is needed to determine an answer.
   • EScenario #1

   Show answer

   Answer: C

   Explanation

   The most likely scenario to experience an out-of-memory error in response to data skew in a single partition is: C. Scenario #6: 12.5 GB Worker Node, 12.5 GB Executor. 1 Driver & 8 Executors. Explanation: Data skew refers to an uneven distribution of data across partitions. When there is significant skew in a single partition, it can lead to increased memory usage for that specific partition, potentially causing out-of-memory errors. The smaller the available memory per executor, the higher the likelihood of encountering such issues. In this case, Scenario #6 has the smallest worker node and executor configuration, with only 12.5 GB of RAM available for each executor. With 8 executors, the total available memory is still 100 GB (similar to other scenarios), but the reduced memory per executor increases the risk of encountering out-of-memory errors when handling skewed data in a single partition.

   Source: Adapted from community discussion — TmData

   Next question

6. Question 36 · 219 in bank

   Spark's execution/deployment mode determines where the driver and executors are physically located when a Spark application is run. Which of the following Spark execution/deployment modes does not exist? If they all exist, please indicate so with Response E.

   • AClient mode
   • BCluster mode
   • CStandard mode
   • DLocal mode
   • EAll of these execution/deployment modes exist

   Show answer

   Answer: C

   Next question

7. Question 37 · 219 in bank

   Which of the following will cause a Spark job to fail?

   • ANever pulling any amount of data onto the driver node.
   • BTrying to cache data larger than an executor's memory.
   • CData needing to spill from memory to disk.
   • DA failed worker node.
   • EA failed driver node.

   Show answer

   Answer: E

   Next question

8. Question 38 · 219 in bank

   Which of the following best describes the similarities and differences between the MEMORY_ONLY storage level and the MEMORY_AND_DISK storage level?

   • AThe MEMORY_ONLY storage level will store as much data as possible in memory and will store any data that does on fit in memory on disk and read it as it's called. The MEMORY_AND_DISK storage level will store as much data as possible in memory and will recompute any data that does not fit in memory as it’s called.
   • BThe MEMORY_ONLY storage level will store as much data as possible in memory on two cluster nodes and will recompute any data that does not fit in memory as it’s called. The MEMORY_AND_DISK storage level will store as much data as possible in memory on two cluster nodes and will store any data that does on fit in memory on disk and read it as it's called.
   • CThe MEMORY_ONLY storage level will store as much data as possible in memory on two cluster nodes and will store any data that does on fit in memory on disk and read it as it's called. The MEMORY_AND_DISK storage level will store as much data as possible in memory on two cluster nodes and will recompute any data that does not fit in memory as it's called.
   • DThe MEMORY_ONLY storage level will store as much data as possible in memory and will recompute any data that does not fit in memory as it's called. The MEMORY_AND_DISK storage level will store as much data as possible in memory and will store any data that does on fit in memory on disk and read it as it's called.
   • EThe MEMORY_ONLY storage level will store as much data as possible in memory and will recompute any data that does not fit in memory as it’s called. The MEMORY_AND_DISK storage level will store half of the data in memory and store half of the memory on disk. This provides quick preview and better logical plan design.

   Show answer

   Answer: D

   Next question

9. Question 39 · 219 in bank

   Which of the following Spark properties is used to configure whether DataFrames found to be below a certain size threshold at runtime will be automatically broadcasted?

   • Aspark.sql.broadcastTimeout
   • Bspark.sql.autoBroadcastJoinThreshold
   • Cspark.sql.shuffle.partitions
   • Dspark.sql.inMemoryColumnarStorage.batchSize
   • Espark.sql.adaptive.localShuffleReader.enabled

   Show answer

   Answer: B

   Next question

10. Question 40 · 219 in bank

    The code block shown below contains an error. The code block is intended to return a new DataFrame from DataFrame storesDF where column storeId is of the type string. Identify the error. Code block: storesDF.withColumn(“storeId”, cast(col(“storeId”), StringType()))

    • ACalls to withColumn() cannot create a new column of the same name on which it is operating.
    • BDataFrame columns cannot be converted to a new type inside of a call to withColumn().
    • CThe call to StringType should not be followed by parentheses.
    • DThe column name storeId inside the col() operation should not be quoted.
    • EThe cast() operation is a method in the Column class rather than a standalone function.

    Show answer

    Answer: E

    Next question

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

SPARK-DEV practice, question 31. 219 questions available.