SPARK-DEV practice questions 11–20

1. Question 11 · 219 in bank

   The code block shown below should cache DataFrame storesDF only in Spark's memory. Choose the response that correctly fil ls in the numbered blanks within the code block to complete this task. Code block: __1__.__2__(__3__).count()

   • A1. storesDF 2. cache 3. StorageLevel.MEMORY_ONLY
   • B1. storesDF 2. storageLevel 3. cache
   • C1. storesDF 2. cache 3. Nothing
   • D1. storesDF 2. persist 3. Nothing
   • E1. storesDF 2. persist 3. StorageLevel.MEMORY_ONLY

   Show answer

   Answer: E

   Next question

2. Question 12 · 219 in bank

   Which of the following code blocks returns a DataFrame containing a column month, an integer representation of the day of the year from column openDate from DataFrame storesDF. Note that column openDate is of type integer and represents a date in the UNIX epoch format – the number of seconds since midnight on January 1st, 1970. A sample of storesDF is displayed below:

   

   Code block: stored.withColumn(“openTimestamp”, col(“openDate”).cast(__1__)) .withColumn(__2__, __3__(__4__))

   • A1. “Data” 2. month 3. “month” 4. “openTimestamp”
   • B1. “Timestamp” 2. month 3. “month” 4. col(“openTimestamp”)
   • C1. “Timestamp” 2. month 3. getMonth 4. col(“openTimestamp”)
   • D1. “Timestamp” 2. “month” 3. month 4. col(“openTimestamp”)

   Show answer

   Answer: D

   Next question

3. Question 13 · 219 in bank

   Which of the following describes the difference between cluster and client execution modes?

   • AThe cluster execution mode runs the driver on a worker node within a cluster, while the client execution mode runs the driver on the client machine (also known as a gateway machine or edge node).
   • BThe cluster execution mode is run on a local cluster, while the client execution mode is run in the cloud.
   • CThe cluster execution mode distributes executors across worker nodes in a cluster, while the client execution mode runs a Spark job entirely on one client machine.
   • DThe cluster execution mode runs the driver on the cluster machine (also known as a gateway machine or edge node), while the client execution mode runs the driver on a worker node within a cluster.
   • EThe cluster execution mode distributes executors across worker nodes in a cluster, while the client execution mode submits a Spark job from a remote machine to be run on a remote, unconfigurable cluster.

   Show answer

   Answer: A

   Explanation

   Explanation: In cluster mode, the driver runs on the master node, while in client mode, the driver runs on a virtual machine in the cloud. This is wrong, since execution modes do not specify whether workloads are run in the cloud or on-premise. In cluster mode, each node will launch its own executor, while in client mode, executors will exclusively run on the client machine. Wrong, since in both cases executors run on worker nodes. In cluster mode, the driver runs on the edge node, while the client mode runs the driver in a worker node. Wrong C in cluster mode, the driver runs on a worker node. In client mode, the driver runs on the client machine. In client mode, the cluster manager runs on the same host as the driver, while in cluster mode, the cluster manager runs on a separate node. No. In both modes, the cluster manager is typically on a separate node C not on the same host as the driver. It only runs on the same host as the driver in local execution mode. More info: Learning Spark, 2nd Edition, Chapter 1, and Spark: The Definitive Guide, Chapter 15. ()

   Source: Adapted from community discussion — zozoshanky

   Next question

4. Question 14 · 219 in bank

   The code block shown below contains an error. The code block intended to return a new DataFrame that is the result of an inner join between DataFrame storesDF and DataFrame employeesDF on column storeId. Identify the error. Code block: StoresDF.join(employeesDF, Seq("storeId")

   • AThe key column storeId needs to be a string like “storeId”.
   • BThe key column storeId needs to be specified in an expression of both Data Frame columns like storesDF.storeId ===employeesDF.storeId.
   • CThe default argument to the joinType parameter is “inner” - an additional argument of “left” must be specified.
   • DThere is no DataFrame.join() operation - DataFrame.merge() should be used instead.
   • EThe key column storeId needs to be wrapped in the col() operation.

   Show answer

   Answer: A

   Next question

5. Question 15 · 219 in bank

   Which of the following pairs of arguments cannot be used in DataFrame.join() to perform an inner join on two DataFrames, named and aliased with "a" and "b" respectively, to specify two key columns column1 and column2?

   • AjoinExprs = col(“a.column1”) === col(“b.column1”) and col(“a.column2”) === col(“b.column2”)
   • BusingColumns = Seq(col(“column1”), col(“column2”))
   • CAll of these options can be used to perform an inner join with two key columns.
   • DjoinExprs = storesDF(“column1”) === employeesDF(“column1”) and storesDF(“column2”) === employeesDF (“column2”)
   • EusingColumns = Seq(“column1”, “column2”)

   Show answer

   Answer: B

   Explanation

   Will option D work? I have never seen this way of accessing columns in a dataframe : df("column") , it should be either df.column or col("a.column") [ considering a as an alias of df]

   Source: Adapted from community discussion — Alucard069

   Next question

6. Question 16 · 219 in bank

   The code block shown below contains an error. The code block is intended to return a new DataFrame that is the result of a position-wise union between DataFrame storesDF and DataFrame acquiredStoresDF.

   • Aconcat(storesDF, acquiredStoresDF)
   • BstoresDF.unionByName(acquiredStoresDF)
   • Cunion(storesDF, acquiredStoresDF)
   • DunionAll(storesDF, acquiredStoresDF)
   • EstoresDF.union(acquiredStoresDF)

   Show answer

   Answer: E

   Next question

7. Question 17 · 219 in bank

   Which of the following code blocks writes DataFrame storesDF to file path filePath as parquet overwriting any existing files in that location?

   • AstoresDF.write(filePath, mode = “overwrite”)
   • BstoresDF.write().mode(“overwrite”).parquet(filePath)
   • CstoresDF.write.mode(“overwrite”).parquet(filePath)
   • DstoresDF.write.option(“parquet”, “overwrite”).path(filePath)
   • EstoresDF.write.mode(“overwrite”).path(filePath)

   Show answer

   Answer: C

   Next question

8. Question 18 · 219 in bank

   Which of the following code blocks reads a CSV at the file path filePath into a Data Frame with the specified schema schema?

   • Aspark.read().csv(filePath)
   • Bspark.read().schema(“schema”).csv(filePath)
   • Cspark.read.schema(schema).csv(filePath)
   • Dspark.read.schema(“schema”).csv(filePath)
   • Espark.read().schema(schema).csv(filePath)

   Show answer

   Answer: C

   Next question

9. Question 19 · 219 in bank

   Which of the following code blocks returns a DataFrame containing only the rows from DataFrame storesDF where the value in column sqft is less than or equal to 25,000 AND the value in column customerSatisfaction is greater than or equal to 30?

   • AstoresDF.filter(col("sqft") <= 25000 and col("customerSatisfaction") >= 30)
   • BstoresDF.filter(col("sqft") <= 25000 or col("customerSatisfaction") >= 30)
   • CstoresDF.filter(sqft) <= 25000 and customerSatisfaction >= 30)
   • DstoresDF.filter(col("sqft") <= 25000 & col("customerSatisfaction") >= 30)
   • EstoresDF.filter(sqft <= 25000) & customerSatisfaction >= 30)

   Show answer

   Answer: D

   Explanation

   in pyspark, all wrong as the conditions inside the filter should be wrapped inside parentesis. should be: D. storesDF.filter((col("sqft") = 30))

   Source: Adapted from community discussion — gaco

   Next question

10. Question 20 · 219 in bank

    Which of the following sets of DataFrame methods will both return a new DataFrame only containing rows that meet a specified logical condition?

    • Adrop(), where()
    • Bfilter(), select()
    • Cfilter(), where()
    • Dselect(), where()
    • Efilter(), drop()

    Show answer

    Answer: C

    Next question

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

SPARK-DEV practice, question 11. 219 questions available.