SPARK-DEV practice questions 161–170

1. Question 161 · 219 in bank

   Which of the following code blocks returns the first 3 rows of DataFrame storesDF?

   • AstoresDF.top_n(3)
   • BstoresDF.n(3)
   • CstoresDF.take(3)
   • DstoresDF.head(3)
   • EstoresDF.collect(3)

   Show answer

   Answer: C

   Explanation

   odd question indeed. head() and take() methods perform a similar operation of retrieving rows from the beginning of the DataFrame. However, head() may be slightly slower than take() due to additional checks for boundary conditions. I can't see the difference in terms of retrieving 1st three rows.

   Source: Adapted from community discussion — azure_bimonster

   Next question

2. Question 162 · 219 in bank

   The code block shown below contains an error. The code block is intended to print the schema of DataFrame storesDF. Identify the error. Code block: storesDF.printSchema.getAs[String]

   • AThere is no printSchema member of DataFrame – the getSchema() operation should be used instead.
   • BThere is no printSchema member of DataFrame – the schema() operation should be used instead.
   • CThe entire line needs to be a string – it should be wrapped by str().
   • DThe printSchema member of DataFrame is an operation prints the DataFrame – there is no need to call getAs.
   • EThere is no printSchema member of DataFrame – schema and the print() function should be used instead.

   Show answer

   Answer: D

   Next question

3. Question 163 · 219 in bank

   Which of the following code blocks creates and registers a SQL UDF named "ASSESS_PERFORMANCE" using the Scala function assessPerformance() and applies it to column customerSatisfaction in table stores?

   • Aspark.udf.register("ASSESS_PERFORMANCE", assessPerformance) spark.sql("SELECT customerSatisfaction, ASSESS_PERFORMANCE(customerSatisfaction) AS result FROM stores")
   • Bspark.udf.register("ASSESS_PERFORMANCE", assessPerformance)
   • Cspark.udf.register("ASSESS_PERFORMANCE", assessPerformance) spark.sql("SELECT customerSatisfaction, assessPerformance(customerSatisfaction) AS result FROM stores")
   • Dspark.udf.register("ASSESS_PERFORMANCE", assessPerformance) storesDF.withColumn("result", assessPerformance(col("customerSatisfaction")))
   • Espark.udf.register("ASSESS_PERFORMANCE", assessPerformance) storesDF.withColumn("result", ASSESS_PERFORMANCE(col("customerSatisfaction")))

   Show answer

   Answer: A

   Next question

4. Question 164 · 219 in bank

   The code block shown below should use SQL to return a new DataFrame containing column storeId and column managerName from a table created from DataFrame storesDF. Choose the response that correctly fills in the numbered blanks within the code block to complete this task. Code block: __1__.__2__("stores") __3__.__4__("SELECT storeId, managerName FROM stores")

   • A1. spark 2. createOrReplaceTempView 3. storesDF 4. query
   • B1. spark 2. createTable 3. storesDF 4. sql
   • C1. storesDF 2. createOrReplaceTempView 3. spark 4. query
   • D1. spark 2. createOrReplaceTempView 3. storesDF 4. sql
   • E1. storesDF 2. createOrReplaceTempView 3. spark 4. sql

   Show answer

   Answer: E

   Next question

5. Question 165 · 219 in bank

   The code block shown below contains an error. The code block intended to create a single-column DataFrame from Scala List years which is made up of integers. Identify the error. Code block: spark.createDataset(years)

   • AThe years list should be wrapped in another list like List(years) to make clear that it is a column rather than a row.
   • BThe data type is not specified – the second argument to createDataset should be IntegerType.
   • CThere is no operation createDataset – the createDataFrame operation should be used instead.
   • DThe result of the above is a Dataset rather than a DataFrame – the toDF operation must be called at the end.
   • EThe column name must be specified as the second argument to createDataset.

   Show answer

   Answer: D

   Explanation

   Correct answer is D: spark.createDataset() creates a Dataset, not a DataFrame. To convert a Dataset to a DataFrame, you use toDF().

   Source: Adapted from community discussion — Dharma49

   Next question

6. Question 166 · 219 in bank

   Which of the following code blocks will always return a new 4-partition DataFrame from the 8-partition DataFrame storesDF without inducing a shuffle?

   • AstoresDF.repartition(4, "sqft")
   • BstoresDF.repartition()
   • CstoresDF.coalesce(4)
   • DstoresDF.repartition(4)
   • EstoresDF.coalesce

   Show answer

   Answer: C

   Next question

7. Question 167 · 219 in bank

   The code block shown below should return a new 12-partition DataFrame from DataFrame storesDF. Choose the response that correctly fills in the numbered blanks within the code block to complete this task. Code block: __1__.__2__(__3__)

   • A1. storesDF 2. coalesce 3. 4
   • B1. storesDF 2. coalesce 3. 4, "storeId"
   • C1. storesDF 2. repartition 3. "storeId"
   • D1. storesDF 2. repartition 3. 12
   • E1. storesDF 2. repartition 3. Nothing

   Show answer

   Answer: D

   Next question

8. Question 168 · 219 in bank

   The code block shown below contains an error. The code block is intended to adjust the number of partitions used in wide transformations like join() to 32. Identify the error. Code block: spark.conf.set("spark.default.parallelism", "32")

   • Aspark.default.parallelism is not the right Spark configuration parameter – spark.sql.shuffle.partitions should be used instead.
   • BThere is no way to adjust the number of partitions used in wide transformations – it defaults to the number of total CPUs in the cluster.
   • CSpark configuration parameters cannot be set in runtime.
   • DSpark configuration parameters are not set with spark.conf.set().
   • EThe second argument should not be the string version of "32" – it should be the integer 32.

   Show answer

   Answer: A

   Next question

9. Question 169 · 219 in bank

   The code block shown below contains an error. The code block intended to return a DataFrame containing a column dayOfYear, an integer representation of the day of the year from column openDate from DataFrame storesDF. Identify the error. Note that column openDate is of type integer and represents a date in the UNIX epoch format – the number of seconds since midnight on January 1st, 1970. A sample of storesDF is displayed below:

   

   Code block: storesDF.withColumn("dayOfYear", dayofyear(col("openDate")))

   • AThe dayofyear() operation cannot extract the day of year from a column of type integer – column openDate must first be converted to type Timestamp.
   • BThe dayofyear() operation takes a quoted column name rather than a Column object as its first argument – the first argument should be "openDate".
   • CThe dayofyear() operation cannot extract the day of year from a column of type integer – column openDate must first be converted to type Date.
   • DThe dayofyear() operation is not applicable in a withColumn() call – the newColumn() operation must be used instead.
   • EThere is no dayofyear() operation – the day of year number must be extracted using substring utilities.

   Show answer

   Answer: A

   Next question

10. Question 170 · 219 in bank

    Which of the following DataFrame operations is classified as an action?

    • ADataFrame.drop()
    • BDataFrame.coalesce()
    • CDataFrame.take()
    • DDataFrame.join()
    • EDataFrame.filter()

    Show answer

    Answer: C

    Explanation

    The DataFrame operation classified as an action is: C. DataFrame.take() Explanation: In Spark, actions are operations that trigger the execution of transformations on a DataFrame and return results or side effects. Actions are evaluated eagerly, meaning they initiate the execution of the computation plan built by transformations. Among the options provided, DataFrame.take() is an action because it returns an array with the first n elements from the DataFrame as an array. It triggers the execution of any pending transformations and collects the resulting data.

    Source: Adapted from community discussion — TmData

    Next question

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

SPARK-DEV practice, question 161. 219 questions available.