SPARK-DEV practice questions 81–90

1. Question 81 · 219 in bank

   The code block shown below contains an error. The code block is intended to create and register a SQL UDF named "ASSESS_PERFORMANCE" using the Python function assessPerformance() and apply it to column customerSatistfaction in table stores. Identify the error. Code block: spark.udf.register("ASSESS_PERFORMANCE", assessPerformance) spark.sql("SELECT customerSatisfaction, assessPerformance(customerSatisfaction) AS result FROM stores")

   • AThere is no sql() operation — the DataFrame API must be used to apply the UDF assessPerformance().
   • BThe order of the arguments to spark.udf.register() should be reversed.
   • CThe customerSatisfaction column cannot be called twice inside the SQL statement.
   • DRegistered UDFs cannot be applied inside of a SQL statement.
   • EThe wrong SQL function is used to compute column result — it should be ASSESS_PERFORMANCE instead of assessPerformance.

   Show answer

   Answer: E

   Next question

2. Question 82 · 219 in bank

   Which of the following code blocks attempts to cache the partitions of DataFrame storesDF only in Spark’s memory?

   • AstoresDF.cache(StorageLevel.MEMORY_ONLY).count()
   • BstoresDF.persist().count()
   • CstoresDF.cache().count()
   • DstoresDF.persist(StorageLevel.MEMORY_ONLY).count()
   • EstoresDF.persist("MEMORY_ONLY").count()

   Show answer

   Answer: D

   Next question

3. Question 83 · 219 in bank

   Which of the following operations will always return a new DataFrame with updated partitions from DataFrame storesDF by inducing a shuffle?

   • AstoresDF.coalesce()
   • BstoresDF.rdd.getNumPartitions()
   • CstoresDF.repartition()
   • DstoresDF.union()
   • EstoresDF.intersect()

   Show answer

   Answer: C

   Next question

4. Question 84 · 219 in bank

   Which of the following code blocks returns a DataFrame containing a column month, an integer representation of the month from column openDate from DataFrame storesDF? Note that column openDate is of type integer and represents a date in the UNIX epoch format — the number of seconds since midnight on January 1 st, 1970. A sample of storesDF is displayed below:

   

   • AstoresDF.withColumn("month", getMonth(col("openDate")))
   • BstoresDF.withColumn("month", substr(col("openDate"), 4, 2))
   • C(storesDF.withColumn("openDateFormat", col("openDate").cast("Date")) .withColumn("month", month(col("openDateFormat"))))
   • D(storesDF.withColumn("openTimestamp", col("openDate").cast("Timestamp")) .withColumn("month", month(col("openTimestamp"))))
   • EstoresDF.withColumn("month", month(col("openDate")))

   Show answer

   Answer: D

   Next question

5. Question 85 · 219 in bank

   The code block shown below should read a parquet at the file path filePath into a DataFrame. Choose the response that correctly fills in the numbered blanks within the code block to complete this task. Code block: __1__.__2__.__3__(__4__)

   • A1.spark 2. read() 3. parquet 4. filePath
   • B1. spark 2. read() 3. load 4. filePath
   • C1. spark 2. read 3. load 4. filePath, source = "parquet"
   • D1. storesDF 2. read() 3. load 4. filePath
   • E1. spark 2. read 3. load 4. filePath

   Show answer

   Answer: E

   Next question

6. Question 86 · 219 in bank

   Which of the following statements describing a difference between transformations and actions is incorrect?

   • AThere are wide and narrow transformations but there are not wide and narrow actions.
   • BTransformations do not trigger execution while actions do trigger execution.
   • CTransformations work on DataFrames/Datasets while actions are reserved for native language objects.
   • DSome actions can be used to return data objects in a format native to the programming language being used to access the Spark API while transformations do not provide this ability.
   • ETransformations are typically logic operations while actions are typically focused on returning results.

   Show answer

   Answer: C

   Next question

7. Question 87 · 219 in bank

   Which of the following describes why garbage collection in Spark is important?

   • ALogical results will be incorrect if inaccurate data is not collected and removed from the Spark job.
   • BSpark jobs will fail or run slowly if inaccurate data is not collected and removed from the Spark job.
   • CSpark jobs will fail or run slowly if memory is not available for new objects to be created.
   • DSpark jobs will produce inaccurate results if there are too many different transformations called before a single action.
   • ESpark jobs will produce inaccurate results if memory is not available for new tasks to run and complete.

   Show answer

   Answer: C

   Next question

8. Question 88 · 219 in bank

   Which of the following operations can be used to create a DataFrame with a subset of columns from DataFrame storesDF that are specified by name?

   • AstoresDF.subset()
   • BstoresDF.select()
   • CstoresDF.selectColumn()
   • DstoresDF.filter()
   • EstoresDF.drop()

   Show answer

   Answer: B

   Explanation

   The operation that can be used to create a DataFrame with a subset of columns from DataFrame storesDF that are specified by name is storesDF.select(). The select() operation allows you to specify the columns you want to keep in the resulting DataFrame by passing in the column names as arguments. For example, to create a new DataFrame that contains only the columns store_id and store_name from the storesDF DataFrame, you can use the following code: newDF = storesDF.select("store_id", "store_name")

   Source: Adapted from community discussion — 4be8126

   Next question

9. Question 89 · 219 in bank

   Which of the following code blocks returns a DataFrame where rows in DataFrame storesDF containing missing values in every column have been dropped?

   • AstoresDF.na.drop()
   • BstoresDF.dropna()
   • CstoresDF.na.drop("all", subset = "sqft")
   • DstoresDF.na.drop("all")
   • EstoresDF.nadrop("all")

   Show answer

   Answer: D

   Next question

10. Question 90 · 219 in bank

    The code block shown below contains an error. The code block is intended to return a new DataFrame where column productCategories only has one word per row, resulting in a DataFrame with many more rows than DataFrame storesDF. Identify the error and how to fix it. A sample of storesDF is displayed below:

    

    storesDF.withColumn("productCategories", split(col("productCategories")))

    • AThe split() operation does not accomplish the requested task in the way that it is used. It should be used provided an alias.
    • BThe split() operation does not accomplish the requested task. The broadcast() operation should be used instead.
    • CThe split() operation does not accomplish the requested task in the way that it is used. It should be used as a column object method instead.
    • DThe split() operation does not accomplish the requested task. The explode() operation should be used instead.
    • EThe split() operation does not accomplish the requested task. The array_distinct() operation should be used instead.

    Show answer

    Answer: D

    Explanation

    Based on the condition "column productCategories only has one word per row, resulting in a DataFrame with many more rows than DataFrame storesDF" there should be an explode() function

    Source: Adapted from community discussion — Oks_An

    Next question

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

SPARK-DEV practice, question 81. 219 questions available.