PDE practice questions 171–180

1. Question 171 · 340 in bank

   You are running a Dataflow streaming pipeline, with Streaming Engine and Horizontal Autoscaling enabled. You have set the maximum number of workers to 1000. The input of your pipeline is Pub/Sub messages with notifications from Cloud Storage. One of the pipeline transforms reads CSV files and emits an element for every CSV line. The job performance is low, the pipeline is using only 10 workers, and you notice that the autoscaler is not spinning up additional workers. What should you do to improve performance?

   • AEnable Vertical Autoscaling to let the pipeline use larger workers.
   • BChange the pipeline code, and introduce a Reshuffle step to prevent fusion.
   • CUpdate the job to increase the maximum number of workers.
   • DUse Dataflow Prime, and enable Right Fitting to increase the worker resources.

   Show answer

   Answer: B

   Explanation

   - Fusion optimization in Dataflow can lead to steps being "fused" together, which can sometimes hinder parallelization. - Introducing a Reshuffle step can prevent fusion and force the distribution of work across more workers. - This can be an effective way to improve parallelism and potentially trigger the autoscaler to increase the number of workers.

   Source: Adapted from community discussion — raaad

   References

   • Google Cloud documentation
   • Google Cloud documentation

   Next question

2. Question 172 · 340 in bank

   You have an Oracle database deployed in a VM as part of a Virtual Private Cloud (VPC) network. You want to replicate and continuously synchronize 50 tables to BigQuery. You want to minimize the need to manage infrastructure. What should you do?

   • ADeploy Apache Kafka in the same VPC network, use Kafka Connect Oracle Change Data Capture (CDC), and Dataflow to stream the Kafka topic to BigQuery.
   • BCreate a Pub/Sub subscription to write to BigQuery directly. Deploy the Debezium Oracle connector to capture changes in the Oracle database, and sink to the Pub/Sub topic.
   • CDeploy Apache Kafka in the same VPC network, use Kafka Connect Oracle change data capture (CDC), and the Kafka Connect Google BigQuery Sink Connector.
   • DCreate a Datastream service from Oracle to BigQuery, use a private connectivity configuration to the same VPC network, and a connection profile to BigQuery.

   Show answer

   Answer: D

   Explanation

   - Datastream is a serverless and easy-to-use change data capture (CDC) and replication service. - You would create a Datastream service that sources from your Oracle database and targets BigQuery, with private connectivity configuration to the same VPC. - This option is designed to minimize the need to manage infrastructure and is a fully managed service.

   Source: Adapted from community discussion — raaad

   Next question

3. Question 173 · 340 in bank

   You are deploying an Apache Airflow directed acyclic graph (DAG) in a Cloud Composer 2 instance. You have incoming files in a Cloud Storage bucket that the DAG processes, one file at a time. The Cloud Composer instance is deployed in a subnetwork with no Internet access. Instead of running the DAG based on a schedule, you want to run the DAG in a reactive way every time a new file is received. What should you do?

   • A1. Enable Private Google Access in the subnetwork, and set up Cloud Storage notifications to a Pub/Sub topic. 2. Create a push subscription that points to the web server URL.
   • B1. Enable the Cloud Composer API, and set up Cloud Storage notifications to trigger a Cloud Function. 2. Write a Cloud Function instance to call the DAG by using the Cloud Composer API and the web server URL. 3. Use VPC Serverless Access to reach the web server URL.
   • C1. Enable the Airflow REST API, and set up Cloud Storage notifications to trigger a Cloud Function instance. 2. Create a Private Service Connect (PSC) endpoint. 3. Write a Cloud Function that connects to the Cloud Composer cluster through the PSC endpoint.
   • D1. Enable the Airflow REST API, and set up Cloud Storage notifications to trigger a Cloud Function instance. 2. Write a Cloud Function instance to call the DAG by using the Airflow REST API and the web server URL. 3. Use VPC Serverless Access to reach the web server URL.

   Show answer

   Answer: C

   Explanation

   - Enable Airflow REST API: In Cloud Composer, enable the "Airflow web server" option. - Set Up Cloud Storage Notifications: Create a notification for new files, routing to a Cloud Function. - Create PSC Endpoint: Establish a PSC endpoint for Cloud Composer. - Write Cloud Function: Code the function to use the Airflow REST API (via PSC endpoint) to trigger the DAG. ======== Why not Option D - Using the web server URL directly wouldn't work without internet access or a direct path to the web server.

   Source: Adapted from community discussion — raaad

   References

   • Google Cloud documentation
   • Google Cloud documentation

   Next question

4. Question 174 · 340 in bank

   You are planning to use Cloud Storage as part of your data lake solution. The Cloud Storage bucket will contain objects ingested from external systems. Each object will be ingested once, and the access patterns of individual objects will be random. You want to minimize the cost of storing and retrieving these objects. You want to ensure that any cost optimization efforts are transparent to the users and applications. What should you do?

   • ACreate a Cloud Storage bucket with Autoclass enabled.
   • BCreate a Cloud Storage bucket with an Object Lifecycle Management policy to transition objects from Standard to Coldline storage class if an object age reaches 30 days.
   • CCreate a Cloud Storage bucket with an Object Lifecycle Management policy to transition objects from Standard to Coldline storage class if an object is not live.
   • DCreate two Cloud Storage buckets. Use the Standard storage class for the first bucket, and use the Coldline storage class for the second bucket. Migrate objects from the first bucket to the second bucket after 30 days.

   Show answer

   Answer: A

   Explanation

   - Autoclass automatically analyzes access patterns of objects and automatically transitions them to the most cost-effective storage class within Standard, Nearline, Coldline, or Archive. - This eliminates the need for manual intervention or setting specific age thresholds. - No user or application interaction is required, ensuring transparency.

   Source: Adapted from community discussion — raaad

   Next question

5. Question 175 · 340 in bank

   You have several different file type data sources, such as Apache Parquet and CSV. You want to store the data in Cloud Storage. You need to set up an object sink for your data that allows you to use your own encryption keys. You want to use a GUI-based solution. What should you do?

   • AUse Storage Transfer Service to move files into Cloud Storage.
   • BUse Cloud Data Fusion to move files into Cloud Storage.
   • CUse Dataflow to move files into Cloud Storage.
   • DUse BigQuery Data Transfer Service to move files into BigQuery.

   Show answer

   Answer: B

   Explanation

   - Cloud Data Fusion is a fully managed, code-free, GUI-based data integration service that allows you to visually connect, transform, and move data between various sources and sinks. - It supports various file formats and can write to Cloud Storage. - You can configure it to use Customer-Managed Encryption Keys (CMEK) for the buckets where it writes data.

   Source: Adapted from community discussion — raaad

   References

   • Google Cloud documentation
   • Google Cloud documentation

   Next question

6. Question 176 · 340 in bank

   Your business users need a way to clean and prepare data before using the data for analysis. Your business users are less technically savvy and prefer to work with graphical user interfaces to define their transformations. After the data has been transformed, the business users want to perform their analysis directly in a spreadsheet. You need to recommend a solution that they can use. What should you do?

   • AUse Dataprep to clean the data, and write the results to BigQuery. Analyze the data by using Connected Sheets.
   • BUse Dataprep to clean the data, and write the results to BigQuery. Analyze the data by using Looker Studio.
   • CUse Dataflow to clean the data, and write the results to BigQuery. Analyze the data by using Connected Sheets.
   • DUse Dataflow to clean the data, and write the results to BigQuery. Analyze the data by using Looker Studio.

   Show answer

   Answer: A

   References

   • Google Cloud documentation
   • Google Cloud documentation

   Next question

7. Question 177 · 340 in bank

   You are working on a sensitive project involving private user data. You have set up a project on Google Cloud Platform to house your work internally. An external consultant is going to assist with coding a complex transformation in a Google Cloud Dataflow pipeline for your project. How should you maintain users' privacy?

   • AGrant the consultant the Viewer role on the project.
   • BGrant the consultant the Cloud Dataflow Developer role on the project.
   • CCreate a service account and allow the consultant to log on with it.
   • DCreate an anonymized sample of the data for the consultant to work with in a different project.

   Show answer

   Answer: D

   Explanation

   I chose D. I wouldn't want to be a consultant who only gets the permissions in B: *This would not allow you to run the pipeline locally, *This would not allow you to verify that the data you're producing is in any way correct. Security-wise, you're also not helping yourself much by choosing B, as the consultant could use the permissions granted to the workers he's controlling to direct the data to a place where he can read it. An anonymized data set put in a separate project solves all of these issues. To reinforce the last point: GCP has recently recognized this by requiring you to to be able to impersonate the controller service account (compute engine default service account by default) if you want to be able to submit jobs. Dataflow Developer alone just won't allow the consultant to run jobs at all.

   Source: Adapted from community discussion — IKGx1iGetOWGSjAQDD2x3

   References

   • Google Cloud documentation
   • Google Cloud documentation
   • Google Cloud documentation
   • Google Cloud documentation
   • Google Cloud documentation

   Next question

8. Question 178 · 340 in bank

   You have two projects where you run BigQuery jobs: • One project runs production jobs that have strict completion time SLAs. These are high priority jobs that must have the required compute resources available when needed. These jobs generally never go below a 300 slot utilization, but occasionally spike up an additional 500 slots. • The other project is for users to run ad-hoc analytical queries. This project generally never uses more than 200 slots at a time. You want these ad-hoc queries to be billed based on how much data users scan rather than by slot capacity. You need to ensure that both projects have the appropriate compute resources available. What should you do?

   • ACreate a single Enterprise Edition reservation for both projects. Set a baseline of 300 slots. Enable autoscaling up to 700 slots.
   • BCreate two reservations, one for each of the projects. For the SLA project, use an Enterprise Edition with a baseline of 300 slots and enable autoscaling up to 500 slots. For the ad-hoc project, configure on-demand billing.
   • CCreate two Enterprise Edition reservations, one for each of the projects. For the SLA project, set a baseline of 300 slots and enable autoscaling up to 500 slots. For the ad-hoc project, set a reservation baseline of 0 slots and set the ignore idle slots flag to False.
   • DCreate two Enterprise Edition reservations, one for each of the projects. For the SLA project, set a baseline of 800 slots. For the ad-hoc project, enable autoscaling up to 200 slots.

   Show answer

   Answer: B

   Explanation

   - The SLA project gets a dedicated reservation with autoscaling to handle spikes, ensuring it meets its strict completion time SLAs. - The ad-hoc project uses on-demand billing, which means it will be billed based on the amount of data scanned rather than slot capacity, fitting the billing preference for ad-hoc queries.

   Source: Adapted from community discussion — raaad

   References

   • Google Cloud documentation

   Next question

9. Question 179 · 340 in bank

   You want to migrate your existing Teradata data warehouse to BigQuery. You want to move the historical data to BigQuery by using the most efficient method that requires the least amount of programming, but local storage space on your existing data warehouse is limited. What should you do?

   • AUse BigQuery Data Transfer Service by using the Java Database Connectivity (JDBC) driver with FastExport connection.
   • BCreate a Teradata Parallel Transporter (TPT) export script to export the historical data, and import to BigQuery by using the bq command-line tool.
   • CUse BigQuery Data Transfer Service with the Teradata Parallel Transporter (TPT) tbuild utility.
   • DCreate a script to export the historical data, and upload in batches to Cloud Storage. Set up a BigQuery Data Transfer Service instance from Cloud Storage to BigQuery.

   Show answer

   Answer: A

   Explanation

   - Reduced Local Storage: By using FastExport, data is directly streamed from Teradata to BigQuery without the need for local storage, addressing your storage limitations. - Minimal Programming: BigQuery Data Transfer Service offers a user-friendly interface, eliminating the need for extensive scripting or coding.

   Source: Adapted from community discussion — raaad

   References

   • Google Cloud documentation

   Next question

10. Question 180 · 340 in bank

    You are on the data governance team and are implementing security requirements. You need to encrypt all your data in BigQuery by using an encryption key managed by your team. You must implement a mechanism to generate and store encryption material only on your on-premises hardware security module (HSM). You want to rely on Google managed solutions. What should you do?

    • ACreate the encryption key in the on-premises HSM, and import it into a Cloud Key Management Service (Cloud KMS) key. Associate the created Cloud KMS key while creating the BigQuery resources.
    • BCreate the encryption key in the on-premises HSM and link it to a Cloud External Key Manager (Cloud EKM) key. Associate the created Cloud KMS key while creating the BigQuery resources.
    • CCreate the encryption key in the on-premises HSM, and import it into Cloud Key Management Service (Cloud HSM) key. Associate the created Cloud HSM key while creating the BigQuery resources.
    • DCreate the encryption key in the on-premises HSM. Create BigQuery resources and encrypt data while ingesting them into BigQuery.

    Show answer

    Answer: B

    Explanation

    - Cloud EKM allows you to use encryption keys managed in external key management systems, including on-premises HSMs, while using Google Cloud services. - This means that the key material remains in your control and environment, and Google Cloud services use it via the Cloud EKM integration. - This approach aligns with the need to generate and store encryption material only on your on-premises HSM and is the correct way to integrate such keys with BigQuery. ====== Why not Option C - Cloud HSM is a fully managed service by Google Cloud that provides HSMs for your cryptographic needs. However, it's a cloud-based solution, and the keys generated or managed in Cloud HSM are not stored on-premises. This option doesn't align with the requirement to use only on-premises HSM for key storage.

    Source: Adapted from community discussion — raaad

    References

    • Google Cloud documentation

    Next question

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

PDE practice, question 171. 340 questions available.