PMLE practice questions 181–190

1. Question 181 · 364 in bank

   You recently deployed a model to a Vertex AI endpoint and set up online serving in Vertex AI Feature Store. You have configured a daily batch ingestion job to update your featurestore. During the batch ingestion jobs, you discover that CPU utilization is high in your featurestore’s online serving nodes and that feature retrieval latency is high. You need to improve online serving performance during the daily batch ingestion. What should you do?

   • ASchedule an increase in the number of online serving nodes in your featurestore prior to the batch ingestion jobs
   • BEnable autoscaling of the online serving nodes in your featurestore
   • CEnable autoscaling for the prediction nodes of your DeployedModel in the Vertex AI endpoint
   • DIncrease the worker_count in the ImportFeatureValues request of your batch ingestion job

   Show answer

   Answer: B

   Explanation

   Option A: Manually scheduling node increases requires prior knowledge of batch ingestion times and might not be as responsive to unexpected workload spikes. Option C: Autoscaling prediction nodes in the Vertex AI endpoint might help with model prediction latency but doesn't directly address feature retrieval latency from the featurestore. Option D: Increasing worker_count in the batch ingestion job could speed up ingestion but might further strain online serving nodes, potentially worsening latency.

   Source: Adapted from community discussion — pikachu007

   References

   • Google Cloud documentation

   Next question

2. Question 182 · 364 in bank

   You are developing a custom TensorFlow classification model based on tabular data. Your raw data is stored in BigQuery. contains hundreds of millions of rows, and includes both categorical and numerical features. You need to use a MaxMin scaler on some numerical features, and apply a one-hot encoding to some categorical features such as SKU names. Your model will be trained over multiple epochs. You want to minimize the effort and cost of your solution. What should you do?

   • A1. Write a SQL query to create a separate lookup table to scale the numerical features. 2. Deploy a TensorFlow-based model from Hugging Face to BigQuery to encode the text features. 3. Feed the resulting BigQuery view into Vertex AI Training.
   • B1. Use BigQuery to scale the numerical features. 2. Feed the features into Vertex AI Training. 3. Allow TensorFlow to perform the one-hot text encoding.
   • C1. Use TFX components with Dataflow to encode the text features and scale the numerical features. 2. Export results to Cloud Storage as TFRecords. 3. Feed the data into Vertex AI Training.
   • D1. Write a SQL query to create a separate lookup table to scale the numerical features. 2. Perform the one-hot text encoding in BigQuery. 3. Feed the resulting BigQuery view into Vertex AI Training.

   Show answer

   Answer: C

   Explanation

   "Full-pass stateful transformations aren't suitable for implementation in BigQuery. If you use BigQuery for full-pass transformations, you need auxiliary tables to store quantities needed by stateful transformations, such as means and variances to scale numerical features. Further, implementation of full-pass transformations using SQL on BigQuery creates increased complexity in the SQL scripts, and creates intricate dependency between training and the scoring SQL scripts." https://www.tensorflow.org/tfx/guide/tft_bestpractices#where_to_do_preprocessing

   Source: Adapted from community discussion — b2aaace

   Next question

3. Question 183 · 364 in bank

   You work for a retail company. You have been tasked with building a model to determine the probability of churn for each customer. You need the predictions to be interpretable so the results can be used to develop marketing campaigns that target at-risk customers. What should you do?

   • ABuild a random forest regression model in a Vertex AI Workbench notebook instance. Configure the model to generate feature importances after the model is trained.
   • BBuild an AutoML tabular regression model. Configure the model to generate explanations when it makes predictions.
   • CBuild a custom TensorFlow neural network by using Vertex AI custom training. Configure the model to generate explanations when it makes predictions.
   • DBuild a random forest classification model in a Vertex AI Workbench notebook instance. Configure the model to generate feature importances after the model is trained.

   Show answer

   Answer: D

   Explanation

   We can't use AutoML due to the lack of explicability. AutoML is a black box, and we can't know which model is GCP using under the hood: Whether is true that you can use the feature importance tool when using AutoML, GCP doesn't publicly disclose the specific models used internally for each type of problem (classification, regression, etc.). AutoML employs a wide range of algorithms, from linear models and decision trees to more complex neural networks. Consequently, the lack of explicability lead us to discard any AutoML option. Regarding the classification/regression discussion, as Roulle says "Churn problems are cases of classification. We don't predict the label, but the probability of belonging to a given class (churn or not). We then set a threshold to indicate the probability at which we can affirm that the person will or will not unsubscribe."

   Source: Adapted from community discussion — tardigradum

   References

   • Google Cloud documentation

   Next question

4. Question 184 · 364 in bank

   You work for a company that is developing an application to help users with meal planning. You want to use machine learning to scan a corpus of recipes and extract each ingredient (e.g., carrot, rice, pasta) and each kitchen cookware (e.g., bowl, pot, spoon) mentioned. Each recipe is saved in an unstructured text file. What should you do?

   • ACreate a text dataset on Vertex AI for entity extraction Create two entities called “ingredient” and “cookware”, and label at least 200 examples of each entity. Train an AutoML entity extraction model to extract occurrences of these entity types. Evaluate performance on a holdout dataset.
   • BCreate a multi-label text classification dataset on Vertex AI. Create a test dataset, and label each recipe that corresponds to its ingredients and cookware. Train a multi-class classification model. Evaluate the model’s performance on a holdout dataset.
   • CUse the Entity Analysis method of the Natural Language API to extract the ingredients and cookware from each recipe. Evaluate the model's performance on a prelabeled dataset.
   • DCreate a text dataset on Vertex AI for entity extraction. Create as many entities as there are different ingredients and cookware. Train an AutoML entity extraction model to extract those entities. Evaluate the model’s performance on a holdout dataset.

   Show answer

   Answer: A

   Explanation

   A. "... you might create an entity extraction model to identify specialized terminology in legal documents or patents." I prefer this over C, which might classify carrot as vegetable, chicken as meat... custom entity extraction allows you to specify what entities you wish to extract from the text.

   Source: Adapted from community discussion — b1a8fae

   References

   • Google Cloud documentation

   Next question

5. Question 185 · 364 in bank

   You work for an organization that operates a streaming music service. You have a custom production model that is serving a “next song” recommendation based on a user's recent listening history. Your model is deployed on a Vertex AI endpoint. You recently retrained the same model by using fresh data. The model received positive test results offline. You now want to test the new model in production while minimizing complexity. What should you do?

   • ACreate a new Vertex AI endpoint for the new model and deploy the new model to that new endpoint. Build a service to randomly send 5% of production traffic to the new endpoint. Monitor end-user metrics such as listening time. If end-user metrics improve between models over time, gradually increase the percentage of production traffic sent to the new endpoint.
   • BCapture incoming prediction requests in BigQuery. Create an experiment in Vertex AI Experiments. Run batch predictions for both models using the captured data. Use the user’s selected song to compare the models performance side by side. If the new model’s performance metrics are better than the previous model, deploy the new model to production.
   • CDeploy the new model to the existing Vertex AI endpoint. Use traffic splitting to send 5% of production traffic to the new model. Monitor end-user metrics, such as listening time. If end-user metrics improve between models over time, gradually increase the percentage of production traffic sent to the new model.
   • DConfigure a model monitoring job for the existing Vertex AI endpoint. Configure the monitoring job to detect prediction drift and set a threshold for alerts. Update the model on the endpoint from the previous model to the new model. If you receive an alert of prediction drift, revert to the previous model.

   Show answer

   Answer: C

   Explanation

   Here's why the option C is preferable: Minimized complexity: Leverages existing endpoint: No need to create and manage a new endpoint, reducing setup and maintenance overhead. Traffic splitting readily available: Vertex AI provides built-in traffic splitting functionality, simplifying traffic distribution. Efficient testing and monitoring: Direct comparison: Sending a percentage of traffic to the new model allows for direct comparison with the current model's performance on real user data. Gradual rollout: Starting with a small percentage mitigates potential risks and allows for gradual transition based on observed improvements. End-user metric monitoring: Focusing on metrics like listening time directly reflects user engagement and preference for the new recommendations.

   Source: Adapted from community discussion — daidai75

   Next question

6. Question 186 · 364 in bank

   You created a model that uses BigQuery ML to perform linear regression. You need to retrain the model on the cumulative data collected every week. You want to minimize the development effort and the scheduling cost. What should you do?

   • AUse BigQuery’s scheduling service to run the model retraining query periodically.
   • BCreate a pipeline in Vertex AI Pipelines that executes the retraining query, and use the Cloud Scheduler API to run the query weekly.
   • CUse Cloud Scheduler to trigger a Cloud Function every week that runs the query for retraining the model.
   • DUse the BigQuery API Connector and Cloud Scheduler to trigger Workflows every week that retrains the model.

   Show answer

   Answer: A

   Explanation

   No additional setup: BigQuery's scheduling feature is built-in, eliminating the need to create pipelines, functions, or workflows. Straightforward configuration: Setting up a schedule for a query is a simple process within the BigQuery interface.

   Source: Adapted from community discussion — daidai75

   Next question

7. Question 187 · 364 in bank

   You want to migrate a scikit-learn classifier model to TensorFlow. You plan to train the TensorFlow classifier model using the same training set that was used to train the scikit-learn model, and then compare the performances using a common test set. You want to use the Vertex AI Python SDK to manually log the evaluation metrics of each model and compare them based on their F1 scores and confusion matrices. How should you log the metrics?

   • AUse the aiplatform.log_classification_metrics function to log the F1 score, and use the aiplatform.log_metrics function to log the confusion matrix.
   • BUse the aiplatform.log_classification_metrics function to log the F1 score and the confusion matrix.
   • CUse the aiplatform.log_metrics function to log the F1 score and the confusion matrix.
   • DUse the aiplatform.log_metrics function to log the F1 score: and use the aiplatform.log_classification_metrics function to log the confusion matrix.

   Show answer

   Answer: D

   Explanation

   I go with D. log_classification_metrics currently support confusion matrix and ROC curve. https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform#google_cloud_aiplatform_log_classification_metrics Because it is not explicitly mentioned in the docs of log_classification_metrics, I assume F1 Score must be logged with log_metrics. https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform#google_cloud_aiplatform_log_metrics (if accuracy and recall are logged in the example, probably F1 is done the same way)

   Source: Adapted from community discussion — b1a8fae

   References

   • Google Cloud documentation
   • Google Cloud documentation

   Next question

8. Question 188 · 364 in bank

   You are developing a model to help your company create more targeted online advertising campaigns. You need to create a dataset that you will use to train the model. You want to avoid creating or reinforcing unfair bias in the model. What should you do? (Choose two.)

   • AInclude a comprehensive set of demographic features
   • BInclude only the demographic groups that most frequently interact with advertisements
   • CCollect a random sample of production traffic to build the training dataset
   • DCollect a stratified sample of production traffic to build the training dataset
   • EConduct fairness tests across sensitive categories and demographics on the trained model

   Show answer

   Answer: D

   Explanation

   D E. ChatGPT explanation below (but I think makes quite a lot of sense) Collect a Stratified Sample (Option D): Stratified sampling involves dividing the population into subgroups (strata) and then randomly sampling from each subgroup. This ensures that the training dataset represents the diversity of the population, helping to avoid biases. By collecting a stratified sample of production traffic, you are more likely to have a balanced representation of different demographic groups, reducing the risk of biased model outcomes. Conduct Fairness Tests (Option E): After training the model, it's crucial to conduct fairness tests to evaluate its performance across different sensitive categories and demographics. This involves measuring the model's predictions and outcomes for various groups to identify any disparities. Fairness tests help you assess and address biases that may have been inadvertently introduced during the training process.

   Source: Adapted from community discussion — b1a8fae

   Next question

9. Question 189 · 364 in bank

   You are an ML engineer at a regulated insurance company. You are asked to develop an insurance approval model that accepts or rejects insurance applications from potential customers. What factors should you consider before building the model?

   • ARedaction, reproducibility, and explainability
   • BTraceability, reproducibility, and explainability
   • CFederated learning, reproducibility, and explainability
   • DDifferential privacy, federated learning, and explainability

   Show answer

   Answer: B

   Explanation

   I think the answer should be B. as I review the OECD document on impact of AI on insurance, the document mention explainability, traceable. However, open for discussion. https://www.oecd.org/finance/Impact-Big-Data-AI-in-the-Insurance-Sector.pdf

   Source: Adapted from community discussion — gcp2021go

   Next question

10. Question 190 · 364 in bank

    You are developing an ML model in a Vertex AI Workbench notebook. You want to track artifacts and compare models during experimentation using different approaches. You need to rapidly and easily transition successful experiments to production as you iterate on your model implementation. What should you do?

    • A1. Initialize the Vertex SDK with the name of your experiment. Log parameters and metrics for each experiment, and attach dataset and model artifacts as inputs and outputs to each execution. 2. After a successful experiment create a Vertex AI pipeline.
    • B1. Initialize the Vertex SDK with the name of your experiment. Log parameters and metrics for each experiment, save your dataset to a Cloud Storage bucket, and upload the models to Vertex AI Model Registry. 2. After a successful experiment, create a Vertex AI pipeline.
    • C1. Create a Vertex AI pipeline with parameters you want to track as arguments to your PipelineJob. Use the Metrics, Model, and Dataset artifact types from the Kubeflow Pipelines DSL as the inputs and outputs of the components in your pipeline. 2. Associate the pipeline with your experiment when you submit the job.
    • D1. Create a Vertex AI pipeline. Use the Dataset and Model artifact types from the Kubeflow Pipelines DSL as the inputs and outputs of the components in your pipeline. 2. In your training component, use the Vertex AI SDK to create an experiment run. Configure the log_params and log_metrics functions to track parameters and metrics of your experiment.

    Show answer

    Answer: A

    Explanation

    Option B: Manually saving datasets and models to Cloud Storage and Model Registry introduces extra steps and potential for inconsistencies. Options C and D: Prioritizing pipeline creation limits flexibility and visibility during the experimentation phase, making it harder to track artifacts and compare models effectively.

    Source: Adapted from community discussion — pikachu007

    Next question

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

PMLE practice, question 181. 364 questions available.