GENAI-ASSOCIATE practice questions 41–50

1. Question 41 · 92 in bank

   A Generative AI Engineer has just deployed an LLM application at a manufacturing company that assists with answering customer service inquiries. They need to identity the key enterprise metrics to monitor the application in production. Which is NOT a metric they will implement for their customer service LLM application in production?

   • AMassive Multi-task Language Understanding (MMLU) score
   • BNumber of customer inquiries processed per unit of time
   • CFactual accuracy of the response
   • DTime taken for LLM to generate a response

   Show answer

   Answer: A

   Next question

2. Question 42 · 92 in bank

   A Generative AI Engineer wants their finetuned LLMs in their prod Databricks workspace available for testing in their dev workspace as well. All of their workspaces are Unity Catalog enabled and they are currently logging their models into the Model Registry in MLflow. What is the most cost-effective and secure option for the Generative AI Engineer to accomplish their goal?

   • AUse an external model registry which can be accessed from all workspaces.
   • BUse MLflow to log the model directly into Unity Catalog, and enable READ access in the dev workspace to the model.
   • CSetup a duplicate training pipeline in dev, so that an identical model is available in dev.
   • DSetup a script to export the model from prod and import it to dev.

   Show answer

   Answer: B

   Next question

3. Question 43 · 92 in bank

   Generative AI Engineer needs to build an LLM application that can understand medical documents, including recently published ones. They want to select an open model available on HuggingFace’s model hub. Which step is most appropriate for selecting an LLM?

   • APick any model in the Mistral family, as Mistral models are good with all types of use cases
   • BSelect a model based on the highest number of downloads, as this indicates popularity, reliability, and general suitability
   • CSelect a model that is most recently uploaded, as this indicates the model is the newest and highly likely to be the most performant
   • DCheck for the model and training data description to identify if the model is trained on any medical data.

   Show answer

   Answer: D

   Next question

4. Question 44 · 92 in bank

   A Generative AI Engineer at an automotive company would like to build a question-answering chatbot for customers to inquire about their vehicles. They have a database containing various documents of different vehicle makes, their hardware parts, and common maintenance information. Which of the following components will NOT be useful in building such a chatbot?

   • AInvite users to submit long, rather than concise, questions
   • BResponse-generating LLM
   • CEmbedding model
   • DVector database

   Show answer

   Answer: A

   Next question

5. Question 45 · 92 in bank

   Select 2

   Which TWO chain components are required for building a basic LLM-enabled chat application that includes conversational capabilities, knowledge retrieval, and contextual memory? (Choose two.)

   • AVector Stores
   • BConversation Buffer Memory
   • CExternal tools
   • DChat loaders
   • EReact Components

   Show answer

   Answer: AB

   Next question

6. Question 46 · 92 in bank

   A Generative AI Engineer needs to design an LLM pipeline to conduct multi-stage reasoning that leverages external tools. To be effective at this, the LLM will need to plan and adapt actions while performing complex reasoning tasks. Which approach will do this?

   • ATrain the LLM to generate a single, comprehensive response without interacting with any external tools, relying solely on its pre-trained knowledge.
   • BUse a Chain-of-Thought (CoT) prompting technique to guide the LLM through a series of reasoning steps, then manually input the results from external tools for the final answer.
   • CImplement a framework like ReAct, which allows the LLM to generate reasoning traces and perform task-specific actions that leverage external tools if necessary.
   • DEncourage the LLM to make multiple API calls in sequence without planning or structuring the calls, allowing the LLM to decide when and how to use external tools spontaneously.

   Show answer

   Answer: C

   Next question

7. Question 47 · 92 in bank

   A team wants to serve a code generation model as an assistant for their software developers. It should support multiple programming languages. Quality is the primary objective. Which of the Databricks Foundation Model APIs, or models available in the Marketplace, would be the best fit?

   • ALlama2-70b
   • BBGE-large
   • CMPT-7b
   • DCodeLlama-34B

   Show answer

   Answer: D

   Next question

8. Question 48 · 92 in bank

   A Generative AI Engineer is building an LLM-based application that has an important transcription (speech-to-text) task. Speed is essential for the success of the application. Which open Generative AI models should be used?

   • ADBRX
   • BMPT-30B-Instruct
   • CLlama-2-70b-chat-hf
   • Dwhisper-large-v3 (1.6B)

   Show answer

   Answer: D

   Next question

9. Question 49 · 92 in bank

   A Generative AI Engineer is setting up a Databricks Vector Search that will lookup news articles by topic within 10 days of the date specified. An example query might be “Tell me about monster truck news around January 5th 1992”. They want to do this with the least amount of effort. How can they set up their Vector Search index to support this use case?

   • ACreate separate indexes by topic and add a classifier model to appropriately pick the best index.
   • BInclude metadata columns for article date and topic to support metadata filtering.
   • CPass the query directly to the vector search index and return the best articles.
   • DSplit articles by 10 day blocks and return the block closest to the query.

   Show answer

   Answer: B

   Next question

10. Question 50 · 92 in bank

    A Generative AI Engineer has created a RAG application which can help employees retrieve answers from an internal knowledge base, such as Confluence pages or Google Drive. The prototype application is now working with some positive feedback from internal company testers. Now the Generative Al Engineer wants to formally evaluate the system’s performance and understand where to focus their efforts to further improve the system. How should the Generative AI Engineer evaluate the system?

    • AUse cosine similarity score to comprehensively evaluate the quality of the final generated answers.
    • BCurate a dataset that can test the retrieval and generation components of the system separately. Use MLflow’s built in evaluation metrics to perform the evaluation on the retrieval and generation components.
    • CBenchmark multiple LLMs with the same data and pick the best LLM for the job.
    • DUse an LLM-as-a-judge to evaluate the quality of the final answers generated.

    Show answer

    Answer: B

    Next question

Keyboard:J Kmove between questionsA–Dselect an answerRshow or hide the answer← →change page

GENAI-ASSOCIATE practice, question 41. 92 questions available.