Free Microsoft AI-300 Practice Test & Real Exam Questions
Drag and Drop Question
You develop a flow for a Microsoft Foundry project.
You plan to use outputs generated by running the flow to determine the following information:
- the number of tokens used by each large language model (LLM) node of
the flow
- the accuracy of the model used by the flow
You need to examine the output that provides the required information.
Which output type should you examine? To answer, move the appropriate output types to the correct evaluations. You may use each output type once, more than once, or not at all. You may need to move the split bar between panes or scroll to view content.
NOTE: Each correct selection is worth one point.
You develop a flow for a Microsoft Foundry project.
You plan to use outputs generated by running the flow to determine the following information:
- the number of tokens used by each large language model (LLM) node of
the flow
- the accuracy of the model used by the flow
You need to examine the output that provides the required information.
Which output type should you examine? To answer, move the appropriate output types to the correct evaluations. You may use each output type once, more than once, or not at all. You may need to move the split bar between panes or scroll to view content.
NOTE: Each correct selection is worth one point.
Correct Answer:
Explanation:
Box 1: Traces
The appropriate output flow type to determine the number of tokens used by each Large Language Model (LLM) node is Traces.
While Metrics provide a macro-level, aggregated overview of total token usage across an entire application or resource, Traces capture detailed, node-by-node execution details.
When a prompt flow runs in Microsoft Foundry, Traces track:
The precise execution path of individual items
Inputs and outputs for each specific LLM node
Detailed telemetry records-such as input_tokens, output_tokens, and total_tokens-bound to that exact step Box 2: Metrics The most appropriate output flow type to determine the accuracy of the model is Metrics.
Metrics are quantitative measurements (such as accuracy, F1-score, precision, recall, or mean squared error) specifically calculated by evaluating model predictions against ground truth data.
Reference:
https://huggingface.co/docs/evaluate/a_quick_tour
Hotspot Question
You manage a Microsoft Foundry project. You build a solution that uses a set of PDF documents.
You require two large language models (LLMs):
- An embedding model must help categorize the documents.
- A general-purpose model must generate semantically and contextually
accurate output based on the documents.
You need to select benchmarks to observe the quality of the models.
Which metrics should you use for the benchmarks? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.
You manage a Microsoft Foundry project. You build a solution that uses a set of PDF documents.
You require two large language models (LLMs):
- An embedding model must help categorize the documents.
- A general-purpose model must generate semantically and contextually
accurate output based on the documents.
You need to select benchmarks to observe the quality of the models.
Which metrics should you use for the benchmarks? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.
Correct Answer:
Explanation:
Box 1: Accuracy
Embedding model
An embedding model must help categorize the documents.
For an embedding model designed to categorize documents within Azure AI Foundry, cosine similarity is a crucial benchmark metric. It measures the semantic similarity between document embeddings, allowing you to assess how well the model groups related documents together.
Additionally, classification accuracy is important to evaluate how effectively the model assigns documents to predefined categories.
Classification Accuracy:
This metric measures the percentage of documents that are correctly categorized by the model. It directly reflects the model's ability to assign documents to the appropriate category based on their embeddings Box 2: Coherence General-purpose model A general-purpose model must generate semantically and contextually accurate output based on the documents.
Coherence evaluates how well the language model can produce output that flows smoothly, reads naturally, and resembles human-like language.
Incorrect:
* GPTsimilarity
GPT similarity refers to the ability of GPT models to assess and quantify the semantic similarity between pieces of text. This can be achieved by using the model to generate embeddings (numerical representations) of the text and then calculating the similarity (e.g., using cosine similarity) between these embeddings. Essentially, it allows you to determine how closely related two texts are in terms of their meaning.
Reference:
https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/model-benchmarks
https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/concept-model-monitoring-generative-ai-evaluation-metrics
Multiple teams need access to approved models with version tracking, lineage, and governance controls. Models must be discoverable and reusable across projects. What Azure ML feature should you use?
Correct Answer: A
Vote an answer
Explanation: Only visible for Pass4Leader members. You can sign-up / login (it's free).
Case Study 1 - Fabrikam Inc.
Background
Fabrikam Inc. is a mid-sized healthcare analytics company that provides population health dashboards and predictive insights to regional hospital systems across the United States.
Fabrikam Inc. customers rely on near real time analytics to monitor patient flow, staffing needs, and readmission risks. They use multiple traditional forecasting machine learning models for predictions.
Fabrikam Inc. has an established Microsoft Azure footprint. The company uses Jupyter Notebooks that run on a local server as the primary development environment. The data science team is experiencing scalability, asset management and code management issues with the current development platform. Fabrikam Inc. plans to migrate to a cloud-based development environment to mitigate the issues.
Additionally, the company plans to implement a Retrieval-Augmented Generation (RAG)-based chat application for client support. Leadership requires the application to be developed and deployed with a low operational risk.
Current Environment
Fabrikam Inc. operates a single Azure subscription that has the following components:
* Azure Data Lake Storage Gen2 that contains de-identified clinical and operational datasets
* Azure AI Search indexing curated analytical documents and reference materials
* A small set of Python-based training scripts maintained by data scientists
* Azure OpenAI Service with deployed foundational models
* A Microsoft Foundry resource for building a RAG-based solution
Evaluation data has manually defined expected responses.
The current challenges faced by the data science team include the following:
* Model training jobs are run manually from notebooks.
* Experiment tracking is inconsistent
* Model versions are registered without standardized metadata.
* Deployment is performed manually by data scientists, with limited rollback capability.
* The team has no standardized evaluation process for generative AI outputs.
The environment currently allows public network access. Authentication relies on user accounts rather than managed identities. Compute targets are manually created and shared across experiments. This has led to resource contention during peak usage.
Business Requirements
Fabrikam Inc. has the following business requirements for the modernization initiative:
* Provide a conversational interface that answers analytics questions by using internal documents and datasets.
* Ensure that sensitive healthcare-related data is not exposed outside the Fabrikam Inc. Azure tenant.
* Enable repeatable and auditable model training and deployment processes.
* Support experimentation to compare prompt strategies and fine-tuned models.
* Align the model with the ranked preferences and optimize behavior for the long term.
* Minimize disruption to existing analytics workloads during rollout.
Technical Requirements
To support the business goals, Fabrikam Inc. identifies these technical requirements:
* Use Azure Machine Learning workspaces to centrally manage data assets, models, and environments.
* Implement experiment tracking and model versioning for all training jobs.
* Orchestrate training and evaluation by using pipelines rather than manually running notebooks.
* Deploy traditional machine learning models with support for staged rollout and rollback.
* Improve RAG-based solution output quality.
* Use the existing evaluation datasets that are based on real data with input-output pairs.
* Apply advanced fine-tuning techniques only when prompt engineering is insufficient Issues and Constraints Fabrikam Inc. must comply with internal security policies that require the company to restrict network access and avoid long-lived secrets. The data science team has limited Azure DevOps experience, so solutions must favor managed services and automation over custom infrastructure.
Cost predictability is important. Leadership prefers serverless or managed compute options where possible but is willing to approve dedicated compute for stable production workloads.
Problem Statement
Fabrikam Inc. must design and implement an Azure-based AI operations solution that enables reliable training, evaluation, deployment, and iteration of generative AI models. The solution must support experimentation and gradual rollout while ensuring governance, security, and operational stability. The data science and platform teams must collaborate to deliver this solution by using Azure Machine Learning and Microsoft Foundry capabilities.
You need to recommend an experiment-tracking strategy that ensures consistent experiment results. What should you recommend?
Background
Fabrikam Inc. is a mid-sized healthcare analytics company that provides population health dashboards and predictive insights to regional hospital systems across the United States.
Fabrikam Inc. customers rely on near real time analytics to monitor patient flow, staffing needs, and readmission risks. They use multiple traditional forecasting machine learning models for predictions.
Fabrikam Inc. has an established Microsoft Azure footprint. The company uses Jupyter Notebooks that run on a local server as the primary development environment. The data science team is experiencing scalability, asset management and code management issues with the current development platform. Fabrikam Inc. plans to migrate to a cloud-based development environment to mitigate the issues.
Additionally, the company plans to implement a Retrieval-Augmented Generation (RAG)-based chat application for client support. Leadership requires the application to be developed and deployed with a low operational risk.
Current Environment
Fabrikam Inc. operates a single Azure subscription that has the following components:
* Azure Data Lake Storage Gen2 that contains de-identified clinical and operational datasets
* Azure AI Search indexing curated analytical documents and reference materials
* A small set of Python-based training scripts maintained by data scientists
* Azure OpenAI Service with deployed foundational models
* A Microsoft Foundry resource for building a RAG-based solution
Evaluation data has manually defined expected responses.
The current challenges faced by the data science team include the following:
* Model training jobs are run manually from notebooks.
* Experiment tracking is inconsistent
* Model versions are registered without standardized metadata.
* Deployment is performed manually by data scientists, with limited rollback capability.
* The team has no standardized evaluation process for generative AI outputs.
The environment currently allows public network access. Authentication relies on user accounts rather than managed identities. Compute targets are manually created and shared across experiments. This has led to resource contention during peak usage.
Business Requirements
Fabrikam Inc. has the following business requirements for the modernization initiative:
* Provide a conversational interface that answers analytics questions by using internal documents and datasets.
* Ensure that sensitive healthcare-related data is not exposed outside the Fabrikam Inc. Azure tenant.
* Enable repeatable and auditable model training and deployment processes.
* Support experimentation to compare prompt strategies and fine-tuned models.
* Align the model with the ranked preferences and optimize behavior for the long term.
* Minimize disruption to existing analytics workloads during rollout.
Technical Requirements
To support the business goals, Fabrikam Inc. identifies these technical requirements:
* Use Azure Machine Learning workspaces to centrally manage data assets, models, and environments.
* Implement experiment tracking and model versioning for all training jobs.
* Orchestrate training and evaluation by using pipelines rather than manually running notebooks.
* Deploy traditional machine learning models with support for staged rollout and rollback.
* Improve RAG-based solution output quality.
* Use the existing evaluation datasets that are based on real data with input-output pairs.
* Apply advanced fine-tuning techniques only when prompt engineering is insufficient Issues and Constraints Fabrikam Inc. must comply with internal security policies that require the company to restrict network access and avoid long-lived secrets. The data science team has limited Azure DevOps experience, so solutions must favor managed services and automation over custom infrastructure.
Cost predictability is important. Leadership prefers serverless or managed compute options where possible but is willing to approve dedicated compute for stable production workloads.
Problem Statement
Fabrikam Inc. must design and implement an Azure-based AI operations solution that enables reliable training, evaluation, deployment, and iteration of generative AI models. The solution must support experimentation and gradual rollout while ensuring governance, security, and operational stability. The data science and platform teams must collaborate to deliver this solution by using Azure Machine Learning and Microsoft Foundry capabilities.
You need to recommend an experiment-tracking strategy that ensures consistent experiment results. What should you recommend?
Correct Answer: D
Vote an answer
Explanation: Only visible for Pass4Leader members. You can sign-up / login (it's free).
Drag and Drop Question
A customer-facing web application uses a foundational model deployed through Microsoft Foundry.
A new model version must be introduced and validated without disrupting production traffic.
You need to deploy the new version by using a safe promotion strategy.
Which three actions should you perform in sequence? To answer, move the appropriate actions from the list of actions to the answer area and arrange them in the correct order.
A customer-facing web application uses a foundational model deployed through Microsoft Foundry.
A new model version must be introduced and validated without disrupting production traffic.
You need to deploy the new version by using a safe promotion strategy.
Which three actions should you perform in sequence? To answer, move the appropriate actions from the list of actions to the answer area and arrange them in the correct order.
Correct Answer:
Hotspot Question
A team manages an Azure Machine Learning workspace to train and register machine learning models.
Previous model versions must be retained for audit and rollback purposes but must not be used for new deployments.
You need to manage model versions.
What should you do for each requirement? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.
A team manages an Azure Machine Learning workspace to train and register machine learning models.
Previous model versions must be retained for audit and rollback purposes but must not be used for new deployments.
You need to manage model versions.
What should you do for each requirement? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.
Correct Answer:
A team develops multiple AI applications in Microsoft Foundry that rely on shared prompt templates.
The team requires a centralized way to track, version, and reuse prompt content across projects.
You need to recommend a solution to track and reuse prompt content.
Which approach should you recommend?
The team requires a centralized way to track, version, and reuse prompt content across projects.
You need to recommend a solution to track and reuse prompt content.
Which approach should you recommend?
Correct Answer: B
Vote an answer
Explanation: Only visible for Pass4Leader members. You can sign-up / login (it's free).
You manage an Azure Machine Learning workspace. You have an environment for training jobs which uses an existing Docker image.
A new version of the Docker image is available.
You need to use the latest version of the Docker image for the environment configuration by using the Azure Machine Learning SDK v2.
What should you do?
A new version of the Docker image is available.
You need to use the latest version of the Docker image for the environment configuration by using the Azure Machine Learning SDK v2.
What should you do?
Correct Answer: B
Vote an answer
Explanation: Only visible for Pass4Leader members. You can sign-up / login (it's free).
You deploy a new model version to a managed online endpoint. You must test it with 10% traffic and automatically roll back if latency or error rate increases beyond threshold. What should you configure?
Correct Answer: B
Vote an answer
Explanation: Only visible for Pass4Leader members. You can sign-up / login (it's free).
Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.
After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear on the review screen.
An organization provisions Azure Machine Learning workspaces for development, test, and production environments.
Each environment must be deployed consistently and updated through source control. The deployment process must be automated, repeatable, and auditable.
You need to deploy Azure Machine Learning resources in a consistent and controlled manner.
Solution: Define Azure Machine Learning resources in a Bicep template and deploy them within a GitHub Action.
Does the solution meet the goal?
After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear on the review screen.
An organization provisions Azure Machine Learning workspaces for development, test, and production environments.
Each environment must be deployed consistently and updated through source control. The deployment process must be automated, repeatable, and auditable.
You need to deploy Azure Machine Learning resources in a consistent and controlled manner.
Solution: Define Azure Machine Learning resources in a Bicep template and deploy them within a GitHub Action.
Does the solution meet the goal?
Correct Answer: B
Vote an answer
Explanation: Only visible for Pass4Leader members. You can sign-up / login (it's free).
