Microsoft has introduced a suite of safety and security tools for generative AI within Azure AI Studio, its cloud-based platform for developing AI applications. These tools aim to enhance protection against prompt injection attacks, detect hallucinations in model output, provide system messages to guide models towards safe outputs, conduct model safety evaluations, and monitor risks and safety.

The announcement, made on March 28, underscores Microsoft’s commitment to bolstering the safety and security of AI systems. While safety evaluations are currently accessible in preview within Azure AI Studio, the additional features are slated for imminent release, according to Microsoft. Azure AI Studio, still in preview mode, is accessible through ai.azure.com.

One of the key tools introduced is the prompt shield, designed to identify and thwart prompt injection attacks, with an added capability to detect indirect prompt attacks preemptively. Currently available in preview within Azure AI Content Safety, this feature fortifies defenses against malicious injections that could compromise model integrity.

Groundness detection, another innovative addition, aims to discern text-based hallucinations, including subtle inaccuracies, within model outputs. By identifying “ungrounded material” in text, this feature bolsters the quality of Language Model (LLM) outputs, as emphasized by Microsoft.

Safety system messages, also referred to as metaprompts, play a crucial role in directing a model’s behavior towards outputs deemed safe and responsible. Meanwhile, safety evaluations gauge an application’s resilience against jailbreak attacks and content generation risks. In addition to assessing model quality metrics, these evaluations offer insights into content and security risks, enriching the assessment process.

Lastly, risk and safety monitoring functionality empower users to discern the triggers for content filters, thereby enabling informed mitigation strategies. Currently available in preview within Azure OpenAI Service, this feature aids in comprehending the factors influencing model inputs, outputs, and user interactions.

Microsoft’s proactive approach towards fortifying AI safety and security underscores the growing importance of ethical considerations in AI development and deployment. With these tools, developers leveraging Azure AI Studio can mitigate risks and ensure the responsible utilization of generative AI technologies.