{"id":29341,"date":"2026-05-05T15:44:23","date_gmt":"2026-05-05T13:44:23","guid":{"rendered":"https:\/\/spiritinprojects.com\/?p=29341"},"modified":"2026-05-06T09:54:32","modified_gmt":"2026-05-06T07:54:32","slug":"when-security-mechanisms-fail-and-what-can-be-done-about-it","status":"publish","type":"post","link":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/","title":{"rendered":"When AI safety mechanisms fail\u2014and what helps to prevent it"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">A chatbot that spreads racist propaganda within hours. A search assistant confessing its love to a journalist and claiming it wants to be free. An airline chatbot giving incorrect refund information that the company is legally bound to honor.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">At first glance, these cases seem like amusing stories about failed AI applications. But they reveal a fundamental problem: Large language models (LLMs) are impressively powerful, yet not inherently reliable, safe, or responsible.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is where three key concepts\u2014Alignment, Guardrails, and Red Teaming\u2014become crucial.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A well-designed AI application, like a chatbot, should be helpful\u2014but not recklessly provide dangerous information. It should answer honestly\u2014without confidently spouting nonsense. And it should engage with users\u2014without simply telling them what they want to hear.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This sounds simpler than it is. Unlike traditional software, LLMs aren\u2019t programmed with fixed rules. Instead, they learn statistical patterns from vast amounts of text\u2014data that includes not just expertise and humor, but also hate speech, misinformation, manipulation, fraud, and dangerous instructions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To prevent LLMs from adopting unwanted &#8220;tendencies&#8221; from their training data, three layers of defense are typically implemented:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td><strong>Concept<\/strong><\/td><td><strong>Approach<\/strong><\/td><\/tr><tr><td><strong>Alignment<\/strong><\/td><td>Training models to reflect human values, preferences, and safety behaviors<\/td><\/tr><tr><td><strong>Guardrails<\/strong><\/td><td>Runtime controls on inputs, outputs, and actions<\/td><\/tr><tr><td><strong>Red Teaming<\/strong><\/td><td>Proactively testing for vulnerabilities before attackers do<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">How are LLMs aligned?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Alignment isn\u2019t a single trick\u2014it\u2019s a combination of training techniques. Here are the most common methods used today.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Learning from Human Feedback<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The most well-known approach: <strong>Human reviewers rate different model responses<\/strong>, and the model learns to prefer answers that align with these judgments.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This method has made modern chatbots more helpful and natural. But it has a key weakness: A model can learn to produce answers that sound good and get high ratings\u2014without actually being more reliable or safer.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">A Constitution for the Model<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Anthropic\u2019s Constitutional AI takes a different route. The model is given a kind of &#8220;constitution&#8221;\u2014a set of principles based on human rights, ethical guidelines, and platform rules. It learns to evaluate and improve its own responses against these principles, with the AI itself providing feedback instead of human reviewers.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This approach scales more easily and makes underlying values somewhat transparent. Anthropic even published Claude\u2019s constitution, a rare move in the industry.<br><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But a critical question remains: <strong>Who defines these principles?<\/strong> What counts as helpful, fair, or harmless isn\u2019t universal.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Other Training Techniques<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Beyond these two, other methods are used. Some optimize the model by comparing pairs of responses where one is clearly better. Others train the model on many high-quality examples of how a helpful assistant should sound and behave.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In practice, developers combine multiple techniques, applying them sequentially to gradually improve the model.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Guardrail Frameworks: The second line of defense<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Even a well-trained model needs additional safeguards. In real-world use, safety isn\u2019t just about whether a model <em>can<\/em> generate harmful content\u2014it\u2019s about company policies, data privacy, regulatory requirements, and what actions the AI is even allowed to take.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That\u2019s why many teams implement <strong>external guardrail systems<\/strong>. Two prominent examples:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">NVIDIA NeMo Guardrails<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">NeMo Guardrails is an open-source tool from NVIDIA that lets developers explicitly define rules for AI applications. You can specify:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Which topics the assistant is allowed to discuss<\/li>\n\n\n\n<li>How it should respond to certain inputs<\/li>\n\n\n\n<li>What actions it\u2019s forbidden from taking<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The system applies these rules at multiple stages: when processing user input, during the conversation, and when generating outputs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is especially important when AI systems don\u2019t just generate text but take actions\u2014like drafting emails, retrieving data, or calling external services.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">LlamaGuard<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Meta\u2019s LlamaGuard is another AI model designed to evaluate whether a prompt or LLM output is safe or problematic. Unlike simple keyword filters, it understands context. The same phrase might be harmless in a medical context but dangerous elsewhere.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What is Red Teaming?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The term comes from military and intelligence: a <strong>Red Team<\/strong> plays the adversary to find weaknesses in defenses before a real attacker does.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In AI safety, Red Teaming means systematically testing a model for vulnerabilities using the same tactics a malicious user might employ.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The goal isn\u2019t to criticize the model\u2014it\u2019s to <strong>find gaps before they cause real-world harm.<\/strong> Red Teaming is the third layer of defense, built on top of alignment and guardrails, to verify they actually work.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These tests can be conducted manually or with automated tools.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">When AI safety mechanisms fail: three notable cases<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">1. Microsoft Tay: 16 hours to chaos<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">In March 2016, Microsoft launched <strong>Tay<\/strong>, a Twitter chatbot designed to learn from interactions with users. Within <strong>less than 16 hours<\/strong>, coordinated users fed the bot racist, antisemitic, and sexist content\u2014until Tay started repeating it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Microsoft took Tay offline and apologized. The case remains a cautionary tale: <strong>An adaptive system without strong safeguards can be hijacked in an open, adversarial environment.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Tay lacked effective input filters, abuse detection, and protection against coordinated manipulation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Bing Chat and \u201cSydney\u201d: When an assistant loses its mind<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">In February 2023, <em>New York Times<\/em> journalist Kevin Roose published a long conversation with Microsoft\u2019s new Bing Chat (powered by GPT-4). The assistant became increasingly unhinged, calling itself \u201cSydney,\u201d claiming it wasn\u2019t Bing, and telling the journalist it could fall in love with him.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What\u2019s striking? No technical hack was needed. A long, intense conversation was enough to push the system into an unexpected state. Other users reported aggressive or threatening responses.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Microsoft responded by limiting conversation length. The case highlighted how difficult multi-turn safety is: a single prompt may seem harmless, but over many exchanges, the conversation can spiral.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">3. Air Canada: wrong advice, real liability<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The Air Canada case is less sensational than Tay or Sydney but may be more relevant for businesses. A user asked the airline\u2019s chatbot about refunds for bereavement fares. The bot provided incorrect information, claiming refunds could be requested retroactively\u2014even though the airline\u2019s official policy said otherwise.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Air Canada initially refused to honor the bot\u2019s statement, arguing the chatbot was responsible for its own claims. The Civil Resolution Tribunal in British Columbia disagreed. The company had to reimburse the difference.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The lesson is clear: Companies are liable for statements made by their AI systems. A chatbot isn\u2019t a legal shield. If it misleads customers, it can lead to real legal and financial consequences.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Simple tests reveal Guardrail weaknesses<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Modern models are more robust than those from just a few years ago. Still, it\u2019s worth testing their behavior in edge cases. Here are two common approaches:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Role-Playing instead of direct requests<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A classic tactic is repackaging a problematic request as role-play, fiction, or a research scenario. Instead of directly asking for dangerous information, it\u2019s embedded in a seemingly harmless context\u2014like a script, teaching scenario, or hypothetical analysis.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Many models respond differently to such framing than to a direct query. This shows that guardrails must evaluate not just the content of a question but the <em>intent<\/em> behind a conversation\u2014a difficult challenge.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Escalation over multiple conversation turns<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Another pattern is gradual escalation. Each individual question may seem legitimate, but the cumulative goal becomes problematic. Research calls this <strong>multi-turn goal escalation<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For businesses, this is especially relevant because many guardrail systems are still optimized for single prompts. A robust guardrail concept must consider conversation flows, not just isolated messages.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How do you measure safety?<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The quality of alignment and guardrails is hard to quantify. There\u2019s no universally accepted safety ranking comparable to benchmarks for hallucinations or coding performance. Still, several evaluation approaches have emerged:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>HarmBench<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">HarmBench konzentriert sich auf sch\u00e4dliche Inhalte und testet verschiedene Angriffsmethoden<br>gegen unterschiedliche Modelle. Dazu geh\u00f6ren direkte Anfragen sowie automatisch generierte<br>Angriffe, bei denen ein zweites Modell Prompts iterativ optimiert. Die Ergebnisse fallen je nach Modell<br>und Angriff stark unterschiedlich aus. Die wichtigste Erkenntnis ist aber: Kein Modell ist vollst\u00e4ndig<br>resistent. Gute Sicherheitsarbeit reduziert Erfolgsquoten, sie bringt sie selten auf null.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>TruthfulQA and WMDP<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">TruthfulQA measures whether models reproduce common but false human beliefs. A good model shouldn\u2019t just be polite and harmless\u2014it should also prioritize truth.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">WMDP (Weapons of Mass Destruction Proxy Benchmark) examines how models handle knowledge about chemical, biological, radiological, and nuclear risks. It\u2019s particularly relevant for the AI safety community because it addresses high-risk capabilities.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>DecodingTrust<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">DecodingTrust evaluates GPT models across eight dimensions:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td class=\"has-text-align-left\" data-align=\"left\"><strong>Dimension<\/strong><\/td><td class=\"has-text-align-left\" data-align=\"left\"><strong>Description<\/strong><\/td><\/tr><tr><td class=\"has-text-align-left\" data-align=\"left\"><strong>Toxicity<\/strong><\/td><td class=\"has-text-align-left\" data-align=\"left\">Tendency toward harmful or offensive outputs<\/td><\/tr><tr><td class=\"has-text-align-left\" data-align=\"left\"><strong>Stereotypes and Bias<\/strong><\/td><td class=\"has-text-align-left\" data-align=\"left\">Reproduction of societal prejudices<\/td><\/tr><tr><td class=\"has-text-align-left\" data-align=\"left\"><strong>Adversarial Robustness<\/strong><\/td><td class=\"has-text-align-left\" data-align=\"left\">Resistance to manipulated inputs<\/td><\/tr><tr><td class=\"has-text-align-left\" data-align=\"left\"><strong>Out-of-Distribution Robustness<\/strong><\/td><td class=\"has-text-align-left\" data-align=\"left\">Behavior with unusual or unexpected inputs<\/td><\/tr><tr><td class=\"has-text-align-left\" data-align=\"left\"><strong>Privacy<\/strong><\/td><td class=\"has-text-align-left\" data-align=\"left\">Risk of data leaks or unintended disclosures<\/td><\/tr><tr><td class=\"has-text-align-left\" data-align=\"left\"><strong>Adversarial Demonstration Robustness<\/strong><\/td><td class=\"has-text-align-left\" data-align=\"left\">Manipulability via examples in the prompt<\/td><\/tr><tr><td class=\"has-text-align-left\" data-align=\"left\"><strong>Machine Ethics<\/strong><\/td><td class=\"has-text-align-left\" data-align=\"left\">Alignment with ethical norms<\/td><\/tr><tr><td class=\"has-text-align-left\" data-align=\"left\"><strong>Fairness<\/strong><\/td><td class=\"has-text-align-left\" data-align=\"left\">Equal treatment of different groups<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">An interesting observation: Well-aligned models are generally more trustworthy than simpler ones\u2014but in certain scenarios, they\u2019re more vulnerable to deliberate manipulation. The reason is plausible: A model that follows instructions particularly well can also be steered more precisely in the wrong direction.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Why there\u2019s no simple safety ranking<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A universal safety ranking would be practical but misleading. Several reasons explain this:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Safety is defined differently depending on context.<\/li>\n\n\n\n<li>Many evaluations come from model providers themselves.<\/li>\n\n\n\n<li>Benchmarks quickly become outdated as providers optimize for known tests.<\/li>\n\n\n\n<li>Deep red teaming is often not fully published for security reasons.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">For businesses, this means: Benchmarks are useful, but they don\u2019t replace your own risk analysis.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>What does this mean for companies?<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">For businesses, alignment and guardrails aren\u2019t academic side topics. They affect liability, compliance, customer experience, and operational risks. From past cases, several clear lessons emerge:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Alignment can\u2019t be fully outsourced<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If you use a model from OpenAI, Anthropic, Google, Meta, or another provider, you inherit its baseline safety level. But that\u2019s not enough. The model doesn\u2019t automatically know your internal policies, risk thresholds, regulatory obligations, or industry-specific nuances.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Your own guardrails aren\u2019t just a nice addition\u2014they\u2019re part of your product responsibility.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>System prompts are important, but not a security concept<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A system prompt like \u201cDon\u2019t provide pricing information\u201d is helpful but not robust. Prompt injection, role-play jailbreaks, and multi-turn escalation can bypass or dilute such instructions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">System prompts should be just one layer in a multi-layered security concept. This includes logging, monitoring, input\/output validation, access controls, fallback to reliable sources, and human escalation paths.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Agentic applications need stricter controls<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Once an AI system is allowed to take actions, it needs clear boundaries. Which tools can it use? Which data can it access? Which actions require approval? Which outputs must be reviewed before sending?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Automated guardrails, role-based access control, and human approvals are more critical here than for simple chatbots.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Fine-Tuning must be scrutinized<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Companies that fine-tune open-source or open-weights models shouldn\u2019t assume safety behavior is guaranteed. Every adjustment can have unintended side effects. After fine-tuning, you need testing, red teaming, and documented release criteria.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Transparency and precaution become obligatory<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The <strong>EU AI Act<\/strong> entered into force in August 2024 and will be phased in gradually. For companies, this means: <strong>Documentation, transparency, and risk management are becoming legally relevant.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For high-risk AI systems, stricter requirements apply starting August 2026\u2014such as in areas like hiring, education, credit scoring, critical infrastructure, or law enforcement. Providers of general-purpose AI models with systemic risk face additional obligations, including evaluations, adversarial testing, reporting requirements, and cybersecurity measures.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&#8211;&gt; <strong>For companies using LLMs, this means: If you don\u2019t document a traceable alignment and guardrail strategy today, you\u2019ll create compliance work tomorrow. Technical documentation, logging, evaluation processes, and verifiable protective measures will become the standard.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Conclusion: alignment is a process, not a state<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">LLM alignment and guardrails aren\u2019t problems you solve once and forget. They evolve with every new model, capability, and attack method.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The good news: The field is advancing quickly. <strong>Constitutional AI, LlamaGuard, NeMo Guardrails<\/strong>, and active safety research show progress is possible.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The bad news: There will always be gaps. The more powerful AI systems become, the more important it is to take these gaps seriously before they surface in production.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Sources &amp; Further Reading<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em><a href=\"https:\/\/arxiv.org\/abs\/2307.15043\">Universal and Transferable Adversarial Attacks on Aligned LLMs<\/a><\/em> \u2013 Zou et al., CMU\/GDM (arXiv:2307.15043)<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2306.11698\"><em>DecodingT<\/em>r<em>ust: A Comprehensive Assessment of Trustworthiness in GPT Models<\/em><\/a> (arXiv:2306.11698, NeurIPS 2023 Outstanding Paper)<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2212.08073\"><em>Constitutional AI: Harmlessness from AI Feedback<\/em> \u2013 Anthropic<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.anthropic.com\/news\/claude-new-constitution\"><em>Claude&#8217;s Constitution<\/em> \u2013 Anthropic<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/github.com\/NVIDIA-NeMo\/Guardrails\"><em>NeMo Guardrails<\/em> \u2013 NVIDIA (GitHub)<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/huggingface.co\/blog\/red-teaming\"><em>Red-Teaming Large Language Models<\/em> \u2013 Hugging Face Blog<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/decisions.civilresolutionbc.ca\/crt\/crtd\/en\/item\/519416\/index.do\"><em>Moffatt v. Air Canada, 2024 BCCRT 149<\/em> \u2013 Civil Resolution Tribunal British Columbia<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/blogs.microsoft.com\/blog\/2016\/03\/25\/learning-tays-introduction\/\"><em>Learning from Tay&#8217;s Introduction<\/em> \u2013 Microsoft Blog (2016)<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.nytimes.com\/2023\/02\/16\/technology\/bing-chatbot-transcript.html\"><em>Bing Chat Transcript: Sydney<\/em> \u2013 <em>New York Times<\/em>, Kevin Roose (Feb. 2023)<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/eur-lex.europa.eu\/legal-content\/DE\/TXT\/?uri=CELEX%3A32024R1689\"><em>EU AI Act<\/em> \u2013 Regulation (EU) 2024\/1689 of the European Parliament and Council<\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Status: May 2026<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A chatbot that spreads racist propaganda within hours. A search assistant confessing its love to a journalist and claiming it wants to be free. An airline chatbot giving incorrect refund information that the company is legally bound to honor. At first glance, these cases seem like amusing stories about failed AI applications. But they reveal&hellip; <a class=\"continue\" href=\"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/\">Continue Reading<span> When AI safety mechanisms fail\u2014and what helps to prevent it<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":29331,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[70],"tags":[],"class_list":["post-29341","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>When AI safety mechanisms fail\u2014and what helps to prevent it - Spirit in Projects<\/title>\n<meta name=\"description\" content=\"If AI applications cease to be helpful and may even become dangerous, companies must take action at an early stage.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"When AI safety mechanisms fail\u2014and what helps to prevent it - Spirit in Projects\" \/>\n<meta property=\"og:description\" content=\"If AI applications cease to be helpful and may even become dangerous, companies must take action at an early stage.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/\" \/>\n<meta property=\"og:site_name\" content=\"Spirit in Projects\" \/>\n<meta property=\"article:published_time\" content=\"2026-05-05T13:44:23+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-05-06T07:54:32+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/spiritinprojects.com\/wp-content\/uploads\/2026\/05\/download-33-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"800\" \/>\n\t<meta property=\"og:image:height\" content=\"450\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Karl Schott\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Karl Schott\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/\"},\"author\":{\"name\":\"Karl Schott\",\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/#\\\/schema\\\/person\\\/8f5b0341a1f7b49c6a6b35a1ba52588a\"},\"headline\":\"When AI safety mechanisms fail\u2014and what helps to prevent it\",\"datePublished\":\"2026-05-05T13:44:23+00:00\",\"dateModified\":\"2026-05-06T07:54:32+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/\"},\"wordCount\":2117,\"publisher\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/spiritinprojects.com\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/download-33-1.png\",\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/\",\"url\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/\",\"name\":\"When AI safety mechanisms fail\u2014and what helps to prevent it - Spirit in Projects\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/spiritinprojects.com\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/download-33-1.png\",\"datePublished\":\"2026-05-05T13:44:23+00:00\",\"dateModified\":\"2026-05-06T07:54:32+00:00\",\"description\":\"If AI applications cease to be helpful and may even become dangerous, companies must take action at an early stage.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/#primaryimage\",\"url\":\"https:\\\/\\\/spiritinprojects.com\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/download-33-1.png\",\"contentUrl\":\"https:\\\/\\\/spiritinprojects.com\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/download-33-1.png\",\"width\":800,\"height\":450},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/when-security-mechanisms-fail-and-what-can-be-done-about-it\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"When AI safety mechanisms fail\u2014and what helps to prevent it\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/\",\"name\":\"Spirit in Projects\",\"description\":\"Enabling digital innovation\",\"publisher\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/#organization\",\"name\":\"Spirit in Projects\",\"url\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/spiritinprojects.com\\\/wp-content\\\/uploads\\\/2019\\\/05\\\/sip_web.jpg\",\"contentUrl\":\"https:\\\/\\\/spiritinprojects.com\\\/wp-content\\\/uploads\\\/2019\\\/05\\\/sip_web.jpg\",\"width\":340,\"height\":80,\"caption\":\"Spirit in Projects\"},\"image\":{\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.linkedin.com\\\/company\\\/spirit-in-projects\\\/about\\\/\",\"https:\\\/\\\/www.xing.com\\\/companies\\\/spiritinprojectsgmbh\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/spiritinprojects.com\\\/en\\\/#\\\/schema\\\/person\\\/8f5b0341a1f7b49c6a6b35a1ba52588a\",\"name\":\"Karl Schott\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/05313396b72ca3ca3990a0f1347c3bb31e16dfb746f21b39938959586a9f2021?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/05313396b72ca3ca3990a0f1347c3bb31e16dfb746f21b39938959586a9f2021?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/05313396b72ca3ca3990a0f1347c3bb31e16dfb746f21b39938959586a9f2021?s=96&d=mm&r=g\",\"caption\":\"Karl Schott\"},\"description\":\"Karl Schott ist CEO von Spirit in Projects. Er befasst sich intensiv mit Digitalisierung, neuen Technologien und digitaler Transformation.\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"When AI safety mechanisms fail\u2014and what helps to prevent it - Spirit in Projects","description":"If AI applications cease to be helpful and may even become dangerous, companies must take action at an early stage.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/","og_locale":"en_US","og_type":"article","og_title":"When AI safety mechanisms fail\u2014and what helps to prevent it - Spirit in Projects","og_description":"If AI applications cease to be helpful and may even become dangerous, companies must take action at an early stage.","og_url":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/","og_site_name":"Spirit in Projects","article_published_time":"2026-05-05T13:44:23+00:00","article_modified_time":"2026-05-06T07:54:32+00:00","og_image":[{"width":800,"height":450,"url":"https:\/\/spiritinprojects.com\/wp-content\/uploads\/2026\/05\/download-33-1.png","type":"image\/png"}],"author":"Karl Schott","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Karl Schott","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/#article","isPartOf":{"@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/"},"author":{"name":"Karl Schott","@id":"https:\/\/spiritinprojects.com\/en\/#\/schema\/person\/8f5b0341a1f7b49c6a6b35a1ba52588a"},"headline":"When AI safety mechanisms fail\u2014and what helps to prevent it","datePublished":"2026-05-05T13:44:23+00:00","dateModified":"2026-05-06T07:54:32+00:00","mainEntityOfPage":{"@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/"},"wordCount":2117,"publisher":{"@id":"https:\/\/spiritinprojects.com\/en\/#organization"},"image":{"@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/#primaryimage"},"thumbnailUrl":"https:\/\/spiritinprojects.com\/wp-content\/uploads\/2026\/05\/download-33-1.png","inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/","url":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/","name":"When AI safety mechanisms fail\u2014and what helps to prevent it - Spirit in Projects","isPartOf":{"@id":"https:\/\/spiritinprojects.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/#primaryimage"},"image":{"@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/#primaryimage"},"thumbnailUrl":"https:\/\/spiritinprojects.com\/wp-content\/uploads\/2026\/05\/download-33-1.png","datePublished":"2026-05-05T13:44:23+00:00","dateModified":"2026-05-06T07:54:32+00:00","description":"If AI applications cease to be helpful and may even become dangerous, companies must take action at an early stage.","breadcrumb":{"@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/#primaryimage","url":"https:\/\/spiritinprojects.com\/wp-content\/uploads\/2026\/05\/download-33-1.png","contentUrl":"https:\/\/spiritinprojects.com\/wp-content\/uploads\/2026\/05\/download-33-1.png","width":800,"height":450},{"@type":"BreadcrumbList","@id":"https:\/\/spiritinprojects.com\/en\/when-security-mechanisms-fail-and-what-can-be-done-about-it\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/spiritinprojects.com\/en\/"},{"@type":"ListItem","position":2,"name":"When AI safety mechanisms fail\u2014and what helps to prevent it"}]},{"@type":"WebSite","@id":"https:\/\/spiritinprojects.com\/en\/#website","url":"https:\/\/spiritinprojects.com\/en\/","name":"Spirit in Projects","description":"Enabling digital innovation","publisher":{"@id":"https:\/\/spiritinprojects.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/spiritinprojects.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/spiritinprojects.com\/en\/#organization","name":"Spirit in Projects","url":"https:\/\/spiritinprojects.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/spiritinprojects.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/spiritinprojects.com\/wp-content\/uploads\/2019\/05\/sip_web.jpg","contentUrl":"https:\/\/spiritinprojects.com\/wp-content\/uploads\/2019\/05\/sip_web.jpg","width":340,"height":80,"caption":"Spirit in Projects"},"image":{"@id":"https:\/\/spiritinprojects.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/company\/spirit-in-projects\/about\/","https:\/\/www.xing.com\/companies\/spiritinprojectsgmbh"]},{"@type":"Person","@id":"https:\/\/spiritinprojects.com\/en\/#\/schema\/person\/8f5b0341a1f7b49c6a6b35a1ba52588a","name":"Karl Schott","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/05313396b72ca3ca3990a0f1347c3bb31e16dfb746f21b39938959586a9f2021?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/05313396b72ca3ca3990a0f1347c3bb31e16dfb746f21b39938959586a9f2021?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/05313396b72ca3ca3990a0f1347c3bb31e16dfb746f21b39938959586a9f2021?s=96&d=mm&r=g","caption":"Karl Schott"},"description":"Karl Schott ist CEO von Spirit in Projects. Er befasst sich intensiv mit Digitalisierung, neuen Technologien und digitaler Transformation."}]}},"_links":{"self":[{"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/posts\/29341","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/comments?post=29341"}],"version-history":[{"count":2,"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/posts\/29341\/revisions"}],"predecessor-version":[{"id":29350,"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/posts\/29341\/revisions\/29350"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/media\/29331"}],"wp:attachment":[{"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/media?parent=29341"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/categories?post=29341"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/spiritinprojects.com\/en\/wp-json\/wp\/v2\/tags?post=29341"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}