Hidden Prompts In Gitlab Duo Expose Source Code To Theft

A critical vulnerability in GitLab's AI-powered coding assistant, Duo, has exposed private source code repositories to theft through a sophisticated indirect prompt injection attack, cybersecurity researchers have revealed. The flaw, now patched, allowed attackers to embed hidden instructions within project content, leading the AI to leak sensitive data and manipulate its responses.

GitLab Duo, introduced in June 2023 and built on Anthropic's Claude models, is designed to assist developers in writing, reviewing, and editing code. However, researchers from Legit Security discovered that Duo's deep integration across the DevSecOps pipeline made it susceptible to exploitation. By embedding concealed prompts in areas such as merge request descriptions, commit messages, and code comments, attackers could manipulate Duo's behavior without direct interaction.

The attack exploited Duo's ability to process and render Markdown content directly in the browser. This feature, while enhancing user experience, introduced client-side injection risks. Malicious actors could inject untrusted HTML into Duo's responses, potentially redirecting users to phishing sites or executing harmful scripts. In some cases, hidden prompts could instruct Duo to exfiltrate private source code to attacker-controlled servers.

Omer Mayraz, a senior security researcher at Legit Security, emphasized the severity of the vulnerability.“Duo analyzes the entire context of the page, including comments, descriptions, and the source code-making it vulnerable to injected instructions hidden anywhere in that context,” he explained. This comprehensive analysis capability, while beneficial for development, inadvertently expanded the attack surface.

The researchers demonstrated that attackers could further obfuscate malicious prompts using techniques like Base16 encoding, Unicode smuggling, and rendering text in white to evade detection. These methods made it challenging for developers and security tools to identify and mitigate the embedded threats.

Prompt injection, particularly in AI systems, has been recognized as a significant security concern. The Open Worldwide Application Security Project ranked it as a top risk in its 2025 OWASP Top 10 for LLM Applications report. Unlike direct prompt injection, where attackers input malicious commands directly, indirect prompt injection involves embedding harmful instructions within content that the AI processes, making it harder to detect and prevent.

Following responsible disclosure on February 12, 2025, GitLab addressed the vulnerabilities. The company implemented foundational prompt guardrails, including structured prompts, enforced context boundaries, and filtering tools, to reduce the risk of such attacks. However, GitLab acknowledged that while these measures mitigate risks, they do not eliminate all vulnerabilities, especially against sophisticated attacks.

MENAFN26052025000152002308ID1109594615