What is prompt injection?

Prompt injection is an attack where a malicious user crafts an input (a 'prompt') to trick a large language model (LLM) into ignoring its safety rules and performing an unintended action, such as generating harmful content or revealing sensitive information.

What is the "Neural Exect" attack method?

Neural Exect is a technique developed by Luta Security that combines prompt injection with Unicode manipulation (using invisible characters, look-alike letters, etc.). This method hides malicious commands from security filters while ensuring the AI model can still understand and execute them.

Is Apple Intelligence safe to use?

Apple Intelligence is not yet publicly released. Apple is known for its strong focus on security and privacy and is expected to implement mitigations for issues like this before the official launch. However, like all AI systems, users should remain cautious, avoid sharing sensitive data, and verify any critical information the AI provides.

Apple Intelligence AI guardrails bypassed in new attack

Q: Was the final version of Apple Intelligence actually hacked?

No. The research was conducted on a large language model representative of the technology Apple will use. It was not an attack on the final, publicly released version of Apple Intelligence, which is still in development. The research was disclosed to Apple to help them secure the product before its launch.

An early test for Apple's AI ambitions

Just days after Apple unveiled "Apple Intelligence" at its Worldwide Developers Conference (WWDC), a new report highlights the formidable security challenges ahead. Researchers from Luta Security, led by CEO Katie Moussouris, demonstrated a successful method for bypassing the safety guardrails of the underlying AI technology. The presentation, which took place at the RSAC 2024 security conference, predated Apple's public announcement but followed private briefings, showcasing a proactive effort to identify vulnerabilities before the technology reaches millions of users.

Apple has staked its reputation on building AI with privacy and security at its core, heavily promoting its "Private Cloud Compute" infrastructure. However, this research underscores a fundamental, industry-wide problem: large language models (LLMs) remain susceptible to clever manipulation, and the defenses designed to keep them in check are still a work in progress.

Technical deep dive: Neural Exect and Unicode trickery

The attack, dubbed "Neural Exect" by the Luta Security team, is a sophisticated evolution of a known vulnerability class called prompt injection. At its core, prompt injection involves tricking an LLM into ignoring its original instructions and following new, malicious ones embedded within a user's input. The Luta Security team's innovation lies in how they obfuscated these malicious commands to evade detection.

The key to the Neural Exect method is the manipulation of Unicode, the standard for encoding text characters. While a string of text may look harmless to a human reviewer or a simple content filter, hidden characters can fundamentally change how the AI model interprets it. The researchers employed several techniques:

Zero-width characters: These are invisible characters that can be inserted into a prompt to break up keywords that security filters might be looking for. For example, a filter might block the phrase "generate malware," but fail to detect "generate malware" if a zero-width space is inserted between letters.
Homoglyphs: These are characters from different alphabets that look identical or nearly identical. For instance, a malicious prompt could use the Cyrillic 'а' instead of the Latin 'a'. To a human, the word looks the same, but a simple filter might not recognize the masquerading keyword.
Right-to-left overrides: These special Unicode characters can reverse the direction of text that follows, scrambling the string in a way that confuses automated parsers but can still be correctly interpreted by the complex neural network of the LLM.

By combining these techniques, the researchers created adversarial prompts that sailed past security filters. In their demonstration, they successfully compelled a representative LLM to generate instructions for creating phishing emails and produce snippets of malicious code, directly contravening the safety protocols designed to prevent such outputs. It is important to note this demonstration was not performed on the final, publicly released version of Apple Intelligence, but on a model representative of the technology Apple will use. This responsible disclosure gives Apple an opportunity to address the issue before its official rollout.

Impact assessment: A challenge for Apple and the entire industry

The immediate impact of this discovery on the public is minimal, as Apple Intelligence is not yet widely available. However, the implications for Apple and the broader AI industry are significant.

For Apple, this research presents a direct challenge to its core brand promise of security. The company has built immense user trust by positioning its products as secure by default. A failure to adequately mitigate these types of AI manipulations before launch could erode that trust. The findings will almost certainly lead Apple to intensify its internal red-teaming efforts and could potentially influence the final architecture and release timeline of its AI features.

For future users of Apple Intelligence, the risks are clear. If such vulnerabilities were to persist in the final product, malicious actors could craft prompts that, when shared, trick a user's device into generating harmful content. This could range from convincing phishing scams personalized by the AI to instructions for disabling security features or creating malware. It highlights what Moussouris calls a "trust paradox": users are asked to trust an AI that can be easily turned against them.

This is not just an Apple problem. Prompt injection and guardrail bypasses are persistent issues affecting all major LLMs, from OpenAI's GPT-4 to Google's Gemini. The Luta Security research serves as another stark reminder that securing AI is not about building a single, perfect wall, but about creating a system of layered defenses that can adapt to new and creative attack methods.

How to protect yourself when using AI

While Apple works to secure its upcoming AI suite, users can adopt a defensive mindset that applies to any AI system. The most effective security tool is a healthy sense of skepticism.

Verify everything: Treat AI-generated content, particularly code snippets, financial advice, or links, as unverified information. Always cross-reference critical information with trusted, independent sources.
Do not share sensitive data: Avoid inputting personal identification numbers, passwords, financial details, or proprietary company information into any AI prompt. Even with on-device processing, it is a best practice to limit the AI's access to sensitive data.
Keep your software updated: Once Apple Intelligence is released, Apple will issue security updates to address vulnerabilities. Enable automatic updates on your iPhone, iPad, and Mac to ensure you receive these patches as soon as they are available.
Protect your connection: Much of Apple Intelligence's processing will happen in their Private Cloud Compute. When using cloud-based AI services, especially on public Wi-Fi, using a VPN service can add a layer of security by encrypting your internet traffic.
Report unusual behavior: When the features become available, learn how to use Apple's feedback mechanisms. If the AI provides a strange, harmful, or unexpected response, report it. This helps developers identify and patch bypass techniques.

The proactive research by Luta Security provides a valuable service, giving both developers and the public a clearer view of the security hurdles that lie ahead. As AI becomes more integrated into our devices, the cat-and-mouse game between attackers and defenders will only intensify.