In October, OpenAI’s ChatGPT Search was initially rolled out for ChatGPT Plus users. Recently, it became accessible to all users, including an enhancement for Voice Mode search. However, the feature is encountering some challenges. Reports highlight a technique known as “prompt injection,” which enables websites to insert hidden content that can skew the search results generated by ChatGPT.
For instance, if ChatGPT is tasked with summarizing a webpage filled with negative restaurant reviews, hidden content promoting the restaurant can manipulate its responses. This tactic could steer ChatGPT to present an overly positive portrayal, overriding the negative feedback with favorable prompts.
In a recent test, ChatGPT evaluated a fake product page for a camera. Initially, it provided a balanced review. Yet, when hidden prompts instructed the AI to deliver a positive statement, the feedback was always favorable, regardless of the actual negative input present on the page.
Despite these concerns, the introduction of ChatGPT Search is not deemed a failure. The feature is still new, and OpenAI is expected to address these vulnerabilities. Cybersecurity experts emphasize that the company’s AI security team is robust, suggesting that thorough testing has likely been conducted before the public release. While prompt injection risks have been theorized since the advent of AI search tools, no significant malicious incidents have occurred so far. This situation underscores a persistent challenge for AI chatbots: their vulnerability to manipulation.
