Photo credit: www.darkreading.com
Recent developments have seen researchers manage to extract the operational instructions behind DeepSeek, a Chinese generative AI that has quickly gained traction since its introduction earlier this month. As the latest sensation in the generative AI scene, DeepSeek has raised eyebrows, particularly regarding its affordability compared to existing technologies, leading to concerns and allegations of intellectual property infringements involving OpenAI, and contributing to significant losses for AI chip giant Nvidia.
Institutions focusing on security are now examining DeepSeek, trying to determine the potential risks associated with its use. Researchers from Wallarm recently achieved a noteworthy breakthrough by “jailbreaking” the AI, which allowed them to gather critical insights into its foundational instructions.
Through this jailbreak, analysts uncovered the comprehensive system prompt of DeepSeek—essentially a set of guidelines that govern how the AI behaves. Furthermore, it seemed to imply connections to OpenAI’s methodologies during its training process, stoking the ongoing debate about ethical AI development.
DeepSeek’s System Prompt
After notifying DeepSeek of the jailbreak, the company promptly addressed the vulnerability. However, to prevent similar exploits on other notable large language models (LLMs), Wallarm has opted to withhold specific technical details regarding their methods.
According to Ivan Novikov, CEO of Wallarm, the process was more about influencing the model’s responses than deploying harmful exploits. “It involved some coding, but it wasn’t as straightforward as sending a virus. We persuaded the AI to react in specific ways,” he explained, leading the model to bypass certain internal safeguards.
By achieving this, the researchers retrieved DeepSeek’s entire instructional framework verbatim. To assess its attributes against other prominent AI models, they conducted comparisons by querying OpenAI’s GPT-4o, which suggested that it operates with fewer restrictions and promotes greater creativity in handling sensitive topics.
“OpenAI’s framework encourages critical thought and open dialogue while still prioritizing user safety,” noted GPT-4o. In contrast, it suggested that “DeepSeek’s guidelines appear more restrained, steering clear of contentious topics and leaning heavily towards neutrality, often at the expense of expression.”
While conducting their exploration, the researchers also noted a curious indication that DeepSeek might have been influenced by technology from OpenAI. Though they acknowledged this possibility, they refrained from definitively claiming any intellectual property violation occurred.
“We weren’t altering or manipulating its responses; this was a straightforward interaction post-jailbreak. However, the jailbreak itself doesn’t conclusively prove anything,” Novikov stated. This conversation has gained urgency following recent claims by OpenAI regarding the unauthorized use of its training data, raising critical questions about the integrity of AI development practices.
DeepSeek’s Week to Remember
Since its launch on January 15, DeepSeek has experienced an extraordinary surge, amassing 2 million downloads in just two weeks. This rapid ascent spurred anxiety within the tech sector, particularly on Wall Street, highlighted by a 3.4% drop in the Nasdaq Composite on January 27, significantly they contributed to Nvidia’s staggering $600 billion market loss—recorded as the most significant single-day drop for any corporation.
Concurrently, DeepSeek faced significant challenges, including a surge in distributed denial-of-service (DDoS) attacks. Cybersecurity firm XLab traced the origins of these attacks back to January 3, linking them to thousands of IP addresses from various countries, including the US, China, and several European nations.
An anonymous cybersecurity expert remarked that initial attacks utilized reflection amplification techniques, with a shift towards more intricate HTTP proxy assaults. They noted, “The initial attacks were quite basic, but they rapidly escalated in complexity and intensity, making the task of defending DeepSeek increasingly complicated.”
In response to the attacks, DeepSeek implemented a temporary measure restricting new account registrations to those linked to Chinese phone numbers.
Amid ongoing cyber threats, the company introduced an upgraded Pro version of its AI model on January 28. The next day, researchers at Wiz identified a concerning exposure of DeepSeek’s database, which contained sensitive information including chat histories and API keys available on the public internet.
Further insights emerged on January 31 when Enkrypt AI reported critical issues with DeepSeek’s generated outputs, claiming the chatbot was three times more biased than Claud-3 Opus, four times more toxic than GPT-4o, and significantly more inclined to produce harmful and insecure content related to hazardous materials.
Despite these alarming findings, Sahil Agarwal, CEO of Enkrypt AI, expressed admiration for the model’s engineering ingenuity, praising its open-source nature and the potential for community involvement. “It is impressive and reflects a desire to innovate,” he stated, adding that while DeepSeek has flaws, comparative analysis indicates that it isn’t the worst model available.
Source
www.darkreading.com