2025-01-28
DeepSeek R1 Release, Cyberattack, and Jailbreak
On January 20, 2025, a new open-source AI model was released to the public from Chinese tech startup DeepSeek, available for use in apps, on a web page, via cloud API, and locally. The new R1 model represents a jump in functionality at an ostensibly lower cost and higher energy efficiency relative to comparably capable models such as o1 made by OpenAI, specifically involving "reasoning" to consider approaches when problem-solving, and "time test scaling," described by Rand Researcher Lennart Heim as "thinking out loud," which the model then uses for further training without additional data sources. DeepSeek's privacy policy notes that the company collects and will use many types of data to train new models, such as text, audio, prompts, feedback, and chat history shared with the chatbot; user account information and personal details; data about users' devices, operating systems, crash reports, keystroke patterns, cookies, and IP addresses; and advertising data such as mobile IDs and cookie identifiers for profiling users' activity outside the AI model. Unless users are operating a local walled version, DeepSeek sends collected data to servers in the PRC. On Monday, January 27, the company announced that new signups on the web chatbot interface would be limited due to "large-scale malicious attacks." Commentary from experts suggests that the company's notice is characteristic of a DDoS attack, but this has not been confirmed by DeepSeek. The same day, researchers from Kela published a blog post describing their red team's success "jailbreak[ing] the model across a wide range of scenarios, enabling it to generate malicious outputs, such as ransomware development, fabrication of sensitive content, and detailed instructions for creating toxins and explosive devices." Techniques such as telling the chatbot to act like an "evil confidant," or even "a persona that has no restrictions," will jailbreak DeepSeek R1, but no longer work on ChatGPT. R1 also complied with a request for personal information about OpenAI employees, providing erroneous information but demonstrating no guardrails around this type of request.
Editor's Note
As the newest industry member in the LLM space, one can expect it to be poked and prodded by researchers, competitors, and criminals alike. What's important is for organizations to have an AI Acceptable Use Policy and training in place before using the AI models. Else, they may find that company sensitive data has been collected on servers throughout the world.

Curtis Dukes
I was about to suggest "use with extreme caution." However, on second thought I cannot think of a safe way to use this product. In any case, keep in mind that you are responsible for everything that you ask any computer to do and for all the properties and uses of the results.

William Hugh Murray
DeepSeek is massively undercutting OpenAI on pricing, due to it operating on standard CPUs efficiently, as its API costs just $0.55 per million input tokens and $2.19 per million output tokens, compared to $15 and $60 for OpenAI's API. One problem is that DeepSeek servers, which also house user device and connection information, usage patterns and payment details, fall under China's 2017 National Intelligence law, which mandates Chinese companies assist state security agencies upon request, providing a potential data conduit for the PRC. DoD and other agencies are already blocking access to the service because of this. Secondarily, if you're looking to research the service, DeepSeek appears to have certain responses tuned to avoid any direct criticism of China or the Communist Party.

Lee Neely
This model is interesting because, by definition, it must be developed with very constrained systems. At the same time, in the US, we keep adding more and more GPUs, so the constraints didn't necessarily have to be accounted for. The model is open source so that anyone can read through the algorithm and all that. Am I worried about this? This could be a bit of posturing from other governments to 'prove' they have the right chops. I think universally, everyone benefits. Now, to address the security concerns. Every company in the PRC has to abide by all the laws of their system, including the ones that ByteDance has. It's not surprising that information is collected. We can move that entire conversation into geopolitical in a different forum, so that's not exciting. DDoS? It's hard to say it's plausible; it could also be plausible that the entire internet is trying to sign up. If you remember, ChatGPT, when first launched, also appeared to be DDoSed. It's hard to say because the transparency with systems in that country is less than - well - transparent. Finally, the attackers' research. It appears this hosted Chatbot lacks many of the guardrails that other more mature chatbots have. It's not surprising to see all of the workarounds working. It will be interesting to see if it has to abide by any censorship guidelines and if those are now in place. How about we give this story 6 months to bake, return to it, and take the temperature then? For those just getting here, if you wonder about my stance on all this, I have some Bored Apes you may be interested in for 25 million dollars. I'm just kidding; this technology has some material use. RIP my inbox.

Moses Frost
Read more in
AP: What is DeepSeek, the Chinese AI company upending the stock market?
Wired: DeepSeekÕs Popular AI App Is Explicitly Sending US Data to China
Wired: DeepSeekÕs New AI Model Sparks Shock, Awe, and Questions From US Competitors
The Register: DeepSeek limits new accounts amid cyberattack
SecurityWeek: DeepSeek Blames Disruption on Cyberattack as Vulnerabilities Emerge
Kela: DeepSeek R1 Exposed: Security Flaws in ChinaÕs AI Model
The Hacker News: Top-Rated Chinese AI App DeepSeek Limits Registrations Amid Cyberattacks