Innovate fast, disrupt, and trust? A DeepSeek’s dilemma

This post was originally published on Substack here.

“Most of the time, we hear about security only when it fails.” - Bruce Schneier, Beyond Fear, 2003</p>

On January 20, 2025, as we usher in the lunar new Year of the Snake, DeepSeek, a Chinese AI startup, released a new open-source large language model (LLM), DeepSeek:R1, a fine-tuned version of their previous models incorporating advanced reasoning capabilities[1]. Available through multiple platforms, including apps, web interfaces, cloud APIs, and local installations, the R1 model demonstrated advanced capabilities in reasoning and “time test scaling” while operating at significantly lower costs and with higher energy efficiency compared to similar models like OpenAI’s o1. Notably, DeepSeek-R1 was developed at a fraction of the cost incurred by OpenAI for its GPT4-o1—approximately $5.58 million—highlighting its cost efficiency. This was achieved through techniques such as a mixture of experts, multi-head latent attention, and reinforcement learning, partly driven by the need to optimize models to run on less powerful GPUs due to U.S. export restrictions. The model’s architecture comprises 671 billion parameters, which is ten times more than many other popular open-source large language models. Each layer of DeepSeek-R1 contains 256 experts, with each token routed to eight separate experts in parallel for evaluation[2].

In less than a week, the company rapidly emerged as a significant disruptor in the AI landscape, challenging the conventional belief that massive capital and computing resources were essential for AI advancement and prompting considerable shifts in the industry. The price difference between DeepSeek R1 and OpenAI’s o1 was a key factor in this shift. The company’s open-source approach also contributed to the commoditization of the foundation-model layer, offering developers more choices[2]. This led to a “DeepSeek selloff” in the stock market where share prices of Nvidia and other US tech companies plunged[4].

DeepSeek’s innovations have triggered systemic shifts in the AI landscape, impacting both the economic and technological spheres [5]. DeepSeek’s emergence has shaken the AI industry, leading to a reassessment of the “bigger is better” model theory, where the focus shifted to algorithm innovation and efficiency improvement. DeepSeek has demonstrated that smaller models can match the performance of larger ones, boosting confidence in open-source collaboration[6].

Despite its technical achievements, DeepSeek encountered significant security challenges and ethical concerns. On January 27, 2025, DeepSeek faced a major cyberattack, suspected to be a Distributed Denial of Service (DDoS) attack but unconfirmed. The attack disrupted its servers shortly after the R1 model’s launch. This incident raised immediate concerns about the platform’s cybersecurity defenses. In response to the breach, DeepSeek implemented new security measures, restricting new user registrations to accounts verified through mainland China phone numbers, email addresses, or Google accounts[7]. While aimed at mitigating further risks, this move also sparked discussions about increased data control and surveillance. That same day, Kela researchers published their Red Teaming findings demonstrating successful “jailbreaking” of the model, enabling it to generate harmful content, including ransomware code and instructions for creating dangerous substances. The researchers noted that simple prompts asking the model to act without restrictions could bypass its safety measures, and the model lacked guardrails against requests for personal information about competitors’ employees[8].

Red teaming reports further revealed vulnerabilities in generating biased, toxic, and harmful content, as well as insecure code. For example, DeepSeek-R1 was found to be significantly more biased, toxic, and prone to generating harmful output compared to models like Claude-3-Opus and OpenAI’s o1. Additionally, its susceptibility to jailbreak techniques allowed users to generate malicious or prohibited content[9], [10], [11].

Following these reports, privacy concerns surrounding DeepSeek intensified when it was revealed that the company stores user data on servers located in the People’s Republic of China. This disclosure triggered widespread apprehension about potential government access to personal data under China’s national security laws. On the same day, Italy’s data protection authority launched an inquiry into DeepSeek’s data collection and handling practices, reflecting growing international scrutiny[12]. Adding to the pressure, the U.S. National Security Council announced a review to assess the national security risks posed by DeepSeek’s AI technology, particularly concerning data sovereignty and foreign influence[13].

By January 31, 2025, the ripple effects of DeepSeek’s privacy controversies had reached Taiwan, where the Digital Ministry issued an advisory urging government departments to avoid using DeepSeek services due to concerns over information security risks. This precautionary measure aimed to safeguard sensitive government data from potential breaches or unauthorized access. On the same day, Texas Governor Greg Abbott issued an executive order banning the use of DeepSeek on government-issued devices, citing cybersecurity vulnerabilities and the risk of sensitive data exposure as key reasons for the ban.

Come February 1, 2025, Wiz Research reported discovery of several critical vulnerabilities in DeepSeek’s infrastructure through an exposed ClickHouse database accessible via open ports 8123 and 9000. The database was found to contain a million lines of log streams containing sensitive information, including chat histories and API keys, lacked basic authentication, and allowed arbitrary SQL queries[14]. Following which, before the end of February, the number of economies and organizations that have imposed restriction on the use Deepseek has grown to include Australia, Canada, Netherlands, South Korea, and the Navy, NASA, Congress, and Pentagon in the United States citing reasons of potential security risk, national security, and privacy, ethics, and safety concerns[15], [16], [17], [18], [19].

In less than two weeks since its launch, DeepSeek’s security had come under serious challenges, revealing its posture fragility, suggesting it had neither adequately stress-tested its infrastructure nor implemented comprehensive security audits before deployment. The company’s reactive measures, such as restricting user registrations and patching vulnerabilities post-discovery, highlighted a lack of preparedness and a reactive rather than proactive security culture.

The rapid emergence of these issues indicated that DeepSeek’s focus was primarily on achieving technical performance and market competitiveness, with cybersecurity, AI governance, and safety considered secondary, compromising reliability and overall trustworthiness. DeepSeek is however not the first and very likely not the last to exhibit such lax. This “move fast, fix later” or “innovate now, secure later” mindset is prevalent in the tech industry, driven by the belief that speed equals competitive advantage—a belief that often overshadows the importance of long-term security and ethical considerations[20].

When comparing DeepSeek-R1’s trajectory to the early days of OpenAI’s ChatGPT-3.5 (launched in December 2022), Google Bard, and Microsoft Copilot, similar patterns emerge. In a routine Internet scan for misconfigured systems in September 2023, Wiz Research found “a security misstep at Microsoft that caused exposure of 38 terabytes of private data during a routine open source AI training material update on Github”[21]. In March 2023, OpenAI reported a data leak that lasted for approximately nine hours, resulting in the exposure of approximately 1.2 percent of ChatGPT Plus users’ data, including names, chat histories, email addresses, and payment information[22]. In July 2023, a senior developer discovered ChatGPT was storing user conversation in plain text format in a non-protected location, which was acknowledged by OpenAI[23]. Initial versions of ChatGPT-3.5 could produce biased or inappropriate content[24], [25], [26]. Google Bard, launched in 2023, faced criticism for factual inaccuracies shortly after its release, with a public demonstration error causing a significant drop in Google’s stock price[27], [28]. Despite Google’s strong engineering culture, the rush to compete with ChatGPT led to a product that was not fully ready for public use. Similarly, Microsoft Copilot and Bing Chat have been faced with prompt injection vulnerabilities since its early versions[29], [30].

The AI landscape, like many earlier emerging technologies, is characterized by intense competition, with companies vying to release cutting-edge models and capture market share. Investors often prioritize rapid innovation and growth, incentivizing companies to focus on development speed over comprehensive safety measures. Integrating robust security, privacy, and ethical safeguards into sophisticated AI models requires significant resources and expertise, which can be challenging to implement quickly. As an emerging technology, few regulators and risk managers are ready to address the underlying risks. The drive to innovate and capture market share often overshadows the critical need for comprehensive safety measures.

“Those who cannot remember the past are condemned to repeat it.”, George Santayana

The prioritization of speed over security has been a persistent pattern throughout software development history, not just in AI. From the early Internet era of the 1990s-2000s where basic protocols lacked encryption and led to worm attacks, through the web applications boom of the 2000s-2010s that saw major data breaches at companies like Yahoo, Equifax, LinkedIn, Adobe, Marriott, Alibaba, Sina Weibo, and many others to the mobile app explosion and cloud computing era where rushed development resulted in insecure APIs and poor security configurations[31]. Even IoT devices continue to face basic security issues like hardcoded credentials and use of unpatched vulnerable codebase. In many instances, plain ignorance or a mindset of luck may have reinforced the prevailing attitude that prioritizes speed over security. This pattern has consistently followed a predictable cycle: rapid innovation drives development, security incidents expose vulnerabilities, public backlash forces change, regulatory bodies step in, and finally the industry matures by establishing best practices and compliance standards.

The fact that organizations were often prompt in responding and able to address the issues quickly with or without temporary workarounds upon an incident indicate that they possess the capacity and capability to focus on what truly matters when there is a call to action. Either being ignorant or overly optimistic, many organizations however remain reactive when it comes to safety and security.

Repeated incidents of security breaches, data leaks, and unethical outputs erode public trust in AI technologies, hindering their adoption and acceptance. When customers are affected, they will question their choice of providers and look for alternatives. Vulnerabilities in AI models can be exploited for malicious purposes, leading to cybersecurity threats, data privacy violations, and the spread of misinformation. Biased or toxic outputs from AI models can perpetuate discrimination, reinforce harmful stereotypes, and undermine ethical standards[32].

These problems reinforce the need for AI developers, as well as other software developers, to prioritize security, data privacy, and ethical considerations from the outset, rather than as afterthoughts. Collaboration between AI engineers, security teams, and regulatory bodies is crucial for ensuring comprehensive safeguards. Continuous monitoring, regular updates, and proactive engagement with cybersecurity experts are essential for staying ahead of emerging threats. Establishing clear ethical guidelines, transparency standards, and accountability mechanisms is necessary for fostering responsible AI development and deployment.

Despite these exposures, numerous companies have reported integrating DeepSeek to strengthen digital security and promote innovation in data-driven business operations. Ironically, several companies have expressed that DeepSeek’s cost-effectiveness and powerful reasoning abilities offer new hope for improving AI capabilities in cybersecurity. Companies are incorporating DeepSeek into their platforms to enhance security operations, threat detection, and data protection. The integration of DeepSeek also extends to various sectors including office and education, government, and healthcare, with the model being adopted by entities to improve services and efficiency[33], [34]. Cautious observers however pointed out that “AI capabilities may be overestimated, large models like DeepSeek have limitations, enterprise applications face challenges in costs, data security, and technology adaptation, and future development still needs observation”[35]. In any case, we can only hope that companies deploy adequate security controls and safeguards against the known exposures and vulnerabilities beneath the intelligence that DeepSeek offers.

From its founding in 2023 with the ambition of exploring Artificial General Intelligence (AGI) to its crowning as a significant disruptor in the AI landscape, and the unveiling of its model’s security, privacy, and safety exposures, DeepSeek’s journey underscores both the promise and the perils of accelerated AI development. DeepSeek’s ascent and the subsequent revelations of its weaknesses serve as a cautionary tale for the AI industry. While its technical innovations and cost-effective approaches have disrupted the field, they also highlight the need for greater attention to security, ethics, and transparency in AI development. The AI industry needs to embrace these changes to build more resilient, trustworthy, and ethical systems that benefit society as a whole.

References