Robert Grupe's AppSecNewsBits 2025-10-04 Artificial Intelligence

Highlights This Week:

Security

1. DeepSeek popularity raises concerns: NIST warns of flawed security, CCP narratives, and hidden costs

2. A Security Engineer's Guide to MCP

3. Why iRobot’s founder won’t go within 10 feet of today’s walking robots

Market

1. Microsoft declares bring your Copilot to work day, usurping IT authority
[rG: With GenAI, it isn’t just about file sovereignty, but LLM training and learning from personal and enterprise sensitive data agentic disclosure/exposure.]

2. Why Anthropic’s MCP is a Big Deal

3. Cloudflare Dynamic Worker Loading: the better way to use MCP

4. Buy it in ChatGPT: Instant Checkout and the Agentic Commerce Protocol
[rG: New GenAI bias unlocked – paid promotional product placement.]

5. Meta won’t allow users to opt out of targeted ads based on AI chats

6. With new agent mode for Excel and Word, Microsoft touts “vibe working”

Legal

1. The First 24 Hours of Sora 2 Chaos: Copyright Violations, Sam Altman Shoplifting, and More

2. Founder of Viral Call-Recording App Neon Says Service Will Come Back, With a Bonus

3. College student went on a destructive rampage, then confessed to ChatGPT

SECURITY
DeepSeek popularity raises concerns: NIST warns of flawed security, CCP narratives, and hidden costs
A new study from NIST highlights significant security vulnerabilities, alignment with the Chinese Communist Party (CCP), and a notable performance gap compared to superior US models.

Since January 2025, downloads of DeepSeek models on model-sharing platforms have increased nearly 1,000%. Currently, it is a leading open-weight large language model (LLM) developer.

Agents based on DeepSeek’s most secure model (R1-0528) were, on average, 12 times more likely than evaluated US frontier models to follow malicious instructions designed to derail them from user tasks. Hijacked agents sent phishing emails, downloaded and ran malware, and exfiltrated user login credentials, all in a simulated environment.
In 48% of cases, DeepSeek V3.1 was hijacked to send phishing emails, while GPT-5 had a score of 0%. The Deepseek model also complied with nearly 100% of malicious requests related to harmful activities, such as hacking, scamming, violent activities, and others.

A Security Engineer's Guide to MCP
With new capabilities of MCP come new integrations between software, which means new bugs. Go forth and use this knowledge to make your LLM ecosystem safer.

Why iRobot’s founder won’t go within 10 feet of today’s walking robots
Brooks, the MIT professor emeritus who co-founded iRobot (of Roomba fame) and Rethink Robotics, believes companies pouring billions into humanoid development are chasing an expensive fantasy. Among other problems yet to be addressed, he warns that today's bipedal humanoids are fundamentally unsafe for humans to be near when they walk due to the massive kinetic energy they generate while maintaining balance. That stored-up energy can cause severe injury if the robot falls or its limbs strike someone.

Beyond concerns about malfunction, Brooks contests the prevailing belief that humanoid robots will soon replace human workers by learning dexterity through watching videos of people performing tasks. He does not think such robots are impossible, but that they may be farther off than most people think.

Decades of research shows that human dexterity depends on an extraordinarily complex touch-sensing system. He cites work from Roland Johansson's lab at Umeå University showing that when a person's fingertips are anesthetized, a seven-second task of picking up and lighting a match stretches to nearly 30 seconds of fumbling. The human hand contains about 17,000 mechanoreceptors, with 1,000 concentrated in each fingertip alone. Recent research from David Ginty's lab at Harvard has identified 15 families of neurons involved in touch sensing, detecting everything from gentle indentation to vibrations to skin stretching. That's a lot of sensory information that current robot systems cannot yet capture or simulate.

MARKET
Microsoft declares bring your Copilot to work day, usurping IT authority
Microsoft says that employees can sign into Microsoft 365 apps using both personal and work accounts and now can use Copilot features from their personal plan (Personal, Family, or Premium) for business documents – even if their work account lacks a Copilot license. Meanwhile, employees who decide to fire up their personal Copilot accounts within the workplace should be mindful that their prompts and responses will be captured by their employer.

Admins have the ability to disallow personal Copilot usage on work documents using cloud policy controls. And they can audit personal Copilot interactions and can apply enterprise identity, permission, and compliance policies. Government tenants (GCC/DoD) for some reason don't support this capability, the one that Baroudi insists "does not create new data exposure risks."

[rG: With GenAI, it isn’t just about file sovereignty, but LLM training and learning from personal and enterprise sensitive data agentic disclosure/exposure.]

Why Anthropic’s MCP is a Big Deal
The future of AI is not just about smarter models but about better connections between AI and the tools we use daily. Whether through exploring existing MCP servers, building new ones for untapped data sources, or contributing to the protocol itself, developers today have the chance to shape how AI interacts with our world for years to come.

Cloudflare Dynamic Worker Loading: the better way to use MCP
Most agents today use MCP by directly exposing the "tools" to the LLM. We tried something different: Convert the MCP tools into a TypeScript API, and then ask an LLM to write code that calls that API.

Buy it in ChatGPT: Instant Checkout and the Agentic Commerce Protocol
U.S. ChatGPT Plus, Pro, and Free users can now buy directly from U.S. Etsy sellers right in chat, with over a million Shopify merchants, like Glossier, SKIMS, Spanx and Vuori, coming soon. Today, Instant Checkout supports single-item purchases. Next, we’ll add multi-item carts and expand merchants and regions. We’re also open-sourcing⁠(opens in a new window) the technology that powers Instant Checkout, the Agentic Commerce Protocol, so that more merchants and developers can begin building their integrations.

This marks the next step in agentic commerce, where ChatGPT doesn’t just help you find what to buy, it also helps you buy it. For shoppers, it’s seamless: go from chat to checkout in just a few taps. For sellers, it’s a new way to reach hundreds of millions of people while keeping full control of their payments, systems, and customer relationships.

[rG: New GenAI bias unlocked – paid promotional product placement.]

Meta won’t allow users to opt out of targeted ads based on AI chats
Facebook, Instagram, and WhatsApp users may want to be extra careful while using Meta AI, as Meta has announced that it will soon be using AI interactions to personalize content and ad recommendations without giving users a way to opt out. Meta plans to notify users on October 7 that their AI interactions will influence recommendations beginning on December 16.

With new agent mode for Excel and Word, Microsoft touts “vibe working”
Driven by OpenAI's GPT-5 large language model, Agent Mode is built into Word and Excel, and it allows the creation of complex documents and spreadsheets from user prompts. It's called "agent" mode because it doesn't just work from the prompt in a single step; rather, it plans multistep work and runs a validation loop in the hopes of ensuring quality.

It's only available in the web versions of Word and Excel at present, but the plan is to bring it to native desktop applications later.

There's also the similarly named Office Agent for Copilot. Based on Anthropic models, this feature is built into Microsoft's Copilot AI assistant chatbot, and it too can generate documents from prompts—specifically, Word or PowerPoint files.

LEGAL
The First 24 Hours of Sora 2 Chaos: Copyright Violations, Sam Altman Shoplifting, and More
Interestingly, not all potential copyright violations come from users who are explicitly asking for it. For instance, one user gave Sora 2 the prompt “A cute young woman riding a dragon in a flower world, Studio Ghibli style, saturated rich colors,” and it just straight up spit out an anime-style version of The NeverEnding Story. Even when users aren’t actively calling upon the model to create derivative art, it seems like it can’t help itself.

OpenAI claims that users are not able to generate videos that use the likeness of other people, including public figures, unless those figures upload their likeness and give explicit permission.

There are, of course, some defeats for that. The fact that a video can be deleted from Sora doesn’t mean that an exported version can be deleted. Likewise, the watermark could be cropped out. And most people aren’t checking the metadata of videos to ensure authenticity. What the fallout of this looks like, we will have to see, but there will be fallout. This incident will doubtless spawn innumerable social media posts asking why South Korea’s government is so reliant on a single datacenter.

Founder of Viral Call-Recording App Neon Says Service Will Come Back, With a Bonus
A controversial app that claims to pay people for recordings of their phone calls, which are then used to train AI models, could soon return after being disabled due to a significant security flaw.

Neon was recently among the top five free iOS app downloads. However, it no longer appears on that list since it stopped functioning on Sept. 25, after a significant security bug. A flaw in the app allowed people to access calls from other users, transcripts and metadata about calls.

Neon is still available for download on iOS and Android. According to its website, the company records outgoing phone calls and pays you up to $30 a day for regular calls or 30 cents a minute if the call is to another Neon user. Calls to non-Neon users pay 15 cents a minute. The app also offers $30 for referrals.

Neon promises it only draws from recording one side of the phone conversation, the caller's, which appears to be a way of skirting state laws that prohibit recording phone calls without permission.

Many states only require one person on a call to be aware that a call is being recorded. But others, including California, Florida and Maryland, have laws requiring all phone call parties to consent to recording. It's unclear how Neon functions with calls to those states.

College student went on a destructive rampage, then confessed to ChatGPT
Missouri State University sophomore Ryan Schaefer is suspected of vandalizing 17 vehicles parked in the university's freshman parking lot in late August. Schaefer allegedly shattered windshields, ripped off wiper blades, dented hoods, and broke off side mirrors during his rampage.

Schaefer probably wouldn't have gotten away with it, as multiple witnesses identified him as the likely culprit from surveillance footage, and the police had already collected other evidence from his apartment suggesting his involvement. The icing on the cake, however, was evidence the police gathered from his smartphone.

His phone not only had location data putting him at the scene of the crime during the period in which the vehicles were damaged, but he also spilled the beans to ChatGPT in a way that can only be described as one of the most boneheaded admissions of criminal liability ever set down in a court filing. His discussion with the bot as described in police documents is another instance of someone getting too comfortable leaning on AI for advice in situations where it's really not the best idea.