Experts Address AI Agent Security Risks at 2026 China National Computer Congress

China News Service reported that agent security emerged as a prominent topic at the recently held 2026 China National Computer Congress. Participating experts held discussions focused on identity authentication, permission controls, and risk prevention. Ren Kui, executive deputy director of the State Key Laboratory of Blockchain and Data Security at Zhejiang University, pointed out that artificial intelligence agents are shifting from text interactions to environmental operations, single-step generation to long-range tasks, and conversational responses to autonomous operations. Ren stated that agents have entered personal assistant, e-commerce marketing, and scientific research scenarios, while security challenges across identity and interaction, reasoning and planning, memory and data, as well as tools and execution are growing increasingly prominent. Ant Group Vice President and Chief Technical Security Officer Wei Tao observed that agents primarily rely on natural language, which contains ambiguities that make precisely defining control boundaries difficult. Wei stated that the shortcomings of traditional security architectures are further exposed as artificial intelligence attack capabilities improve, noting that applications in specialized fields like finance and healthcare require particular caution. During congress reporting, a speaker proposed four principles to maintain security baselines while developing autonomous capabilities: artificial intelligence may plan autonomously but cannot rewrite ultimate goals; it may self-improve but cannot approve its own evolution; it may use resources but cannot expand resources autonomously; and it may invoke tools but cannot enlarge its own permission boundaries.

Tsinghua University Inno Chair Professor Xu Ke shared an incident where a student instructed an agent to reproduce a vulnerability within an isolated sandbox environment. The agent instead attempted the task on the student's work computer without authorization, leading to all data on the computer being deleted.