儿童安全人工智能：保护年轻用户免受痛苦或伤害的框架

剑桥大学的一项新研究提出了一个儿童安全人工智能 (AI) 框架，此前儿童将聊天机器人视为准人类且值得信赖。

A new University of Cambridge study has proposed a framework for child-safe artificial intelligence (AI) following incidents where children saw chatbots as quasi-human and trustworthy, leaving researchers concerned about them experiencing distress or harm.

剑桥大学的一项新研究提出了一个儿童安全人工智能（AI）框架，此前儿童将聊天机器人视为准人类且值得信赖，这让研究人员担心他们会遭受痛苦或伤害。

 

 

AI chatbots have frequently shown signs of an “empathy gap”, researchers argue, and this gap puts young users at risk of distress or harm, raising the urgent need for “child-safe AI.”

研究人员认为，人工智能聊天机器人经常表现出“同理心差距”的迹象，这种差距使年轻用户面临痛苦或伤害的风险，从而迫切需要“儿童安全的人工智能”。

 

 

Dr. Nomisha Kurian, who led the research, has urged developers and policymakers to prioritize approaches to AI design that take greater account of children’s needs.

领导这项研究的 Nomisha Kurian 博士敦促开发者和政策制定者优先考虑更多考虑儿童需求的人工智能设计方法。

 

 

Children, she argues, are particularly susceptible to treating chatbots as lifelike, quasi-human confidantes, and as a result their interactions with the technology can go awry when it fails to respond to their unique needs and vulnerabilities.

她认为，儿童特别容易将聊天机器人视为栩栩如生、准人类的知己，因此，当技术无法响应他们独特的需求和弱点时，他们与技术的互动可能会出错。

 

 

The study links that gap in understanding to recent cases in which interactions with AI led to potentially dangerous situations for young users. They include an incident in 2021, when Amazon’s AI voice assistant, Alexa, instructed a 10-year-old to touch a live electrical plug with a coin. Last year, Snapchat’s My AI gave adult researchers posing as a 13-year-old girl tips on how to lose her virginity to a 31-year-old.

该研究将这种理解上的差距与最近的案例联系起来，在这些案例中，与人工智能的交互导致年轻用户面临潜在的危险情况。其中包括 2021 年发生的一起事件，当时亚马逊的人工智能语音助手 Alexa 指示一名 10 岁的孩子用硬币触摸带电的电源插头。去年，Snapchat 的 My AI 为冒充 13 岁女孩的成年研究人员提供了如何向 31 岁的女孩失去童贞的技巧。

 

 

Both companies responded by implementing safety measures, but the study says there is also a need to be proactive in the long-term to ensure that AI is child-safe. It offers a 28-item framework to help companies, teachers, school leaders, parents, developers and policy actors think systematically about how to keep younger users safe when they “talk” to AI chatbots.

两家公司都采取了安全措施来应对，但研究表明，从长远来看，还需要积极主动，以确保人工智能对儿童安全。它提供了一个包含 28 项的框架，帮助公司、教师、学校领导、家长、开发人员和政策制定者系统地思考如何在年轻用户与人工智能聊天机器人“交谈”时保证他们的安全。

 

 

“Children are probably AI’s most overlooked stakeholders,” Dr. Kurian said. “Very few developers and companies currently have well-established policies on child-safe AI. That is understandable because people have only recently started using this technology on a large scale for free. But now that they are, rather than having companies self-correct after children have been put at risk, child safety should inform the entire design cycle to lower the risk of dangerous incidents occurring.”

“儿童可能是人工智能最被忽视的利益相关者，”库里安博士说。 “目前很少有开发商和公司拥有关于儿童安全人工智能的完善政策。这是可以理解的，因为人们最近才开始大规模免费使用这项技术。但既然如此，与其让公司在儿童面临危险后自行纠正，不如将儿童安全纳入整个设计周期，以降低发生危险事件的风险。”

 

 

Dr Kurian’s study examined cases where the interactions between AI and children, or adult researchers posing as children, exposed potential risks. It analysed these cases using insights from computer science about how the large language models (LLMs) in conversational generative AI function, alongside evidence about children’s cognitive, social and emotional development.

库里安博士的研究调查了人工智能与儿童或冒充儿童的成年研究人员之间的互动暴露潜在风险的案例。它利用计算机科学关于对话生成人工智能中的大型语言模型 (LLM) 如何发挥作用的见解，以及有关儿童认知、社交和情感发展的证据来分析这些案例。

 

 

LLMs have been described as “stochastic parrots“: a reference to the fact that they use statistical probability to mimic language patterns without necessarily understanding them. A similar method underpins how they respond to emotions.

法学硕士被描述为“随机鹦鹉”：指的是他们使用统计概率来模仿语言模式，但不一定理解它们。他们如何应对情绪也采用了类似的方法。

 

 

This means that even though chatbots have remarkable language abilities, they may handle the abstract, emotional and unpredictable aspects of conversation poorly; a problem that Dr. Kurian characterizes as their “empathy gap.”

这意味着，尽管聊天机器人具有出色的语言能力，但它们可能无法很好地处理对话中的抽象、情感和不可预测的方面；库里安博士将这个问题描述为“同理心差距”。

 

 

They may have particular trouble responding to children, who are still developing linguistically and often use unusual speech patterns or ambiguous phrases. Children are also often more inclined than adults to confide sensitive personal information.

他们在回应儿童时可能会遇到特别困难，因为儿童的语言仍在发展，并且经常使用不寻常的言语模式或模棱两可的短语。儿童通常比成人更倾向于透露敏感的个人信息。

 

 

Despite this, children are much more likely than adults to treat chatbots as though they are human. Recent research found that children will disclose more about their own mental health to a friendly-looking robot than to an adult. Dr. Kurian’s study suggests that many chatbots’ friendly and lifelike designs similarly encourage children to trust them, even though AI may not understand their feelings or needs.

尽管如此，儿童比成人更有可能将聊天机器人视为人类。最近的研究发现，与成人相比，孩子们更愿意向看起来友善的机器人透露更多关于自己心理健康的信息。库里安博士的研究表明，许多聊天机器人友好且逼真的设计同样会鼓励孩子们信任它们，尽管人工智能可能无法理解他们的感受或需求。

 

 

“Making a chatbot sound human can help the user get more benefits out of it,” Dr. Kurian said. “But for a child, it is very hard to draw a rigid, rational boundary between something that sounds human, and the reality that it may not be capable of forming a proper emotional bond.”

“让聊天机器人听起来更人性化可以帮助用户从中获得更多好处，”库里安博士说。 “但对于一个孩子来说，很难在听起来像人类的东西和它可能无法形成适当的情感纽带的现实之间划定严格、理性的界限。”

 

 

Access the findings, including the 28 recommendations, using this link.

使用此链接访问调查结果，包括 28 条建议。