兒童安全人工智慧：保護年輕用戶免受痛苦或傷害的框架

劍橋大學的一項新研究提出了一個兒童安全人工智慧 (AI) 框架，此前兒童將聊天機器人視為準人類且值得信賴。

A new University of Cambridge study has proposed a framework for child-safe artificial intelligence (AI) following incidents where children saw chatbots as quasi-human and trustworthy, leaving researchers concerned about them experiencing distress or harm.

劍橋大學的一項新研究提出了一個兒童安全人工智慧（AI）框架，此前兒童將聊天機器人視為準人類且值得信賴，這讓研究人員擔心他們會遭受痛苦或傷害。

 

 

AI chatbots have frequently shown signs of an “empathy gap”, researchers argue, and this gap puts young users at risk of distress or harm, raising the urgent need for “child-safe AI.”

研究人員認為，人工智慧聊天機器人經常表現出「同理心差距」的跡象，這種差距使年輕用戶面臨痛苦或傷害的風險，從而迫切需要「兒童安全的人工智慧」。

 

 

Dr. Nomisha Kurian, who led the research, has urged developers and policymakers to prioritize approaches to AI design that take greater account of children’s needs.

領導這項研究的 Nomisha Kurian 博士敦促開發者和政策制定者優先考慮更多考慮兒童需求的人工智慧設計方法。

 

 

Children, she argues, are particularly susceptible to treating chatbots as lifelike, quasi-human confidantes, and as a result their interactions with the technology can go awry when it fails to respond to their unique needs and vulnerabilities.

她認為，兒童特別容易將聊天機器人視為栩栩如生、準人類的知己，因此，當科技無法回應他們獨特的需求和弱點時，他們與科技的互動可能會出錯。

 

 

The study links that gap in understanding to recent cases in which interactions with AI led to potentially dangerous situations for young users. They include an incident in 2021, when Amazon’s AI voice assistant, Alexa, instructed a 10-year-old to touch a live electrical plug with a coin. Last year, Snapchat’s My AI gave adult researchers posing as a 13-year-old girl tips on how to lose her virginity to a 31-year-old.

該研究將這種理解上的差距與最近的案例聯繫起來，在這些案例中，與人工智慧的互動導致年輕用戶面臨潛在的危險情況。其中包括 2021 年發生的事件，當時亞馬遜的人工智慧語音助理 Alexa 指示一名 10 歲的孩子用硬幣觸摸帶電的電源插頭。去年，Snapchat 的 My AI 為冒充 13 歲女孩的成年研究人員提供如何向 31 歲的女孩失去童貞的提示。

 

 

Both companies responded by implementing safety measures, but the study says there is also a need to be proactive in the long-term to ensure that AI is child-safe. It offers a 28-item framework to help companies, teachers, school leaders, parents, developers and policy actors think systematically about how to keep younger users safe when they “talk” to AI chatbots.

兩家公司都採取了安全措施來應對，但研究表明，從長遠來看，還需要積極主動，以確保人工智慧對兒童安全。它提供了一個包含 28 項的框架，幫助公司、教師、學校領導、家長、開發人員和政策制定者係統地思考如何在年輕用戶與人工智慧聊天機器人「交談」時確保他們的安全。

 

 

“Children are probably AI’s most overlooked stakeholders,” Dr. Kurian said. “Very few developers and companies currently have well-established policies on child-safe AI. That is understandable because people have only recently started using this technology on a large scale for free. But now that they are, rather than having companies self-correct after children have been put at risk, child safety should inform the entire design cycle to lower the risk of dangerous incidents occurring.”

「兒童可能是人工智慧最被忽視的利益相關者，」庫里安博士說。 「目前很少有開發商和公司擁有關於兒童安全人工智慧的完善政策。這是可以理解的，因為人們最近才開始大規模地免費使用這項技術。但既然如此，與其讓公司在兒童面臨危險後自行糾正，不如將兒童安全納入整個設計週期，以降低發生危險事件的風險。

 

 

Dr Kurian’s study examined cases where the interactions between AI and children, or adult researchers posing as children, exposed potential risks. It analysed these cases using insights from computer science about how the large language models (LLMs) in conversational generative AI function, alongside evidence about children’s cognitive, social and emotional development.

庫里安博士的研究調查了人工智慧與兒童或冒充兒童的成年研究人員之間的互動暴露潛在風險的案例。它利用電腦科學關於對話生成人工智慧中的大型語言模型 (LLM) 如何發揮作用的見解，以及有關兒童認知、社交和情感發展的證據來分析這些案例。

 

 

LLMs have been described as “stochastic parrots“: a reference to the fact that they use statistical probability to mimic language patterns without necessarily understanding them. A similar method underpins how they respond to emotions.

法學碩士被描述為「隨機鸚鵡」：指的是他們使用統計機率來模仿語言模式，但不一定理解它們。他們如何面對情緒也採用了類似的方法。

 

 

This means that even though chatbots have remarkable language abilities, they may handle the abstract, emotional and unpredictable aspects of conversation poorly; a problem that Dr. Kurian characterizes as their “empathy gap.”

這意味著，儘管聊天機器人具有出色的語言能力，但它們可能無法很好地處理對話中的抽象、情感和不可預測的方面；庫里安博士將這個問題描述為「同理心差距」。

 

 

They may have particular trouble responding to children, who are still developing linguistically and often use unusual speech patterns or ambiguous phrases. Children are also often more inclined than adults to confide sensitive personal information.

他們在回應兒童時可能會遇到特別困難，因為兒童的語言仍在發展，並且經常使用不尋常的言語模式或模棱兩可的短語。兒童通常比成人更傾向於透露敏感的個人資訊。

 

 

Despite this, children are much more likely than adults to treat chatbots as though they are human. Recent research found that children will disclose more about their own mental health to a friendly-looking robot than to an adult. Dr. Kurian’s study suggests that many chatbots’ friendly and lifelike designs similarly encourage children to trust them, even though AI may not understand their feelings or needs.

儘管如此，兒童比成人更有可能將聊天機器人視為人類。最近的研究發現，與成人相比，孩子們更願意向看起來友善的機器人透露更多關於自己心理健康的資訊。庫里安博士的研究表明，許多聊天機器人友好且逼真的設計同樣會鼓勵孩子們信任它們，儘管人工智慧可能無法理解他們的感受或需求。

 

 

“Making a chatbot sound human can help the user get more benefits out of it,” Dr. Kurian said. “But for a child, it is very hard to draw a rigid, rational boundary between something that sounds human, and the reality that it may not be capable of forming a proper emotional bond.”

「讓聊天機器人聽起來更人性化可以幫助用戶從中獲得更多好處，」庫里安博士說。 “但對於一個孩子來說，很難在聽起來像人類的東西和它可能無法形成適當的情感紐帶的現實之間劃定嚴格、理性的界限。”

 

 

Access the findings, including the 28 recommendations, using this link.

使用此連結存取調查結果，包括 28 個建議。