In a digital environment full of risks, artificial intelligence is emerging as a key tool for protecting children online. We explore global case studies, research, and the ethical dilemmas surrounding this technology.
In the 21st century, childhood no longer takes place only on playgrounds—it unfolds on screens. Children spend hours online every day—playing games, learning, socializing. While technology opens the doors to knowledge and creativity, it also introduces serious risks: cyberbullying, online predators, exposure to inappropriate content, and digital manipulation.
In this complex landscape, artificial intelligence (AI) is increasingly stepping into the role of protector—but only if used carefully, ethically, and with proper oversight.
AI on the Front Line of Defense
On platforms like YouTube Kids and TikTok, AI algorithms automatically detect and remove content deemed inappropriate for children using machine learning and computer vision. However, as Dr. Hany Farid, professor of digital forensics at UC Berkeley and a pioneer in combating child exploitation online, emphasizes:
“Algorithms are powerful, but not infallible—they require constant human oversight and adaptation.”
On educational platforms such as Khan Academy and Google for Education, AI is used to detect potentially harmful communication between students and to alert educators, in line with privacy regulations.
Behavioral Analytics: A Help or a Breach of Privacy?
More advanced systems utilize behavioral analytics to identify patterns that may indicate risky behaviors—such as depression, anxiety, or exposure to abuse. For example, Bark Technologies employs AI to monitor messages, emails, and online activity, alerting parents to potentially harmful situations.
But how far is too far?
Dr. Sonia Livingstone, professor of psychology at the London School of Economics and a leading researcher in the field of children’s digital rights, warns:
“Children have the right to safety, but also to privacy. The balance between protection and control is delicate, and AI systems often cross that line without sufficient transparency.”
The Legal Framework—and Its Gaps
In Europe, the Digital Services Act (DSA) and the General Data Protection Regulation (GDPR) provide a framework for responsible use of AI in protecting children’s data. In the United States, the Children’s Online Privacy Protection Act (COPPA) safeguards those under 13 by restricting data collection.
Yet, AI technologies are evolving faster than legislation. A 2021 UNICEF report warns that “the global legislative infrastructure is failing to keep up with the development of automated decision-making that directly affects children’s lives.”
AI Tools in Parents’ Hands—With Caution
Today, parents can use various AI-powered apps such as Qustodio, Net Nanny, and the aforementioned Bark, which offer detailed insights into children’s online activities along with recommendations for preventive measures.
However, OECD research shows that parents often lack proper understanding of how AI works, which can lead to excessive monitoring and erosion of trust between parents and children.
Global Recommendation: AI with Human Ethics
A 2023 UNESCO report states: “AI in child protection must be developed in accordance with human rights principles, transparency, and active child participation in decisions that affect them.”
Organizations like the 5Rights Foundation and the Center for Humane Technology advocate for the implementation of so-called “design ethics,” where products and algorithms are developed with consideration for age, cognitive development, and a child’s right to privacy.
Conclusion: AI Is Not a Guardian—But It Can Be an Ally
Artificial intelligence is not a substitute for parenting, education systems, or the law. But with clear guidelines, ethical design, and cross-sector collaboration, it can become a powerful ally in building a digital world where children are not just users—but safe, respected, and empowered participants.
As Dr. Farid concluded at the Global Forum on Digital Child Safety:
“AI can help. But it must serve the child—not the market.”
Autor: Olivera Krčadinac