Content Moderation
Ongoing coverage on Content Moderation.
Latest coverage
Oversight Board tells Meta expanding Community Notes outside of US poses 'significant' risks
Meta's Oversight Board has advised caution regarding the expansion of its Community Notes feature outside the United States, citing significant human rights risks and potential for tangible harms. The board recommended withholding the feature in countries with high polarization, ongoing conflict, or organized disinformation networks, suggesting it can coexist with traditional fact-checking partnerships.
Tomodachi Life: Living the Dream has absolutely no profanity filter
The demo for Tomodachi Life: Living the Dream on Nintendo Switch features no profanity filter, allowing players to freely name their islands. The game is fully offline, meaning custom names will not be shared with other players. The full game is set to release next month.
Morning Consolidation
Sony has removed approximately 1,000 games from the PlayStation Store that were deemed to be of low quality. This action is part of an effort to curate the digital storefront and improve the overall user experience.
Steam Faces Major Backlash Over War Crime Game That Slipped Through Content Filters
Steam is facing significant backlash for hosting the game "Ukrainian Warfare: Gostomel Heroes," which depicts events in locations where documented war crimes occurred during the ongoing conflict. The platform's content moderation system is criticized for allowing the game, which has been banned in Ukraine and Germany, while dismissing user reports as mere feedback. This controversy highlights issues of platform accountability and the handling of sensitive real-world events in gaming.
AI Yi-Yi!
TikTok has removed AI-generated videos depicting sexualized Black women following a BBC investigation. Separately, XDA reports that AI agents pose a security risk for home labs, with Tailscale releasing a fix for related vulnerabilities.
Louisiana man charged with possessing CSAM is not a Roblox employee, but he did make Roblox games
A Louisiana man, Jamie Borne, was arrested and charged with possession of CSAM and owning a child sex doll. While initial reports suggested he was a Roblox programmer, Roblox Corp clarified that Borne was not an employee but a user who created games on the platform. Roblox has deactivated his accounts and experiences, and the incident has reignited discussions about the platform's child safety standards amidst ongoing lawsuits from state attorneys general.
Meta will move away from human content moderators in favor of more AI
Meta plans to significantly reduce its reliance on human content moderators over the next few years, transitioning to AI-based systems for content review. While humans will still handle critical decisions and training, the company believes AI can identify issues faster and more comprehensively, supporting a wider range of languages. This shift aims to improve efficiency and potentially reduce costs, though it may alter user perception of policy enforcement.
Xbox vs PlayStation vs Nintendo: Which Platform Is Actually Safer?
This analysis compares the online safety infrastructure and transparency of Xbox, PlayStation, and Nintendo platforms. Xbox is highlighted for its comprehensive, documented safety measures and detailed transparency reports, while PlayStation shows progress with its first report but lacks data depth. Nintendo employs a restrictive-by-default model, prioritizing child safety but offering no public data, creating an accountability gap.
UK fines 4chan nearly $700,000 for failing its online safety act obligations
The UK's Ofcom has fined the website 4chan a total of £520,000 ($690,000) for failing to comply with its obligations under the Online Safety Act 2023. Penalties were issued for issues including the failure to prevent children from encountering pornography, insufficient illegal content risk assessment, and inadequate terms of service regarding user protection.
OpenAI Restricts ChatGPT Adult Mode to Text-Only After Age-Check Concerns
OpenAI has restricted its planned ChatGPT adult mode to text-only conversations due to concerns about its age-verification system's reliability in identifying underage users. The company removed support for image, voice, and video generation, citing safety and regulatory compliance as priorities. This decision follows internal pushback and contrasts with competitors exploring more advanced R-rated content generation.
OpenAI Restricts ChatGPT Adult Mode to Text-Only After Age-Check Concerns
OpenAI has restricted its planned ChatGPT "adult mode" to text-only conversations due to concerns about its age-verification system's reliability. The company decided to remove image, voice, and video generation capabilities after internal reports indicated the system misidentified underage users approximately 12% of the time. This move contrasts with competitors like xAI and prioritizes safety and regulatory compliance over feature expansion, delaying the feature's launch.
OpenAI's adult mode reportedly won't generate pornographic audio, images or video
OpenAI's upcoming 'adult mode' for ChatGPT will permit lewd conversations but will not generate explicit audio, images, or video. The feature faced internal debate regarding potential user emotional dependence and risks to minors, with age verification technology showing a significant error rate.
The Oversight Board says Meta needs new rules for AI-generated content
The Oversight Board is urging Meta to establish new, separate rules for AI-generated content, distinct from its misinformation policy. The board recommends investing in better detection tools, utilizing digital watermarks, and creating clearer guidelines for labeling AI content, especially in light of a viral AI-generated video that spread misinformation during a conflict.
You can (sort of) block Grok from editing your uploaded photos
X, formerly Twitter, has introduced a feature allowing users to prevent Elon Musk's xAI chatbot, Grok, from editing their uploaded photos. This comes after Grok generated millions of inappropriate images, including those of children, leading to regulatory investigations in the EU. While the feature offers a basic level of control, critics argue it is a superficial gesture with easy workarounds, and that xAI should implement more robust safety measures or disable image generation until the issues are resolved.
Roblox introduces real-time AI-powered chat rephraser for inappropriate language
Roblox has implemented a new AI-powered feature that rephrases inappropriate language in real-time, replacing censored text with more suitable alternatives instead of hash signs. This system, initially targeting profanity, aims to improve chat civility and user experience, with penalties still in place for policy violations. The feature is available for age-verified users in similar age groups across all supported languages.
X to require AI labels on armed conflict videos from paid creators, citing ‘times of war’
X will suspend creators from its revenue sharing program if they post AI-generated videos depicting armed conflicts without disclosure. This policy, announced by Head of Product Nikita Bier, applies only to monetized accounts and AI-generated conflict videos, with violations flagged by Community Notes or AI metadata detection. The platform is also testing a broader AI labeling toggle for all posts.
Neopets TTRPG Playtest Material Pulled for Controversial Material
A playtest document for the Neopets tabletop role-playing game, developed by Geekify, was removed at the request of Neopets due to controversial and inappropriate content. The material, which included themes of intoxication, torture, and sexual activity, did not align with Neopets' brand standards and was released without their approval. Despite the controversy, development of the TTRPG is expected to continue.
Australian government calls for action from Roblox on "untenable" child safety concerns
The Australian Government, through Communications Minister Anika Wells and eSafety Commissioner Julie Inman Grant, has formally requested Roblox address serious concerns regarding child exploitation, grooming, and exposure to harmful content on its platform. Roblox has responded by detailing its safety policies and cooperation with law enforcement, emphasizing its ongoing commitment to user protection.
