Reddit is upping its AI game again, this time with the implementation of an LLM-powered harassment filter for the benefit of its army of volunteer moderators.

The existence of the new Harassment Filter was discovered during an APK teardown by Android Authority, which spotted its existence in version 2024.10.0 of the official Reddit app for Android. That version of the official Android Reddit app has appeared in various unofficial sources online, but the Google Play Store still shows 2024.08.0 as the most recent generally available version.

The app contains lines of code referencing a large language model trained to assist moderators in preventing publication of harassing posts, and is backed up by a Reddit help page updated last week that reflects the existence of the new tool.

"The Harassment Filter is an optional community safety setting that lets moderators automatically filter posts and comments that are likely to be considered harassing," the help page reads. "The filter is powered by a Large Language Model (LLM) that's trained on moderator actions and content removed by Reddit's internal tools and enforcement teams."

The filter can be enabled in a Reddit community's mod tools, but individual moderators will need to have permissions to change subreddit settings to enable it.

The harassment filter can be set to low ("filters the least content but with the most accurate results") and high ("filters the most content but may be less accurate"), and also includes an explicit allow list to force the AI to ignore certain keywords, up to 15 of which can be added.

Once enabled, the filter creates a new tag in the moderation queue called "potential harassment," which moderators can review for accuracy.

Reddit's help page says the feature is now available on desktop and the official Reddit apps, though it's not clear when the feature was added. We asked Reddit for more info, but haven't heard back.

More like (A)IPO

Reddit, which is busily planning its IPO, has been doing everything it can to drum up investor support ahead of its imminent stock market debut. As with all things to do with raising cash in 2024, that means sticking AI anywhere it can.

The popular social media platform signed a deal with a previously unknown AI business last month to let its models slurp up user conversations to train products, earning it $60 million for its efforts. While that figure hasn't been explicitly linked to Google, both Reddit and the Chocolate Factory announced an expanded partnership days after news of the AI deal broke.

According to Reddit, the deal gave Google access to its Data API, and through it "an efficient and structured way to access the vast corpus of existing content on Reddit … including supporting new ways to display Reddit content and providing more efficient ways to train models."

Google, meanwhile, said that Reddit will gain the ability to integrate Google's Vertex AI "to enhance search and other capabilities on the Reddit platform" as part of the deal. For those unfamiliar, Vertex is Google's tool for integrating Gemini capabilities into enterprise environments.

How Reddit users - a notoriously unruly bunch - will react to news that AI is being foisted on moderators isn't clear. Redditors haven't necessarily reacted well to the company's planned IPO, with some referring to the plan as a "pump-and-dump" scheme to enrich Reddit leadership at the expense of the site. 

Reddit has experienced the effect of a blowback before, with thousands of the site's subreddits going dark last year in protest of a plan to begin charging developers to use the site's APIs, effectively killing many third-party Reddit clients.

Reddit knows its users are a risk - it even disclosed its concerns in an SEC filing last month regarding its IPO plans. Ostensibly a tool to assist moderators, the addition of AI could also be seen as a subtle threat - protest again and we'll simply replace you volunteers with Gemini, at least until it gets trained on enough Reddit data to start siding with the protesters. ®

https://www.theregister.com//2024/03/07/reddit_introduces_aipowered_harassment_filter/