Content Moderation in Facebook

Workers at a Facebook content moderation center

Content moderation is the process of screening content users post online by applying a set of pre-set rules or guidelines to see if it is appropriate or not . Facebook is a social media platform that allows people to connect with friends, family and communities of people who share common interests . As with many other popular social media platforms, Facebook has come up with an approach to moderate and control the type of content users see and engage with. Facebook has two main approaches when it comes to content moderation; they utilize both AI moderators and human moderators. The way Facebook moderates its content is through its community standards that lay out what they believe each post should follow . There have been many instances where Facebook’s content moderation tactics have been a success, but also many instances where it has failed. The ethics behind Facebook’s content moderation approach has also been widely controversial from the mental health struggles human moderators are forced to deal with to questioning how the AI is trained to flag inappropriate content.

Overview/Background

How content is filtered

When it comes to content moderation, Facebook utilizes both AI moderators as well as human moderators. While a majority of content that is deemed inappropriate is found through the AI moderators, the human moderators are responsible for posts that the AI is not quite sure about and are vital to improving the machine learning model the AI technology uses .

AI Moderators

Facebook filters all posts initially through its AI technology. Facebook’s AI technology starts with building machine learning (ML) models that have the ability to analyze the content of a post or recognize different items in a photo. These models are used to determine whether what the post contains fits within the community guidelines or if there is a need to take action on the post, such as removing it .

Sometimes, the AI technology is unsure if content violates the community guidelines so it will send over the content to the human review teams. Once the review teams make a decision, the AI technology is able to learn and improve from that decision. This is how technology is trained over time and gets better .

Oftentimes, Facebook’s community guideline policies change to keep up with changes in social norms, language and our products and services. This requires the content review process to always be evolving and changing .