Content moderation

From SI410
Revision as of 17:19, 27 March 2019 by Tbourani (Talk | contribs)

Jump to: navigation, search

Content moderation is the process of monitoring, filtering and removing online user-generated content according to the rules of a private organization or the regulations of a government. It is used to restrict illegal or obscene content, spam, and content considered offensive or incongruous with the values of the moderator. When applied to dominant platforms with significant influence, content moderation may be conflated with censorship. Ethical issues involving content moderation include the psychological effects on content moderators, human and algorithmic bias in moderation, the trade-off between free speech and free association, and the impact of content moderation on minority groups.

Overview

Most types of moderation involve a top-down approach, where a moderator or small group of moderators are give discretionary power by a platform to approve or disapprove user-generated content. These moderators may be paid contractors or unpaid volunteers. A moderation hierarchy may exist or each moderator may have independent and absolute authority to make decisions.

In general, content moderation can be broken down into 6 major categories.[1]

  • Pre-Moderation screens each submission before it is visible to the public. This creates a bottleneck in user-engagement, and the delay may cause frustration in the user-base. However, it ensures maximum protection against undesired content, eliminating the risk of exposure to unsuspecting users. It is only practical for small user communities, and was common in moderated newsgroups on Usenet.[2]
  • Post-Moderation screens each submission after it is visible to the public. While preventing the bottleneck problem, it is still impractical for large user communities. Furthermore, as the content is often reviewed in a queue, undesired content may remain visible for an extended period of time, drowned out by benign content ahead of it, which must still be reviewed.
  • Reactive moderation reviews only that content which has been flagged by users. It retains the benefits of both pre- and post-moderation, allowing for real-time user-engagement and the immediate review of only potentially undesired content. However, it is reliant on user participation and is still susceptible to benign content being falsely flagged. Most modern social media platforms, including Facebook and YouTube, rely on this method.
  • Distributed moderation is an exception to the top-down approach. It instead gives the power of moderation to the users, often making use of a voting system. This is common on Reddit and Slashdot, the latter also using a meta-moderation system, in which users also rate the decisions of other users.[3] This method scales well across user-communities of all sizes, but also relies on users having the same perception of undesired content as the platform. It is also susceptible to groupthink and malicious coordination, also known as brigading.[4]
  • Automated moderation is the use of software to automatically assess content for desirability. It can be used in conjunction with any of the above moderation types. Its accuracy is dependent on the quality of its implementation, and it is susceptible to algorithmic bias and adversarial examples[5]. Copyright detection software on YouTube and spam filtering are examples of automated moderation[6].
  • No moderation is the lack of moderation entirely. Such platforms are often hosts to illegal and obscene content, and typically operate outside the law, such as The Pirate Bay and Dark Web markets. Spam is a perennial problem for unmoderated platforms, but may be mitigated by other methods, such as limited posting frequency and monetary barriers to entry. However, small communities with shared values and few bad actors can also thrive under no moderation, like unmoderated Usenet newsgroups.

History

Pre-1993: Usenet and the Open Internet

Usenet emerged in the early 1980s as a network of university and private computers, and quickly became the world's first Internet community. The decentralized platform hosted a collection of message boards known as newsgroups. These newsgroups were small communities by modern standards, and consisted of like-minded, technologically-inclined users sharing the hacker ethic.[7] This collection of principles, including "access to computer should be unlimited", "mistrust authority; promote decentralization" and "information wants to be free", enabled a community that was mostly free of moderation. Because the network was distributed, it was resistant to top-down censorship, and only a minority of the newsgroups were moderated.[8] Users instead self-moderated and slow growth allowed new users to gradually become acclimated to its cultural norms, known as "netiquette".[9]

1994 - 2005: Eternal September and Growth

In September 1993, AOL offered access to Usenet to the general public, ushering in the Eternal September, a massive influx of users that did not share the same values as the founder population[10] In 1994, the first instance of spam was recorded, instigating a massive backlash by the users. [11] In response, the first anti-spam bot was created, and the era of content moderation began.[12]

With the invention of the World Wide Web, users began to drift away from Usenet and forums and blogs proliferated as replacements. These small communities had much stronger moderation than early Usenet groups, in response to the growth of spam and bad actors in the prevailing years. The distributed nature of these groups was such that, if a user did not like the moderation policies in one forum, they could easily move to another.

As platforms matured, they began to adopt limited content policies, in an ad-hoc manner. In 2000, Compuserve was the first platform to develop an "Acceptable Use" policy, which banned racist speech[13] eBay soon followed in 2001, banning the sale of hate memorabilia and propaganda.[14]

2006 - 2010: Social Media and Early Corporate Moderation

In the mid-2000s, social media platforms such as YouTube, Twitter, Tumblr, Reddit, and Facebook began to emerge, and quickly became massive solitary platforms that far outstripped in influence the old distributed blogs and message boards. These platforms initially struggled with content moderation. YouTube in particular developed ad-hoc rules from individual cases, gradually building up an internal set of rules that was opaque, arbitrary, and difficult for moderators to apply.[13][15]

Other platforms, such as Twitter and Reddit initially adopted the unmoderated, free speech ethos of old, with Twitter claiming to be the "free speech wing of the free speech party" and Reddit stating that "distasteful" subreddits would not be removed, "even if we find it odious or if we personally condemn it."[16][17]

2010 - Present: Platform Dominance and Moderation Expansion

Throughout the 2010s, as the influence of social media platforms continued to grow and become more ubiquitous, the ethics of their moderation policies were brought into question. In having significant influence over national and international conversation, concerns about the presence of offensive content as well as concerns over the stifling of expression were raised.[18] Additionally, internet infrastructure providers also began to remove content hosted on their platforms.

In 2010, Wikileaks' leaked US Diplomatic Cables were removed by Amazon from their Cloud hosting service and their DNS provider also made the decision to drop their website.[19]

In 2012, Reddit user /u/violentacrez was doxxed by Gawker, and the subsequent media spotlight causes Reddit to break with their previous stance, and ban /r/Creepshots for their controversial content.[20] This led to further bannings for subreddits over the next few years, including /r/DeepFakes and /r/FatPeopleHate.[21][22][23]

In 2015, Instagram came under fire for moderating female nipples, which it viewed as obscene content, but not male nipples.[24]

In 2016, in the aftermath of Gamergate and it's associated harrassment, Twitter instituted the Trust and Safety Council, also breaking with their previous free speech ethos.[25]

In 2018, Tumblr banned adult content from their platform, leading to a mass removal of LGBT support groups and communities.[26]

Ethical Issues

Psychological Effects on Moderators

Information Transparency in Moderation Policies

Algorithmic Bias

Cultural Bias

Free Speech

Impact on Minority Groups

See Also

References

  1. ↑ Grime-Viort, Blaise (December 7, 2010). "6 Types of Content Moderation You Need to Know About". Social Media Today. Retrieved March 26, 2019.
  2. ↑ "Moderated Newsgroups". Big-8.org. August 4, 2012. Archived from [ the original] on August 4, 2012. Retrieved March 26, 2019.
  3. ↑ "Moderation and Metamoderation". Slashdot. Retrieved March 26, 2019.
  4. ↑ "Reddiquette: In Regard to Voting" Reddit. January 18, 2018. Retrieved March 26, 2019.
  5. ↑ Goodfellow, Ian; Papernot, Nicolas; et al (February 24, 2017). "Attacking Machine Learning with Adversarial Examples". OpenAI. Retrieved March 26, 2019.
  6. ↑ Tassi, Paul (December 19, 2013). "The Injustice of the YouTube Content ID Crackdown Reveals Google's Dark Side". Forbes. Retrieved March 26, 2019.
  7. ↑ Levy, Steven (2010). "Chapter 2: The Hacker Ethic". Hackers: Heroes of the Computer Revolution. pp. 27-31. ISBN 978-1-449-38839-3. Retrieved March 26, 2019.
  8. ↑ Palfrey, John (2010). "Four Phases of Internet Regulation". Social Research. 77 (3): 981-996. Retrieved from http://www.jstor.org/stable/40972303 on March 26, 2019.
  9. ↑ Kehoe, Brendan P. (January 1992). "4. Usenet News". Zen and the Art of the Internet. Retrieved March 26, 2019.
  10. ↑ Koebler, Jason (September 30, 2015). "It's September Forever". Motherboard. Retrieved March 26, 2019.
  11. ↑ Everett-Church, Ray (April 13, 1999). "The Spam That Started It All". Wired. Retrieved March 26, 2019.
  12. ↑ Gulbrandsen, Arnt (October 12, 2009). "Canter & Siegel: What actually happened". Retrieved March 26, 2019.
  13. ↑ 13.0 13.1 Buni, Catherine; Chemaly, Soraya (March 13, 2016). "The Secret Rules of the Internet". The Verge. Retrieved March 26, 2019.
  14. ↑ Cox, Beth (May 3, 2001). "eBay Bans Nazi, Hate Group Memorabilia". Internet News. Retrieved March 26, 2019.
  15. ↑ Rosen, Jeffrey (November 28, 2008). "Google's Gatekeepers". New York Times. Retrieved March 26, 2019.
  16. ↑ Halliday, Josh (March 22, 2012). "Twitter's Tony Wang: 'We are the free speech wing of the free speech party'". The Guardian. Retrieved March 26, 2019.
  17. ↑ "Reddit will not ban 'distasteful' content, chief executive says". October 17, 2012. BBC News. Retrieved March 26, 2019.
  18. ↑ Masnick, Mike (August 9, 2019). "Platforms, Speech and Truth: Policy, Policing and Impossible Choices". Techdirt. Retrieved March 26, 2019.
  19. ↑ Arthur, Charles; Halliday, Josh (December 3, 2010). "WikiLeaks fights to stay online after US company withdraws domain name". The Guardian. Retrieved March 26, 2019.
  20. ↑ Boyd, Danah (October 29, 2012). "Truth, Lies and 'Doxing": The Real Moral of the Gawker/Reddit Story". Wired. Retrieved March 26, 2019.
  21. ↑ Hawkins, Derek (February 8, 2018). "Reddit bans 'deepfakes', pornography using the faces of celebrities such as Taylor Swift and Gal Gadot". Washington Post. Retrieved March 26, 2019.
  22. ↑ "Removing Harassing Subreddits". June 10, 2015. Reddit. Retrieved March 26, 2019.
  23. ↑ Hatmaker, Taylor (March 15, 2019). "After Christchruch, Reddit bans communites infamous for sharing graphic videos of death". TechCrunch. Retrieved March 26, 2019.
  24. ↑ Kleeman, Sophie (October 1, 2015). "Instagram Finally Revealed the Reason It Banned Nipples - It's Apple". Mic. Retrieved March 26, 2019.
  25. ↑ Cartes, Patricia (February 9, 2016). "Announcing the Twitter Trust & Safety Council". Twitter. Retrieved March 26, 2019.
  26. ↑ Ho, Vivian (December 4, 2018). "Tumblr's adult content ban dismays some users: 'It was a safe space'". The Guardian. Retrieved March 26, 2019.
Retrieved from "http://si410wiki.sites.uofmhosting.net/index.php?title=Content_moderation&oldid=77439"