Difference between revisions of "Generative Media"

From SI410
Jump to: navigation, search
m (Generative Screenplay)
(Addition of subsections in Ethics header + intro subsection sentence)
Line 41: Line 41:
 
Society (2019).</ref>
 
Society (2019).</ref>
  
 +
====Manipulation===
 
Several studies have been carried out and are currently taking place to help understand how humans perceive generative media and their feelings towards algorithms in the world of fine arts and literature. "Investigating American and Chinese Subjects’ explicit and implicit perceptions of AI-Generated artistic work" is an academic paper that analyzes one such study between American and Chinese citizens. <ref>Authors: Yuheng Wu, Yi Mou, Zhipeng Li, Kun Xu, Investigating American and Chinese Subjects’ explicit and implicit perceptions of AI-Generated artistic work, Computers in Human Behavior, Volume 104, 2020, 106186, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2019.106186. Retrieved from: (https://www.sciencedirect.com/science/article/pii/S074756321930398X)</ref> The paper reveals a general opinion of artificial intelligence from the public of two of the biggest leaders of technology in the world to get a notion of how much fluctuation there is from one society to the next. The study reports that in China, the risks and dangers of fully autonomous systems have not been thoroughly articulated and debated; therefore, most Chinese citizens welcome and celebrate the technology, in sharp contrast to Western nations that are cautious with research and development. The government heavily influences public opinion in China, making generative media an efficient outlet to mislead and misinform. Society will overlook the dangerous capabilities of manipulated media in societies that show blind trust in novel technology. <ref>Authors: Yuheng Wu, Yi Mou, Zhipeng Li, Kun Xu, Investigating American and Chinese Subjects’ explicit and implicit perceptions of AI-Generated artistic work, Computers in Human Behavior, Volume 104, 2020, 106186, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2019.106186. Retrieved from: (https://www.sciencedirect.com/science/article/pii/S074756321930398X)</ref>
 
Several studies have been carried out and are currently taking place to help understand how humans perceive generative media and their feelings towards algorithms in the world of fine arts and literature. "Investigating American and Chinese Subjects’ explicit and implicit perceptions of AI-Generated artistic work" is an academic paper that analyzes one such study between American and Chinese citizens. <ref>Authors: Yuheng Wu, Yi Mou, Zhipeng Li, Kun Xu, Investigating American and Chinese Subjects’ explicit and implicit perceptions of AI-Generated artistic work, Computers in Human Behavior, Volume 104, 2020, 106186, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2019.106186. Retrieved from: (https://www.sciencedirect.com/science/article/pii/S074756321930398X)</ref> The paper reveals a general opinion of artificial intelligence from the public of two of the biggest leaders of technology in the world to get a notion of how much fluctuation there is from one society to the next. The study reports that in China, the risks and dangers of fully autonomous systems have not been thoroughly articulated and debated; therefore, most Chinese citizens welcome and celebrate the technology, in sharp contrast to Western nations that are cautious with research and development. The government heavily influences public opinion in China, making generative media an efficient outlet to mislead and misinform. Society will overlook the dangerous capabilities of manipulated media in societies that show blind trust in novel technology. <ref>Authors: Yuheng Wu, Yi Mou, Zhipeng Li, Kun Xu, Investigating American and Chinese Subjects’ explicit and implicit perceptions of AI-Generated artistic work, Computers in Human Behavior, Volume 104, 2020, 106186, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2019.106186. Retrieved from: (https://www.sciencedirect.com/science/article/pii/S074756321930398X)</ref>
  
[https://en.wikipedia.org/wiki/Christie%27s Christie's] New York is a prestigious auction house, and in 2018 a historical event unfolded. The first-ever original work of art generated by artificial intelligence was sold at the final price of $432,500, second to Andy Warhol’s suite of 10 screenprints, surpassing its opening price of $10,000. To successfully generate this piece, algorithms had to go under excessive training by being fed a slew of fine Renaissance paintings from the WikiArt database, which collected the specifications of its creators [https://obvious-art.com Obvious] a Paris based collective, to learn to paint like a master. However, Robbie Barrat, the artist, and programmer who wrote the code to generate Renaissance-style images, was not amongst those who were compensated for the sale. The relationships formed when generating a synthetic piece of art get very complex, which obscures the proportions of involvement, subsequently splitting the reward in an unfair and unethical fashion.<ref>Epstein, Ziv, et al. “Who Gets Credit for AI-Generated Art?” IScience, vol. 23, no. 9, Elsevier Inc, 2020, p. 101515, doi:10.1016/j.isci.2020.101515.</ref>
+
====Credit, Copyright, & Ownership====
 +
The rise of generative media has brought on new ethical concerns regarding ownership and copyright of the media among human collaborators. [https://en.wikipedia.org/wiki/Christie%27s Christie's] New York is a prestigious auction house, and in 2018 a historical event unfolded. The first-ever original work of art generated by artificial intelligence was sold at the final price of $432,500, second to Andy Warhol’s suite of 10 screenprints, surpassing its opening price of $10,000. To successfully generate this piece, algorithms had to go under excessive training by being fed a slew of fine Renaissance paintings from the WikiArt database, which collected the specifications of its creators [https://obvious-art.com Obvious] a Paris based collective, to learn to paint like a master. However, Robbie Barrat, the artist, and programmer who wrote the code to generate Renaissance-style images, was not amongst those who were compensated for the sale. The relationships formed when generating a synthetic piece of art get very complex, which obscures the proportions of involvement, subsequently splitting the reward in an unfair and unethical fashion.<ref>Epstein, Ziv, et al. “Who Gets Credit for AI-Generated Art?” IScience, vol. 23, no. 9, Elsevier Inc, 2020, p. 101515, doi:10.1016/j.isci.2020.101515.</ref>
  
 
==References==
 
==References==

Revision as of 19:01, 7 April 2021

Generative Media (also known as AI-generated media,[1][2] synthetic media,[3] and personalized media,[4]) is an umbrella term for the production of algorithmically and autonomously generating media forms, such as art, photographs, music, literature, and any other means of communicating creatively. It is media constructed with complex systems of mathematical formulas, which can magnify or mimic human ingenuity. Generative media goes by many names, such as "organic media," because it is created either with mathematical tools or autonomously i.e., without human intervention. The creation of creative material usually takes a significant amount of time, which generative media streamlines... drastically cutting the creation process time down, while enhancing precision. The term “synthetic media” denotes artificially generated or manipulated media, such as synthesized audio, virtual reality, and even advanced digital-image creation, which today is highly believable and “true to life.” [5]

History

B.C.E.

Algorithms and mathematical perspectives used to generate art was being done by the ancient Greeks, who lived before the current era. The Greek sculptor Polykleitos who wrote Canon, fixed proportions based on the ratio 1:√2 to construct the embodiment of, what he thought, represented a physical god (the epitome of human physique)[6] The notion of autonomously generating media was also popular amongst Greek inventors such as Daedalus and Hero of Alexandria, which designed machines capable of generating media without human intervention. [7]


C.E.

In the eighth century, current era, the Islamic world implemented complex geometric patterns to avoid figurative images becoming objects of worship. A recurring pattern is the 8-pointed star, often seen in Islamic tilework; it is made of two squares, one rotated 45 degrees with respect to the other. The fourth basic shape is the polygon, including pentagons and octagons. All of these can be recursively configured to generate a slew of trigonometric symmetries and linear transformations that include shears, shifts, reflections and rotations.[8] Then, during The Renaissance period, Italians implemented a mathematical perspective to generate their paintings. In 1415, the Italian architect Filippo Brunelleschi demonstrated the geometric linear perspective used to generate art in Florence, using trigonometry formulated by Euclid, to find the distant dimension's of objects.[9]


Digital Age

Desmond Paul Henry is a pioneer in generative media for producing some of the first works of art created by his "Drawing Machine 1", an analogue machine based on a bombsight computer and exhibited in 1962. The machine was capable of creating complex, abstract, asymmetrical, curvilinear, but repetitive line drawings. [10] Ian Goodfellow is a modern day computer scientist, which in 2014, invented a new class of machine learning named generative adversarial network (GAN), which elevated a computers creativity significantly. GAN systems work much like neural networks, in that a computer system can be trained on data to learn how to mimic the creativity of established humans. Today, GANs can be trained on a single photograph and then can generate a seemingly authentic video with many realistic characteristics. Generative media is rapidly expanding by giving rise to some unique subsets.

Subsets

Deepfakes

Deepfake videos made with a single image in 2021. [11]

Deepfakes, the subset of generative media that is of the most concern today, [12] was brought forward in late 2017 with the use of artificial intelligence algorithms to insert famous actresses' faces into adult videos.[13] Deepfakes utilize GAN systems and have been the fastest evolving subset of generative media, primarily because its building blocks, the computer source code, is open and free to anyone. When the deepfake source code became publicly available, demonstrations of its capabilities were generated, and the world sat confused when an impersonation of the 44th president, Barack Obama, seemed to portray him out of character. Jordan Peele, the comedian behind the demonstration, quickly revealed himself and explained the danger of such technology when in the hands of questionable individuals. Today, deepfake algorithms are making strides in progression, in that now they only need one picture to generate fabricated media, as oppose to a couple of years ago when it took hundreds of pictures for the system to generate videos. [14]

Researchers can identify poor-quality deep fakes quickly due to their poor skin texture mapping, unaligned or inaccurate audio alignment, flickering transposed faces, and hair movement, which is particularly difficult for computers to render. However, there are ways to use computers to examine video content and spot deep fakes. [15] In a 2018 study aiming to expose AI-generated fake faces by analyzing the eye blinking patterns, researchers Yuezun Li, Ming-Ching Chang, Siwei Lyu created a method to detect and identify abnormal blinking patterns.[16] However, while their results were promising, anonymous developers introduced new higher quality Deepfake technology for improving their blinking to pass detection. A 2020 publication discusses a method where Mittal et al. developed a deep learning network that used emotion to identify deep fakes in video content. When tested against two large deep fake identification testing datasets, they scored 84.4% accuracy on the DFDC dataset and a 96.6% on the DF-TIMT datasets.[17] The method uses video and audio data from real and fake videos to establish a baseline for emotional responses and phrases. Algorithms are used to both create deep fakes and detect them. As the technology to combat deep fakes develops, so will the ability to pass detection. For this reason, mainstream media has taken to the outright banning of deepfakes.[18]

Generative Art

Generative art a subset of generative media, is when art is generated algorithmically and autonomously, made possible by the advancements in machine learning and neural networks. Anyone can then train these algorithmic systems regarding a wide range of contexts. For example, with such a system, an artist would feed it data, such as Leonardo Da Vinci's entire painting catalog, while fine-tuning the neural net to make the decisions that the artist deems satisfactory, then generate a painting with the specifications that were taught.

The New Rembrandt, an artificially generated painting by a GAN system. [19]
The partition that once existed between fabrication and originality is nearly non-existent when analyzing contemporary pieces such as “the Next Rembrandt”. In 2016, a Rembrandt painting was designed by a computer and created by a 3D printer 351 years after the painter’s death. [20] Tech giant Microsoft and ING Group, a Dutch bank, were the two companies that put their abundance of resources together to undertake this endeavor.

Paintings are perceived by many as two-dimensional. However, upon a thorough analysis of the world's greatest paintings, it is revealed that many layers of paint create the third dimension of height. The state-of-the-art algorithms that produced The New Rembrandt created a data set of topological maps that represented brushstrokes and layers of paint, which the artist used to make the very lifelike depths and tones. [21]


Generative Poetry

A synthetic poem generated by Racter, a computer program. [22]

Generative Poetry, another subset of generative media, is when algorithms attempt to mimic the meaning, phrasing, structure, and rhyme aspects of poetry. Generating algorithmic poetry has been successful in many instances, as Zackary Scholl, from Duke University has demonstrated. He successfully generated poetry that, when posted, received positive feedback. Other engineers have developed systems that use case-based-reasoning to generate formulations of a given input text via a composition of poetic fragments that are retrieved from a case-base of existing poems. The fragments are then configured according to metrical rules, which govern well-formed poetic pieces. Racter is one such program used to generate this synthetic poetry.

Generative Screenplay

Generative media can also take the form of generative filmmaking, where neural networks analyze, write, and generate films and screenplays. The machine learning models use neural networks to learn from existing text and film in order to produce new text and film based on the data. These models not only consider plot, characters, and dialogue but also keep track of the behavioral cues of characters or the background (e.g., environment, music, etc.) associated with scenes. [23] This allows them to analyze the emotional creativity associated with filmmaking to gain insight into what types of scenes elicit what feelings in the audience.

In 2016, Sunspring, a science fiction short film written entirely by an artificial intelligence bot named Benjamin, was created for the Sci-Fi London film festival’s 48hr Challenge. [24] Director Oscar Sharp and NYU AI researcher, Ross Goodwin, developed the original idea. Benjamin learned to mimic the structure of films by using a recurrent neural network called long short-term memory (LSTM) to analyze dozens of science fiction subtitle files and film scripts. [24][25] It was able to produce stage directions and well-formatted character lies, but experienced significant difficulties with character names that are more unpredictable than other words. [24] Sharp and Goodwin then edited the script for length, but preserved the content, which was entirely generated by AI.

Ethical Dilemmas

Manipulation and credit/copyright issues are some of the ethical dilemmas surrounding generative media.[26]

=Manipulation

Several studies have been carried out and are currently taking place to help understand how humans perceive generative media and their feelings towards algorithms in the world of fine arts and literature. "Investigating American and Chinese Subjects’ explicit and implicit perceptions of AI-Generated artistic work" is an academic paper that analyzes one such study between American and Chinese citizens. [27] The paper reveals a general opinion of artificial intelligence from the public of two of the biggest leaders of technology in the world to get a notion of how much fluctuation there is from one society to the next. The study reports that in China, the risks and dangers of fully autonomous systems have not been thoroughly articulated and debated; therefore, most Chinese citizens welcome and celebrate the technology, in sharp contrast to Western nations that are cautious with research and development. The government heavily influences public opinion in China, making generative media an efficient outlet to mislead and misinform. Society will overlook the dangerous capabilities of manipulated media in societies that show blind trust in novel technology. [28]

Credit, Copyright, & Ownership

The rise of generative media has brought on new ethical concerns regarding ownership and copyright of the media among human collaborators. Christie's New York is a prestigious auction house, and in 2018 a historical event unfolded. The first-ever original work of art generated by artificial intelligence was sold at the final price of $432,500, second to Andy Warhol’s suite of 10 screenprints, surpassing its opening price of $10,000. To successfully generate this piece, algorithms had to go under excessive training by being fed a slew of fine Renaissance paintings from the WikiArt database, which collected the specifications of its creators Obvious a Paris based collective, to learn to paint like a master. However, Robbie Barrat, the artist, and programmer who wrote the code to generate Renaissance-style images, was not amongst those who were compensated for the sale. The relationships formed when generating a synthetic piece of art get very complex, which obscures the proportions of involvement, subsequently splitting the reward in an unfair and unethical fashion.[29]

References

  1. Goodstein, Anastasia. "Will AI Replace Human Creativity?" https://www.adlibbing.org/2019/10/07/will-ai-replace-human-creativity/ (30 January 2020)
  2. Waddell, Kaveh "Welcome to our new synthetic realities" https://www.axios.com/synthetic-realities-fiction-stories-fact-misinformation-ed86ce3b-f1a5-4e7b-ba86-f87a918d962e.html (30 January 2020)
  3. Hunt "Why Now Is The Time to Be a Maker in Generative Media". https://www.producthunt.com/stories/why-now-is-the-time-to-be-a-maker-in-generative-media (2020-02-15)
  4. Ignatidou, Sophia "AI-driven Personalization in Digital Media Political and Societal Implications" https://www.chathamhouse.org/sites/default/files/021219%20AI-driven%20Personalization%20in%20Digital%20Media%20final%20WEB.pdf, International Security Department (30 January 2020)
  5. Antonio Daniele and Yi-Zhe Song. 2019. AI+Art= Human. AAAI AI Ethics and Society (2019).
  6. Stewart, Andrew "One Hundred Greek Sculptors: Their Careers and Extant Works" from: Journal of Hellenic Studies (November 1978)
  7. Brett, Gerard "The Automata in the Byzantine "Throne of Solomon"", Speculum, 29 (3): 477–487, doi:10.2307/2846790, ISSN 0038-7134, JSTOR 2846790.(July 1954)
  8. Bouaissa, Malikka. "The crucial role of geometry in Islamic art" http://www.alartemag.be/en/en-art/the-crucial-role-of-geometry-in-islamic-art/ "Al Arte Magazine" (1 December 2015) (27 July 2013)
  9. Vasari, Giorgio (1550). Lives of the Artists. Torrentino. p. Chapter on Brunelleschi.
  10. Beddard, Honor. "Computer art at the V&A". Victoria and Albert Museum. (22 September 2015)
  11. Egor Zakharov, Aliaksandra Shysheya published on: 05.26.2019 "Deepfakes generated from a single image" https://www.wired.com/story/deepfakes-getting-better-theyre-easy-spot/
  12. L. Whittaker, T. C. Kietzmann, J. Kietzmann, and A. Dabirian, "“All Around Me Are Synthetic Faces”: The Mad World of AI-Generated Media," in IT Professional, vol. 22, no. 5, pp. 90-99, 1 Sept.-Oct. 2020, doi: 10.1109/MITP.2020.2985492.
  13. L. Whittaker, T. C. Kietzmann, J. Kietzmann, and A. Dabirian, "“All Around Me Are Synthetic Faces”: The Mad World of AI-Generated Media," in IT Professional, vol. 22, no. 5, pp. 90-99, 1 Sept.-Oct. 2020, doi: 10.1109/MITP.2020.2985492.
  14. Author: Dane Mitrev (March 2021). Few-Shot Adversarial Learning of Realistic Neural Talking Head Models. Retrieved from https://arxiv.org/pdf/1905.08233.pdf
  15. "How To Spot Deepfake Videos — 15 Signs To Watch For". Us.Norton.Com, 2021, https://us.norton.com/internetsecurity-emerging-threats-how-to-spot-deepfakes.html.
  16. Li, Yuezun, Ming-Ching Chang, and Siwei Lyu. "In ictu oculi: Exposing ai generated fake face videos by detecting eye blinking." arXiv preprint arXiv:1806.02877 (2018).
  17. Mittal, Trisha, et al. "Emotions Don't Lie: A Deepfake Detection Method using Audio-Visual Affective Cues." arXiv preprint arXiv:2003.06711 (2020).
  18. Hern, Alex. "Facebook Bans 'Deepfake' Videos in Run-Up To US election". The Guardian, 2021, https://www.theguardian.com/technology/2020/jan/07/facebook-bans-deepfake-videos-in-run-up-to-us-election
  19. https://www.nextrembrandt.com.
  20. United Nations Educational, Scientific and Cultural Organization (March 2021). The Next Rembrandt. Retrieved from https://en.unesco.org/artificial-intelligence/ethics/cases
  21. (March 2021). The Next Rembrandt. Retrieved from https://www.nextrembrandt.com
  22. Racter, published in 1984. "The Policeman’s Beard is Half Constructed: Computer Prose and Poetry" http://www.101bananas.com/poems/racter.html.
  23. Chu, Eric & Roy, Deb. "Audio-Visual Sentiment Analysis for Learning Emotional Arcs in Movies", https://arxiv.org/pdf/1712.02896.pdf (8 Dec 2017)
  24. 24.0 24.1 24.2 Newitz, Annalee. "Movie written by algorithm turns out to be hilarious and intense", https://arstechnica.com/gaming/2016/06/an-ai-wrote-this-movie-and-its-strangely-moving/ (9 June 2016)
  25. Frohlick, Alex. "The possibility of an AI Auteur? AI Authorship In the AI Film", https://www.researchgate.net/publication/342591536_The_Possibility_of_an_AI_Auteur_AI_Authorship_In_the_AI_Film (July 2020)
  26. Antonio Daniele and Yi-Zhe Song. 2019. AI+Art= Human. AAAI AI Ethics and Society (2019).
  27. Authors: Yuheng Wu, Yi Mou, Zhipeng Li, Kun Xu, Investigating American and Chinese Subjects’ explicit and implicit perceptions of AI-Generated artistic work, Computers in Human Behavior, Volume 104, 2020, 106186, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2019.106186. Retrieved from: (https://www.sciencedirect.com/science/article/pii/S074756321930398X)
  28. Authors: Yuheng Wu, Yi Mou, Zhipeng Li, Kun Xu, Investigating American and Chinese Subjects’ explicit and implicit perceptions of AI-Generated artistic work, Computers in Human Behavior, Volume 104, 2020, 106186, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2019.106186. Retrieved from: (https://www.sciencedirect.com/science/article/pii/S074756321930398X)
  29. Epstein, Ziv, et al. “Who Gets Credit for AI-Generated Art?” IScience, vol. 23, no. 9, Elsevier Inc, 2020, p. 101515, doi:10.1016/j.isci.2020.101515.