A bot (or, more specifically, an Internet bot) is a  Bots carry out repetitive and predefined tasks, often in a manner that mimics human behavior. Due to their automated nature, bots can complete tasks more efficiently and at a larger scale than humans. Compared with other software applications, the time needed to develop and deploy bots (especially those that do not converse with online users) is relatively short. Consequently, bots hold a significant presence on the Internet and power core components of the modern , such as . In light of this ability, however, bots are frequently utilized to execute a variety of malicious activities.that performs automated tasks on top of a (commonly the ).
- 1 History
- 2 Design
- 3 Types of bots
- 4 Examples of bots
- 5 Traffic and management
- 6 Ethical concerns
- 7 See also
- 8 References
The notion of a bot dates back to 1950, when  The test, or game, revolves around three players: two humans and one computer. One of the human players, the interrogator, is isolated and inputs questions into a computer. The questions are structured in a particular format and a limited number are asked. The other human player and the computer player answer the questions asked. As the game proceeds, the interrogator uses the answers to determine which of the other players is the human and which is the computer. The computer player attempts to answer questions in a manner that imitates humans and is said to have passed the Turing test if the interrogator falsely concludes that it is the other human player. The computer player in this sense is thus seen as a representation of a bot.created the , later named the Turing test.
The computer player used in the Turing test closely resembles a  Over the next few decades following the inception of the Turing test, the development of chatbots grew significantly. During the 1960s, professor created , a bot that utilizes (NLP) to converse with humans. Overall, the original version of ELIZA was quite limited in terms of functionality and ability to hold conversations, but it is nevertheless remembered as being one of the first true instances of a bot., one that interacts with humans via dialogue.
Following ELIZA, several chatbots emerged with varying behavior and oftentimes enhancements to preceding chatbots. These include  Many of the improvements made to chatbots over time are a result of advances in communication technologies, such as (IM) and (IRC).(1972), (1988), (1991), and (1995).
With the development of the Web, the usage and capabilities of bots expanded beyond chatting. In particular, , which are bots used by search engines to web pages, arose during the 1990s. The first web crawler was developed in 1994 by Brian Pinkerton, a student at the . Pinkerton’s web crawler led to the first search engine capable of text search, . A few years later, in 1997, created its first web crawler, BackRub (now known as ).
As the Web and Internet have grown, so have the roles and functions of bots. Bots are currently being used in several contexts, including , , and .
Like most software applications, bots can be designed in several ways; however, there are a few underlying components that most designs share. These include application logic,  Application logic is the code that drives a bot (i.e., the program for a bot). It determines a bot’s behavior and actively interacts with any databases or external integrations that a bot utilizes. Databases store data, which may be pre-existing (before a bot runs) or created as a bot runs. Pre-existing data can help guide a bot’s decisions. Newly created data can be processed for purposes independent of a bot’s functionality, as is the case with web crawlers. External integrations, which usually occur through an (API), enable a bot to utilize services provided by another party (e.g., ). Such integrations can help prevent a bot’s developer from writing complicated code needed to perform a task (e.g., posting on Twitter)., and external integrations.
In cases where a bot requires  This can be in a bot’s application logic directly or through an external integration.(AI), NLP or (ML) are oftentimes utilized.
In the general sense, a  Each device in the network is a bot, or, rather, runs any number of bots, and the botnet works to accomplish a specific task. Communication in early botnets occurred through IRC.is a network of computers connected through the Internet.
Although botnets can be used for well-founded purposes, such as ensuring a website remains alive, they are oftentimes used for malicious purposes. This occurs when a remote party, the bot herder, penetrates through the of several computers and installs on them. Once the malware has been installed, the bot herder can effectively run bots on the infected devices to carry out malicious operations, such as .
In terms of architecture, botnets can be organized in a wide array of arrangements. Common models used to construct botnets include , , , and .
Types of bots
This is a non-exhaustive list.
Social media bots
A  Actions that such bots may perform vary depending on the specific platform being operated on, but common functions include posting content, following users, attempting to gain followers, and liking or commenting on posts. Social media bots, like chatbots, are typically designed to imitate human behavior; however, they differ from chatbots in that they normally do not have the ability to maintain conversations with other platform users. Additionally, social media bots tend to be much easier to manage than chatbots, and thus they can be deployed at a much larger scale. Indeed, according to a study conducted by the and in 2017, up to 15% of Twitter accounts are controlled by bots.is one that interacts with a social media network.
Web crawlersA web crawler, or spider, is a type of bot that traverses the Web with the intent of discovering a large subset of all web pages and the content they have. Web crawlers are mostly utilized by search engine companies so that they can provide relevant search results to users. A web crawler begins with an initial set of pages that it will crawl. It then uses the on those initial pages to discover new pages, which are then traversed in a likewise manner to discover more pages. While crawling, a web crawler downloads and indexes the pages that it comes across. Web crawlers do not crawl every single page on the Web.
A chatbot is a software application that holds conversations with humans by imitating their dialogue. Chatbots may converse with humans through text or voice. Though functionality can vary tremendously among them, chatbots can be classified into two primary subtypes: rule-based chatbots and AI chatbots. A rule-based chatbot is governed by a predefined set of rules (e.g., a table mapping keywords to output) that dictates how it should respond to human input. An AI chatbot, on the other hand, utilizes ML to understand a human language and provides responses to human input based on the understanding that it develops.
Examples of bots
Googlebot is the web crawler used by Google to build its search engine. More precisely, Googlebot refers to two distinct types of web crawlers used by Google, one of which is used to imitate a user (Googlebot Desktop) and the other of which is used to imitate a user (Googlebot Smartphone). On average, Googlebot will not crawl a particular website more than once every few seconds. Googlebot was designed with performance and scalability in mind. As a result, thousands of Google’s machines are running instances of Googlebot concurrently. Additionally, in order to reduce bandwidth consumption, numerous Googlebot instances are run on machines situated closely to hosting websites that are likely to be crawled. Currently, Googlebot utilizes version 1.1 of the (HTTP) to crawl websites.
A  There are over 2,500 Wikipedia bots that have been approved for use. Previously, Wikipedia bots were used to quickly create a large number of articles; however, such mass-creation is now restricted due to . The bot policy was developed due to concern over technical disruptions that can occur on Wikipedia when bots are not properly designed. The function and scope of Wikipedia bots range quite tremendously. Common tasks carried about bots include editing, adding content, and archiving, all with respect to articles. Some specific Wikipedia bots include (reverts vandalism), (delivers alerts to ), and (mitigates ).is a type of that serves to maintain the more than 55 million articles on the .
Intercom Custom Bots
Custom Bots are a chatbot solution developed by  They are designed to help companies scale their sales, marketing, and customer support efforts. The chatbots are customizable and can thus be extended to add company-specific features. A Custom Bot is typically deployed directly on a company’s website in order to engage with visitors. It initiates conversations by asking targeted questions that help gain insight into a visitor’s needs or concerns. By this means, Custom Bots assist companies in efficiently converting website visitors into customers and resolving existing customer requests. Various enterprise services, including those offered by , , and , can be integrated with a Custom Bot..
Traffic and management
Bots account for a significant portion of total  annual Bad Bot Report found that bots accounted for 40.8% of all traffic in 2020, a 5.7% increase from the previous year. Of this 40.8%, 25.6% came from malicious (bad) bots, while the rest was due to harmless (good) bots. The majority of bad bots utilize moderate to sophisticated persistence mechanisms that make them hard to detect and combat. Moreover, bad bots account for traffic across several industries. The telecommunications, information technology, sports, news, and business industries have bad bots accounting for 45.7%, 41.4%, 33.7%, 33%, and 29.7% of their total traffic, respectively. The percentage of bad bots utilizing mobile is currently on the rise, with 28.1% of all bad bot traffic originating from mobile browsers. In terms of global presence, 40.5% of bad bot traffic originates from the United States, followed by China (5.2%), the United Kingdom (4.9%), Russia (3.9%), and Japan (3.4%)..
A  The objective of a CAPTCHA is to provide a task that is easy for humans to complete but difficult for an automated process, like a bot, to complete. A website may utilize a CAPTCHA for a number of reasons, including limiting the creation of fake accounts, preserving poll accuracy, and reducing false comments. There are several types of CAPTCHAs, such as text-based (e.g., distorted alphanumeric sequence), image-based (e.g., select all images that match a theme), and checkbox (e.g., “I’m not a robot”). One popular CAPTCHA system is .(CAPTCHA) is a security check that aims to discern between bots and human users.
The robots exclusion protocol is an Internet  Web servers host a file named robots.txt (normally in the root directory of the server) that specifies which parts of a website should not be accessed by bots, along with other instructions for them to follow. The instructions contained in a robots.txt file cannot actually be enforced by a website. Rather, the intent is that good bots will follow them in order to avoid being a burden on a website. It is therefore unlikely that the presence of a robots.txt file will control the activity of malicious bots.used to limit the accessibility of bots (primarily web crawlers).
Distributed denial-of-service attacks
A  As the name suggests, a successful DDoS attack can result in a service becoming unavailable to a significant portion of the service’s userbase. A DDoS attack differs from a DoS attack in that the former creates numerous connections to a target service while the latter only creates one. In order to generate many connections, a botnet is normally utilized. For example, the 2016 , which disrupted services such as , , and , was carried out through a botnet created by installing on a large number of devices. DDoS attacks can occur at different layers of the , particularly the (layer 7) and the (layer 3). DDoS attacks that occur at the application layer often focus on exhausting a service's resources. Application layer DDoS attacks include and . DDoS attacks that occur in the network layer frequently aim to create congestion in a service’s network pipelines. Network layer DDoS attacks include and .is a type of in which a substantial amount of traffic is generated on a service by a distributed collection of attackers, typically a botnet, in hopes of overwhelming the service.
Spamming is the use of an online communication channel to deliver unsolicited messages (spam) on a large scale. Spam usually consists of advertisements, , or malware downloads. A is a type of bot designed to facilitate spamming. Spambots operate on a variety of platforms, including social media, online forums, email services, and messaging apps. In order to generate and deliver spam, a spambot needs to create an account on its target platform. This, however, is a fairly simple task to automate, and workarounds exist for common security defenses, such as CAPTCHAs. In the specific case of sending , a spambot needs to a large number of email addresses to which it can deliver spam. One way to obtain a lengthy email address list is by utilizing a bot, which can download web pages and then scan them for specific patterns (e.g., an “@” followed by a ) that match email addresses. Another, perhaps more simple method, is to purchase email address lists on the . Spambots that operate on messaging apps typically can hold rudimentary conversations with any users that respond to them and thus parallel very primitive chatbots.
In the context of bots, scalping refers to the acquisition of a highly-demanded or limited-availability product in hopes of reselling it for a profit. Bots are used to make initial transactions because timing margins are narrow and an automated process can carry out transactions much more quickly than a human. Historically, scalping has focused on industries such as tickets (sporting events and concerts) and clothing (limited releases, such as exclusive sneaker drops). During the , however, scalping shifted towards new markets, in large part because many live-audience events were canceled. Specifically, scalping bots were used to target items such as face coverings, workout equipment, hand sanitizers, and others that became highly demanded as public spaces closed and people were encouraged to control and prevent the spread of COVID-19. Scalping activity is most popular on retail websites during the , when scalpers anticipate gifts and products that will be in high demand. As a result, scalping bots are referred to as “Grinchbots” during the holiday season. Grinchbots targeted the gaming industry heavily during 2020, as scalpers quickly depleted the supply of gaming hardware such as , , and next-generation .
 It includes the circulation of , , and other types of inaccurate information, regardless of whether or not it is intentionally deceiving. Bots, particularly social media bots, play a significant role in spreading misinformation. Social media platforms, such as Twitter and , are targeted destinations for bots aiming to disseminate information because they enable information to propagate quickly and reach a wide-ranging audience. Bots have had a notable impact in spreading misinformation during several far-reaching events, including the COVID-19 pandemic and the .spread pertains to the dispersion of false or misleading information.
A bot can determine the most relevant content to spread by taking into consideration trending topics on social media. Such topics can easily be discovered by bots as most social media platforms make them accessible to their users. Once a bot is aware of a popular or widely-discussed matter, it can search the Web and retrieve content related to it. Following this, a bot can post the content, or portions of it, to social media. Misinformation spread can occur in this process because the content retrieved by a bot may not be curated or verified.
 The underlying assumption of such an attack is that people frequently reuse username and password pairs (or other forms of account credentials) across different services. This allows an attacker to potentially acquire sensitive information, such as credit card numbers, from various sources. Stolen collections of username and password pairs usually stem from and can be obtained from an online . Bots factor into a credential stuffing attack during the process of breaching a service, or multiple services, with a list of compromised account credentials. They assist in scaling and automating an attack, which is necessary due to the considerable number of credentials often involved. More specifically, a bot executing a credential stuffing attack leverages to attempt to log in to several accounts on a service simultaneously. It tracks successful logins as it executes and, to get around security defenses, may on each login attempt to make it appear that each attempt is from a different device. This process may then be repeated on other services.is a type of wherein compromised login credentials from one service are used in an attempt to gain unauthorized access to accounts on another service.
- Suchacka, G., & Iwański, J. (2020). Identifying legitimate web users and bots with different traffic profiles — An information bottleneck approach. Knowledge-Based Systems, 197, 3. https://doi.org/10.1016/j.knosys.2020.105875
- Cloudflare. (n.d.). How is an internet bot constructed? Cloudflare. Retrieved January 20, 2022, from https://www.cloudflare.com/learning/bots/how-is-an-internet-bot-constructed/
- Kaspersky. (2021, March 22). What are bots? – Definition and explanation. Kaspersky. Retrieved January 20, 2022, from https://www.kaspersky.com/resource-center/definitions/what-are-bots
- JavaTpoint. (n.d.). Turing test in AI. JavaTpoint. Retrieved January 28, 2022, from https://www.javatpoint.com/turing-test-in-ai
- Zantal-Wiener, A. (2021, June 11). Where do bots come from? A brief history. HubSpot. Retrieved January 21, 2022, from https://blog.hubspot.com/marketing/where-do-bots-come-from
- Oppy, G., & Dowe, D. (2003, April 9). The Turing Test. In The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University. Retrieved January 21, 2022, from https://plato.stanford.edu/entries/turing-test/
- Adamopoulou, E., & Moussiades, L. (2020). Chatbots: History, technology, and applications. Machine Learning with Applications, 2, 1–3. https://doi.org/10.1016/j.mlwa.2020.100006
- Wood, D. (2020, August 28). What is the history of chatbots? YakBots. Retrieved January 21, 2022, from https://yakbots.com/what-is-the-history-of-chatbots/
- Gillis, A. S. (n.d.). Bot. In WhatIs.com dictionary. Retrieved January 21, 2022, from https://whatis.techtarget.com/definition/bot-robot
- Knecht, T. (2021, May 4). A brief history of bots and how they've shaped the Internet today. Abusix. Retrieved January 21, 2022, from https://abusix.com/resources/botnets/a-brief-history-of-bots-and-how-theyve-shaped-the-internet-today/
- The History of SEO. (n.d.). Short history of early search engines. The History of SEO. Retrieved January 21, 2022, from https://www.thehistoryofseo.com/The-Industry/Short_History_of_Early_Search_Engines.aspx
- Joshi, N. (2020, February 23). Choosing between rule-based bots and AI Bots. Forbes. Retrieved January 22, 2022, from https://www.forbes.com/sites/cognitiveworld/2020/02/23/choosing-between-rule-based-bots-and-ai-bots/
- Nanda, D., Wadhwa, P., Singh, S., & Kumar, D. (2014). Botnet: Lifecycle, architecture and detection model. International Journal of Latest Technology in Engineering, Management & Applied Science, 3(3), 28–29. https://doi.org/10.51583/ijltemas
- Techopedia (2017, January 4). Botnet. In Techopedia dictionary. Retrieved January 22, 2022, from https://www.techopedia.com/definition/384/botnet
- Norton. (2019, August 12). What is a botnet? Norton. Retrieved January 22, 2022, from https://us.norton.com/internetsecurity-malware-what-is-a-botnet.html
- Fortinet. (n.d.). What is a botnet? Fortinet. Retrieved January 22, 2022, from https://www.fortinet.com/resources/cyberglossary/what-is-botnet
- Imperva. (2020, September 24). Bots. Imperva. Retrieved January 23, 2022, from https://www.imperva.com/learn/application-security/what-are-bots/
- Newberg, M. (2017, March 10). As many as 48 million Twitter accounts aren't people, says study. CNBC. Retrieved January 23, 2022, from https://www.cnbc.com/2017/03/10/nearly-48-million-twitter-accounts-could-be-bots-says-study.html
- Cloudflare. (n.d.). What is a web crawler? | How web spiders work. Cloudflare. Retrieved January 23, 2022, from https://www.cloudflare.com/learning/bots/what-is-a-web-crawler/
- Sabin, J. (2020, January 7). Intro to chatbots. Capacity. Retrieved February 9, 2022, from https://capacity.com/chatbots/intro-to-chatbots/
- Cloudflare. (n.d.). What is a chatbot? Cloudflare. Retrieved February 9, 2022, from https://www.cloudflare.com/learning/bots/what-is-a-chatbot/
- Google. (2021, December 4). Googlebot. Google Developers. Retrieved January 24, 2022, from https://developers.google.com/search/docs/advanced/crawling/googlebot
- Wikipedia:Bots. (2022, January 4). In Wikipedia. https://en.wikipedia.org/wiki/Wikipedia:Bots
- Intercom. (n.d.). Custom bots. Intercom. Retrieved February 10, 2022, from https://www.intercom.com/customizable-bots
- Intercom. (n.d.). Custom bots. Intercom. Retrieved February 10, 2022, from https://www.intercom.com/drlp/customizable-bots
- Imperva. (n.d.). CAPTCHA. Imperva. Retrieved January 25, 2022, from https://www.imperva.com/learn/application-security/what-is-captcha/
- Imperva. (2021). Bad bot report 2021: The pandemic of the Internet. https://www.imperva.com/resources/resource-library/reports/bad-bot-report/
- Cloudflare. (n.d.). How CAPTCHAs work | What does CAPTCHA mean? Cloudflare. Retrieved January 25, 2022, from https://www.cloudflare.com/learning/bots/how-captchas-work/
- Cloudflare. (n.d.). What is robots.txt? | How a robots.txt file works. Cloudflare. Retrieved January 25, 2022, from https://www.cloudflare.com/learning/bots/what-is-robots.txt/
- Imperva. (n.d.). Distributed denial of service (DDoS). Imperva. Retrieved January 26, 2022, from https://www.imperva.com/learn/ddos/denial-of-service/
- Cloudflare. (n.d.). Famous DDoS attacks | The largest DDoS attacks of all time. Cloudflare. Retrieved January 26, 2022, from https://www.cloudflare.com/learning/ddos/famous-ddos-attacks/
- Cloudflare. (n.d.). What is a DDoS attack? Cloudflare. Retrieved January 26, 2022, from https://www.cloudflare.com/learning/ddos/what-is-a-ddos-attack/
- Cloudflare. (n.d.). How do layer 3 DDoS attacks work? | L3 DDoS. Cloudflare. Retrieved January 26, 2022, from https://www.cloudflare.com/learning/ddos/layer-3-ddos-attacks/
- Techopedia (n.d.). Spamming. In Techopedia dictionary. Retrieved January 26, 2022, from https://www.techopedia.com/definition/23763/spamming
- Cloudflare. (n.d.). What is a spam bot? | How spam comments and spam messages spread. Cloudflare. Retrieved January 26, 2022, from https://www.cloudflare.com/learning/bots/what-is-a-spambot/
- Okta. (n.d.). What is a spam bot? Definition & defenses. Okta. Retrieved January 26, 2022, from https://www.okta.com/identity-101/spam-bot/
- Netacea. (2021, October 7). What are scalper bots? Netacea. Retrieved January 26, 2022, from https://www.netacea.com/glossary/scalper-bots/
- Himelein-Wachowiak, M., Giorgi, S., Devoto, A., Rahman, M., Ungar, L., Schwartz, H. A., Epstein, D. H., Leggio, L., & Curtis, B. (2021). Bots and misinformation spread on social media: Implications for COVID-19. Journal of Medical Internet Research, 23(5), 5–9. https://doi.org/10.2196/26933
- University of California, Santa Barbara. (n.d.). How is fake news spread? Bots, people like you, trolls, and microtargeting. Center for Information Technology and Society. Retrieved February 7, 2022, from https://www.cits.ucsb.edu/fake-news/spread
- Shao, C., Ciampaglia, G. L., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2018). The spread of low-credibility content by social bots. Nature Communications, 9(1), 2. https://doi.org/10.1038/s41467-018-06930-7
- Cloudflare. (n.d.). What is credential stuffing? | Credential stuffing vs. brute force attacks. Cloudflare. Retrieved February 8, 2022, from https://www.cloudflare.com/learning/bots/what-is-credential-stuffing/
- Imperva. (n.d.). Credential stuffing. Imperva. Retrieved February 8, 2022, from https://www.imperva.com/learn/application-security/credential-stuffing/