Difference between revisions of "Black Box Algorithms"

Revision as of 01:58, 8 February 2023

[File:blackbox.PNG]

Introduction

A black box algorithm is one where the user cannot see the algorithm's inner workings. It is a somewhat controversial system due to its secrecy and lack of transparency. However, its creators defend it as a security and privacy system to avoid data leaks and unfair competition.[1] Black box algorithms are computer programs that make predictions or decisions based on a set of inputs. Still, how the algorithm actually works is not visible or transparent to the user. In machine learning, these black box models are created directly from data by an algorithm, meaning that humans, even those who design them, cannot understand how variables are being combined to make predictions. Even if one has a list of the input variables, black box predictive models can be such complicated functions of the variables that no human can understand how the variables are jointly related to each other to reach a final prediction. [2]These algorithms are increasingly being used in areas such as criminal justice, finance, and healthcare, and their use has raised many important ethical and legal questions. One of the main concerns about black box algorithms is the potential for bias and discrimination. Since the inner workings of the algorithm are not transparent, it can be challenging to detect and correct any biases that may have been built into the system. This can lead to unfair and unjust outcomes, particularly for marginalized groups such as people of color and low-income individuals.

Examples of Black Box Algorithms

Google

COMPAS

As technology advances in our society, more and more decisions affecting individuals are being made by hidden algorithms. One area where this has raised concern is in the criminal justice system, specifically regarding the fairness and due process of these algorithms. One example of a widely-used, yet secretive algorithm is COMPAS (which stands for Correctional Offender Management Profiling for Alternative Sanctions) has been the focus of many legal challenges.[3] COMPAS is a an algorithm that is frequently employed in the criminal justice system that is intended to calculate the risk that offenders would reoffend. In order to help ensure public safety and lower recidivism, the algorithm is used to help guide choices concerning bail, sentence, and parole.

COMPAS was made to try and lessen prejudice and discrimination in the criminal justice system. The algorithm can help in ensuring that choices concerning bail, sentencing, and parole are based on objective criteria rather than on subjective judgments or bias views by employing a standardized, data-driven approach. By doing this, you may make sure that criminals receive proper punishment and that the judicial system if fairly treating everyone despite their differences.

It is thought that the effectiveness of the criminal justice system can be improved by COMPAS. The algorithm can help to ensure that offenders are placed in the proper degree of supervision and that resources are directed to those who are most likely to reoffend by providing a more precise and unbiased estimate of the risk of recidivism. By doing so, you can improve public safety and lower the overall cost of the criminal justice system.

Despite these advantages, there are significant drawbacks to using COMPAS. The algorithm's lack of transparency and difficulty in comprehending how it generates predictions is one of the primary issues. Due to this, it may be challenging to verify if the algorithm is impartial and free from prejudice.

Loomis v. Wisconsin

When deciding if the defendant in Loomis v. Wisconsin should be sentenced to life in prison without the possibility of parole, the trial court took into account the state's employment of a closed-source risk assessment tool to assess the defendant's likelihood of reoffending. There are ethical concerns about fairness, openness, and the possibility of prejudice when closed-source risk assessment tools are used in criminal sentencing. Closed-source tools may be opaque, according to critics, making it challenging for defendants and their attorneys to comprehend the considerations that went into the assessment and contest the validity of the conclusions. Moreover, there are worries that risk assessment tools' closed-source nature may also imply that they are not exposed to the same amount of scrutiny and testing as open-source tools, which could produce unreliable or skewed results.

n preparation for sentencing, a Wisconsin Department of Corrections officer produced a PSI that included a COMPAS risk assessment.COMPAS assessments estimate the risk of recidivism based on both an interview with the offender and information from the offender’s criminal history.As the methodology behind COMPAS is a trade secret, only the estimates of recidivism risk are reported to the court.At Loomis’s sentencing hearing, the trial court referred to the COMPAS assessment in its sentencing determination and, based in part on this assessment, sentenced Loomis to six years of imprisonment and five years of extended supervision.[4]

Medical Algorithms

Advantages of Black Box Algorithms

Ethical Concerns

Bias

In addition to inheriting many of the best traits of the human brain, artificial intelligence has also demonstrated that it knows how to use it skillfully and ably. Object recognition, map navigation, and speech translation are just a few of the many skills that modern AI programs have mastered, and the list will not stop growing anytime soon. [5] Unfortunately, as a result of this it has amplified the bias that humans have. AI can help identify and reduce the impact of human biases, but it can also make the problem worse by baking in and deploying biases at scale in sensitive application areas. [6] For example, Propublica analyzed data from Broward County FLorida and they claimed that the prediction model used there (COMPAS) is racially biased.

Accountability

Another concern is that black box algorithms may lack accountability. Because it is difficult to understand how the algorithm arrives at its predictions, it can be difficult to hold those who created or use the algorithm responsible for any negative consequences. This can make it difficult to ensure that the algorithm is being used ethically and in compliance with laws and regulations.

Transparency

One of the main ethical concerns about black box algorithms is their lack of transparency. The reason they are called Black box algorithms is because they are opaque, meaning that basically impossible to understand how they came up with any of their predictions or decisions. This can make it difficult to ensure that the algorithm is fair and unbiased, and that it is not inadvertently causing harm to certain groups of people. For example, in the medical field if they are unable to figure out where the reliable knowledge is coming from, should they blindly believe that it is correct? There is a growing concern that the black box nature of some algorithms makes it impossible to believe in the reliability of the algorithm and because of this researchers, physicians and patients don't know if they can trust the results of such systems. [7] Biological systems are so complex, and big-data techniques are so opaque, that it can be difficult or impossible to know if an algorithmic conclusion is incomplete, inaccurate, or biased. And these problems can arise due to data limitations, analytical limitations, or even intentional interference.[8]

References