The Black Box Problem in Administering Justice: Risks of Opaque Algorithms in Legal Decision-Making


Abstract views: 2 / PDF downloads: 0

Authors

  • Sojida Murodova Tashkent State University of Law

DOI:

https://doi.org/10.59022/ijlp.331

Keywords:

Black Box Algorithms, Algorithmic Opacity, Legal Decision-Making, Judicial Transparency, AI in the Judiciary, Automated Decision Systems, Algorithmic Accountability, Bias in AI

Abstract

As artificial intelligence (AI) technologies become integrated into judicial systems globally, the “black box problem” has emerged as a critical concern. This term refers to the opacity of machine learning algorithms, where the internal reasoning behind outputs is not transparent to users. This paper explores the implications of the black box problem in the administration of justice, focusing on how opaque algorithmic systems may undermine fairness, accountability, and trust in judicial processes. Drawing from global case studies, expert interviews, and legal theory, the study identifies key risks and proposes safeguards for transparent and ethical AI deployment in courts. The findings highlight the tension between technological efficiency and fundamental legal principles, suggesting that explainable AI must be prioritized to preserve judicial integrity and public confidence in legal institutions.

References

Alexander, M. (2012). The New Jim Crow: Mass incarceration in the age of colorblindness. The New Press.

Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016, May 23). Machine bias. ProPublica.

Babuta, A., Oswald, M., & Rinik, C. (2020). Machine learning algorithms and police decision-making: Legal, ethical and regulatory challenges. Royal United Services Institute.

Barocas, S., & Selbst, A. D. (2016). Big data's disparate impact. California Law Review, 104, 671–732. DOI: https://doi.org/10.2139/ssrn.2477899

Binns, R. (2018). Fairness in machine learning: Lessons from political philosophy. In Proceedings of the 2018 Conference on Fairness, Accountability, and Transparency (FAT).

Bovens, M., & Zouridis, S. (2002). From street‐level to system‐level bureaucracies: How information and communication technology is transforming administrative discretion and constitutional control. Public Administration Review, 62(2), 174–184. DOI: https://doi.org/10.1111/0033-3352.00168

Brayne, S. (2017). Big data surveillance: The case of policing. American Sociological Review, 82(5), 977–1008. DOI: https://doi.org/10.1177/0003122417725865

Burrell, J. (2016). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 1–12. https://doi.org/10.1177/2053951715622512 DOI: https://doi.org/10.1177/2053951715622512

Carnegie Mellon University. (2018, October 11). Amazon scraps secret artificial intelligence recruiting engine that showed biases against women. Machine Learning Department, Carnegie Mellon University. [https://www.ml.cmu.edu/news/news-archive/2016-2020/2018/october/amazon-scraps-secret-artificial-intelligence-recruiting-engine-that-showed-biases-against-women.html](https://www.ml.cmu.edu/news/news-archive/2016-2020/2018/october/amazon-scraps-secret-artificial-intelligence-recruiting-engine-that-showed-biases-against-women.html)

Cerda, L. (2024, November 12). Prometea: Revolutionizing judicial efficiency in Argentina through artificial intelligence powered legal solutions. Inter-American Law Review. [https://inter-american-law-review.law.miami.edu/prometea-revolutionizing-judicial-efficiency-in-argentina-through-artificial-intelligence-powered-legal-solutions/](https://inter-american-law-review.law.miami.edu/prometea-revolutionizing-judicial-efficiency-in-argentina-through-artificial-intelligence-powered-legal-solutions/)

Citron, D., & Pasquale, F. (2014). The scored society: Due process for automated predictions. Washington Law Review, 89(1), 1–33.

Contini, F. (2020). Artificial intelligence and the transformation of humans, law and technology interactions in judicial proceedings. Law, Technology and Humans, 2(1), 4–18. [https://doi.org/10.5204/lthj.v2i1.1478](https://doi.org/10.5204/lthj.v2i1.1478) DOI: https://doi.org/10.5204/lthj.v2i1.1478

Corvalán, J. (n.d.). Prometea: Artificial intelligence to transform justice and public organizations. [https://www.academia.edu/84557698/Prometea_Artificial_intelligence_to_transform_justice_and_public_organizations](https://www.academia.edu/84557698/Prometea_Artificial_intelligence_to_transform_justice_and_public_organizations)

Dias, S. A. de J., & Sátiro, R. M. (2024). Artificial intelligence in the judiciary: A critical view. Futures, 164, 103493. [https://doi.org/10.1016/j.futures.2024.103493](https://doi.org/10.1016/j.futures.2024.103493) DOI: https://doi.org/10.1016/j.futures.2024.103493

European Commission. (2021). Ethics guidelines for trustworthy AI. [https://digital-strategy.ec.europa.eu](https://digital-strategy.ec.europa.eu)

Gao, X. (2019). Chinese justice in the era of artificial intelligence. Journal of Zhejiang University (Humanities and Social Sciences), 49(4), 229–238. (https://doi.org/10.3785/j.issn.1008 942X.2019.04.02](https://doi.org/10.3785/j.issn.1008-942X.2019.04.02)

Garrett, B., & Stevenson, M. (2024). Algorithmic deference: An empirical analysis of judicial decision-making. Yale Law Journal, 133(4), 1021–1078.

Garrett, B. L., & Monahan, J. (2019). Judging risk. California Law Review, 108, 439–494. DOI: https://doi.org/10.2139/ssrn.3190403

Harcourt, B. E. (2007). Against prediction: Profiling, policing, and punishing in an actuarial age. University of Chicago Press. DOI: https://doi.org/10.7208/chicago/9780226315997.001.0001

Hannah-Moffat, K. (2019). Algorithmic risk governance: Big data analytics, race and information activism in criminal justice debates. Theoretical Criminology, 23(4), 453–470. DOI: https://doi.org/10.1177/1362480618763582

Kamarinou, D., Millard, C., & Singh, J. (2024). Algorithm-assisted decision-making in the public sector: A comparative study of three jurisdictions. Public Law, 2024(1), 94–117.

Kostka, G. (2019). China's social credit systems and public opinion: Explaining high levels of approval. New Media & Society, 21(7), 1565–1593. DOI: https://doi.org/10.1177/1461444819826402

Kosinski, M. (2025, April 17). What is black box AI and how does it work? IBM. (https://www.ibm.com/think/topics/black-box-ai)

Ma, L., Christensen, T., & Zheng, Y. (2023). Government technological capacity and public–private partnerships regarding digital service delivery: Evidence from Chinese cities. International Review of Administrative Sciences. DOI: https://doi.org/10.1177/00208523211018849

MacKenzie, D., & Wajcman, J. (Eds.). (2019). The social shaping of technology. Open University Press.

OECD. (2021). The impact of AI on the judicial system: Challenges and opportunities. OECD Publishing. https://www.oecd.org/gov/the-impact-of-ai-on-the-judicial-system.htm

Oswald, M., Grace, J., Urwin, S., & Barnes, G. C. (2018). Algorithmic risk assessment policing models: Lessons from the Durham HART model and “experimental” proportionality. Information & Communications Technology Law, 27(2), 223–250. DOI: https://doi.org/10.1080/13600834.2018.1458455

Papagianneas, S. (2022). Towards smarter and fairer justice? A review of the Chinese scholarship on building smart courts and automating justice. Global Perspectives, 3(1). [https://doi.org/10.1177/18681026211021412](https://doi.org/10.1177/18681026211021412) DOI: https://doi.org/10.1177/18681026211021412

Pasquale, F. (2015). The black box society: The secret algorithms that control money and information. Harvard University Press. DOI: https://doi.org/10.4159/harvard.9780674736061

Peng, J., & Xiang, W. (2020). The rise of smart courts in China: Opportunities and challenges to the judiciary in a digital age. NAVEIÑ REET: Nordic Journal of Law and Social Research, 1(9), 345–372. [https://doi.org/10.7146/nnjlsr.v1i9.122167](https://doi.org/10.7146/nnjlsr.v1i9.122167) DOI: https://doi.org/10.7146/nnjlsr.v1i9.122167

Perry, W. L., McInnis, B., Price, C. C., Smith, S. C., & Hollywood, J. S. (2013). Predictive policing: The role of crime forecasting in law enforcement operations. RAND Corporation. DOI: https://doi.org/10.7249/RR233

Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: 100,000+ questions for machine comprehension of text. arXiv. https://arxiv.org/abs/1606.05250 DOI: https://doi.org/10.18653/v1/D16-1264

Ranchordás, S. (2021). Experimental legislation and the law. In Research Handbook on Experimental Governance (pp. 156–178). Edward Elgar Publishing.

Reiling, D. (2020). Courts and artificial intelligence. International Association for Court Administration. DOI: https://doi.org/10.36745/ijca.343

Roberts, H., Cowls, J., Morley, J., Taddeo, M., Wang, V., & Floridi, L. (2021). The Chinese approach to artificial intelligence: An analysis of policy, ethics, and regulation. AI & Society, 36(1), 59–77. DOI: https://doi.org/10.1007/s00146-020-00992-2

Singh, J., Cobbe, J., & Lee, M. S. A. (2023). Recursive bias in justice: When algorithms learn from algorithmically influenced decisions. In Proceedings of the 2023 Conference on Fairness, Accountability, and Transparency (pp. 112–124).

Susskind, R. (2017). Tomorrow’s lawyers: An introduction to your future. Oxford University Press.

State v. Loomis, 881 N.W.2d 749 (Wis. 2016). https://law.justia.com/cases/wisconsin/supreme-court/2016/2015ap000157-cr.html

Vainsalu, H. (2021). AI in the public sector: Experiences from Estonia. e-Governance Academy Report.

Wang, N. (2020). “Black box justice”: Robot judges and AI-based judgment processes in China’s court system. In 2020 IEEE International Symposium on Technology and Society (ISTAS) (pp. 58–65). IEEE. [https://doi.org/10.1109/ISTAS50296.2020.9462216](https://doi.org/10.1109/ISTAS50296.2020.9462216) DOI: https://doi.org/10.1109/ISTAS50296.2020.9462216

World Justice Project. (2024). Global insights on legal technology: Annual report 2024. World Justice Project Publications.

Zeleznikow, J. (2017). Can artificial intelligence and online dispute resolution enhance efficiency and effectiveness in courts? International Journal for Court Administration, 8(2), 30–45. DOI: https://doi.org/10.18352/ijca.223

Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontiers of power. PublicAffairs.

Published

2025-06-30

How to Cite

Murodova, S. (2025). The Black Box Problem in Administering Justice: Risks of Opaque Algorithms in Legal Decision-Making. International Journal of Law and Policy, 3(6), 1–20. https://doi.org/10.59022/ijlp.331

Issue

Section

Articles