The Black Box Problem in Administering Justice: Risks of Opaque Algorithms in Legal Decision-Making
Abstract views: 29
/ 
PDF downloads: 17
DOI:
https://doi.org/10.59022/ijlp.331Keywords:
Black Box Algorithms, Algorithmic Opacity, Legal Decision-Making, Judicial Transparency, AI in the Judiciary, Automated Decision Systems, Algorithmic Accountability, Bias in AIAbstract
As artificial intelligence (AI) technologies become integrated into judicial systems globally, the “black box problem” has emerged as a critical concern. This term refers to the opacity of machine learning algorithms, where the internal reasoning behind outputs is not transparent to users. This paper explores the implications of the black box problem in the administration of justice, focusing on how opaque algorithmic systems may undermine fairness, accountability, and trust in judicial processes. Drawing from global case studies, expert interviews, and legal theory, the study identifies key risks and proposes safeguards for transparent and ethical AI deployment in courts. The findings highlight the tension between technological efficiency and fundamental legal principles, suggesting that explainable AI must be prioritized to preserve judicial integrity and public confidence in legal institutions.
References
Alexander, M. (2012). The New Jim Crow: Mass incarceration in the age of colorblindness. The New Press.
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016, May 23). Machine bias. ProPublica.
Babuta, A., Oswald, M., & Rinik, C. (2020). Machine learning algorithms and police decision-making: Legal, ethical and regulatory challenges. Royal United Services Institute.
Barocas, S., & Selbst, A. D. (2016). Big data's disparate impact. California Law Review, 104, 671–732. DOI: https://doi.org/10.2139/ssrn.2477899
Binns, R. (2018). Fairness in machine learning: Lessons from political philosophy. In Proceedings of the 2018 Conference on Fairness, Accountability, and Transparency (FAT).
Bovens, M., & Zouridis, S. (2002). From street‐level to system‐level bureaucracies: How information and communication technology is transforming administrative discretion and constitutional control. Public Administration Review, 62(2), 174–184. DOI: https://doi.org/10.1111/0033-3352.00168
Brayne, S. (2017). Big data surveillance: The case of policing. American Sociological Review, 82(5), 977–1008. DOI: https://doi.org/10.1177/0003122417725865
Burrell, J. (2016). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 1–12. https://doi.org/10.1177/2053951715622512 DOI: https://doi.org/10.1177/2053951715622512
Carnegie Mellon University. (2018, October 11). Amazon scraps secret artificial intelligence recruiting engine that showed biases against women. Machine Learning Department, Carnegie Mellon University. [https://www.ml.cmu.edu/news/news-archive/2016-2020/2018/october/amazon-scraps-secret-artificial-intelligence-recruiting-engine-that-showed-biases-against-women.html](https://www.ml.cmu.edu/news/news-archive/2016-2020/2018/october/amazon-scraps-secret-artificial-intelligence-recruiting-engine-that-showed-biases-against-women.html)
Cerda, L. (2024, November 12). Prometea: Revolutionizing judicial efficiency in Argentina through artificial intelligence powered legal solutions. Inter-American Law Review. [https://inter-american-law-review.law.miami.edu/prometea-revolutionizing-judicial-efficiency-in-argentina-through-artificial-intelligence-powered-legal-solutions/](https://inter-american-law-review.law.miami.edu/prometea-revolutionizing-judicial-efficiency-in-argentina-through-artificial-intelligence-powered-legal-solutions/)
Citron, D., & Pasquale, F. (2014). The scored society: Due process for automated predictions. Washington Law Review, 89(1), 1–33.
Contini, F. (2020). Artificial intelligence and the transformation of humans, law and technology interactions in judicial proceedings. Law, Technology and Humans, 2(1), 4–18. [https://doi.org/10.5204/lthj.v2i1.1478](https://doi.org/10.5204/lthj.v2i1.1478) DOI: https://doi.org/10.5204/lthj.v2i1.1478
Corvalán, J. (n.d.). Prometea: Artificial intelligence to transform justice and public organizations. [https://www.academia.edu/84557698/Prometea_Artificial_intelligence_to_transform_justice_and_public_organizations](https://www.academia.edu/84557698/Prometea_Artificial_intelligence_to_transform_justice_and_public_organizations)
Dias, S. A. de J., & Sátiro, R. M. (2024). Artificial intelligence in the judiciary: A critical view. Futures, 164, 103493. [https://doi.org/10.1016/j.futures.2024.103493](https://doi.org/10.1016/j.futures.2024.103493) DOI: https://doi.org/10.1016/j.futures.2024.103493
European Commission. (2021). Ethics guidelines for trustworthy AI. [https://digital-strategy.ec.europa.eu](https://digital-strategy.ec.europa.eu)
Gao, X. (2019). Chinese justice in the era of artificial intelligence. Journal of Zhejiang University (Humanities and Social Sciences), 49(4), 229–238. (https://doi.org/10.3785/j.issn.1008 942X.2019.04.02](https://doi.org/10.3785/j.issn.1008-942X.2019.04.02)
Garrett, B., & Stevenson, M. (2024). Algorithmic deference: An empirical analysis of judicial decision-making. Yale Law Journal, 133(4), 1021–1078.
Garrett, B. L., & Monahan, J. (2019). Judging risk. California Law Review, 108, 439–494. DOI: https://doi.org/10.2139/ssrn.3190403
Harcourt, B. E. (2007). Against prediction: Profiling, policing, and punishing in an actuarial age. University of Chicago Press. DOI: https://doi.org/10.7208/chicago/9780226315997.001.0001
Hannah-Moffat, K. (2019). Algorithmic risk governance: Big data analytics, race and information activism in criminal justice debates. Theoretical Criminology, 23(4), 453–470. DOI: https://doi.org/10.1177/1362480618763582
Kamarinou, D., Millard, C., & Singh, J. (2024). Algorithm-assisted decision-making in the public sector: A comparative study of three jurisdictions. Public Law, 2024(1), 94–117.
Kostka, G. (2019). China's social credit systems and public opinion: Explaining high levels of approval. New Media & Society, 21(7), 1565–1593. DOI: https://doi.org/10.1177/1461444819826402
Kosinski, M. (2025, April 17). What is black box AI and how does it work? IBM. (https://www.ibm.com/think/topics/black-box-ai)
Ma, L., Christensen, T., & Zheng, Y. (2023). Government technological capacity and public–private partnerships regarding digital service delivery: Evidence from Chinese cities. International Review of Administrative Sciences. DOI: https://doi.org/10.1177/00208523211018849
MacKenzie, D., & Wajcman, J. (Eds.). (2019). The social shaping of technology. Open University Press.
OECD. (2021). The impact of AI on the judicial system: Challenges and opportunities. OECD Publishing. https://www.oecd.org/gov/the-impact-of-ai-on-the-judicial-system.htm
Oswald, M., Grace, J., Urwin, S., & Barnes, G. C. (2018). Algorithmic risk assessment policing models: Lessons from the Durham HART model and “experimental” proportionality. Information & Communications Technology Law, 27(2), 223–250. DOI: https://doi.org/10.1080/13600834.2018.1458455
Papagianneas, S. (2022). Towards smarter and fairer justice? A review of the Chinese scholarship on building smart courts and automating justice. Global Perspectives, 3(1). [https://doi.org/10.1177/18681026211021412](https://doi.org/10.1177/18681026211021412) DOI: https://doi.org/10.1177/18681026211021412
Pasquale, F. (2015). The black box society: The secret algorithms that control money and information. Harvard University Press. DOI: https://doi.org/10.4159/harvard.9780674736061
Peng, J., & Xiang, W. (2020). The rise of smart courts in China: Opportunities and challenges to the judiciary in a digital age. NAVEIÑ REET: Nordic Journal of Law and Social Research, 1(9), 345–372. [https://doi.org/10.7146/nnjlsr.v1i9.122167](https://doi.org/10.7146/nnjlsr.v1i9.122167) DOI: https://doi.org/10.7146/nnjlsr.v1i9.122167
Perry, W. L., McInnis, B., Price, C. C., Smith, S. C., & Hollywood, J. S. (2013). Predictive policing: The role of crime forecasting in law enforcement operations. RAND Corporation. DOI: https://doi.org/10.7249/RR233
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: 100,000+ questions for machine comprehension of text. arXiv. https://arxiv.org/abs/1606.05250 DOI: https://doi.org/10.18653/v1/D16-1264
Ranchordás, S. (2021). Experimental legislation and the law. In Research Handbook on Experimental Governance (pp. 156–178). Edward Elgar Publishing.
Reiling, D. (2020). Courts and artificial intelligence. International Association for Court Administration. DOI: https://doi.org/10.36745/ijca.343
Roberts, H., Cowls, J., Morley, J., Taddeo, M., Wang, V., & Floridi, L. (2021). The Chinese approach to artificial intelligence: An analysis of policy, ethics, and regulation. AI & Society, 36(1), 59–77. DOI: https://doi.org/10.1007/s00146-020-00992-2
Singh, J., Cobbe, J., & Lee, M. S. A. (2023). Recursive bias in justice: When algorithms learn from algorithmically influenced decisions. In Proceedings of the 2023 Conference on Fairness, Accountability, and Transparency (pp. 112–124).
Susskind, R. (2017). Tomorrow’s lawyers: An introduction to your future. Oxford University Press.
State v. Loomis, 881 N.W.2d 749 (Wis. 2016). https://law.justia.com/cases/wisconsin/supreme-court/2016/2015ap000157-cr.html
Vainsalu, H. (2021). AI in the public sector: Experiences from Estonia. e-Governance Academy Report.
Wang, N. (2020). “Black box justice”: Robot judges and AI-based judgment processes in China’s court system. In 2020 IEEE International Symposium on Technology and Society (ISTAS) (pp. 58–65). IEEE. [https://doi.org/10.1109/ISTAS50296.2020.9462216](https://doi.org/10.1109/ISTAS50296.2020.9462216) DOI: https://doi.org/10.1109/ISTAS50296.2020.9462216
World Justice Project. (2024). Global insights on legal technology: Annual report 2024. World Justice Project Publications.
Zeleznikow, J. (2017). Can artificial intelligence and online dispute resolution enhance efficiency and effectiveness in courts? International Journal for Court Administration, 8(2), 30–45. DOI: https://doi.org/10.18352/ijca.223
Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontiers of power. PublicAffairs.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Sojida Murodova
This work is licensed under a Creative Commons Attribution 4.0 International License.
