Спиридонов Михаил Сергеевич – кандидат юридических наук, доцент кафедры уголовного процесса, криминалистики и судебной экспертизы
WoS Researcher ID: https://www.webofscience.com/wos/author/record/AAK-9097-2021
РИНЦ Author ID: https://elibrary.ru/author_items.asp?authorid=1089837
454080, Россия, Челябинск, проспект Ленина, 76
Mikhail S. Spiridonov – Cand. Sci. (Law), Associate Professor, Department of Criminal Procedure, Criminology and Forensics
WoS Researcher ID:
RSCI Author ID:
76 prospekt Lenina, 454080 Chelyabinsk
Objective: to experimentally check the ability of publicly available neural networks to solve formalized criminal law problems with a pre-established normatively correct result.
Methods: a set of complementary methods of scientific cognition helped to achieve the work objective. The methods of analysis and synthesis, induction and deduction formed the general scientific basis, which made it possible to systematically comprehend the issues under study. Among special legal tools were formal legal analysis and official interpretation of legal norms, which ensured the rigorous normative assessment of the results obtained. The key empirical research method was a controlled experiment, organically combined with modeling law enforcement situations and a comparative analysis of the answers of six publicly available neural networks to identical criminal law problems.
Results: during the experiment, publicly available neural networks showed significant discrepancies in the accuracy and consistency of answers to formalized criminal law problems: none of the tested models demonstrated a stable and error-free result. In the absence of direct reference to the relevant legal sources, the models systematically made mistakes when determining the term of conviction expungement, applying the rules for sentencing, and determining the type of recidivism of crimes. This indicates reproductive rather than analytical-legal nature of the models. Providing accurate quotations from regulations and explanations of the Russian Supreme Court Plenum significantly improves the correctness of answers from certain neural networks. The most and least effective models were identified, as well as the basic requirements for drafting a legally correct query in the field of criminal proceedings.
Scientific novelty: the study is an attempt to experimentally check the capabilities of publicly available neural networks in relation to specific criminal law problems with a pre-established normatively correct answer. The results obtained made it possible to propose the typology of errors made by neural networks, reveal their procedural causes, and identify the fundamental limitations of using generative artificial intelligence in law enforcement.
Practical significance: the results can be used in law enforcement and education: to determine the acceptable limits of using publicly available neural networks in criminal proceedings; to develop methodological recommendations for making legally correct queries to generative artificial intelligence systems; and to prevent typical errors when using neural networks in professional legal activity.
Цель: исследование направлено на экспериментальную проверку способности общедоступных нейронных сетей решать формализованные задачи уголовного права с заранее установленным нормативно корректным результатом.
Методы: для достижения поставленной цели применялся комплекс взаимодополняющих методов научного познания. Общенаучную основу исследования составили методы анализа и синтеза, индукции и дедукции, позволившие системно осмыслить изучаемую проблематику. В рамках специального юридического инструментария использовались формально-юридический анализ и официальное толкование норм права, что обеспечило строгость нормативной оценки полученных результатов. Ключевым эмпирическим методом исследования выступил контролируемый эксперимент, органично сочетавшийся с моделированием правоприменительных ситуаций и сравнительным анализом ответов шести общедоступных нейронных сетей на идентичные уголовно-правовые задачи.
Результаты: в ходе проведенного эксперимента установлено, что общедоступные нейронные сети обнаруживают существенные расхождения в точности и последовательности ответов при решении формализованных задач уголовного права: ни одна из тестируемых моделей не продемонстрировала стабильного и безошибочного результата. Выявлено, что в отсутствие прямого указания на соответствующие правовые источники модели систематически допускают ошибки при определении момента погашения судимости, применении правил назначения наказания и установлении вида рецидива преступлений, что свидетельствует об их репродуктивном, а не аналитико-правовом характере. Предоставление точных цитат из нормативных актов и разъяснений Пленума Верховного Суда Российской Федерации существенно повышает корректность ответов отдельных нейронных сетей. Определены наиболее и наименее результативные модели, а также сформулированы основные требования к составлению юридически корректного запроса в сфере уголовного судопроизводства.
Научная новизна: настоящее исследование представляет собой попытку экспериментальной верификации возможностей общедоступных нейронных сетей применительно к конкретным задачам уголовного права с заранее известным нормативно верным ответом. Полученные результаты позволили выявить типологию воспроизводимых нейронными сетями ошибок, раскрыть их процессуальные причины, а также обозначить принципиальные ограничения использования генеративного искусственного интеллекта в правоприменительной деятельности.
Практическая значимость: результаты исследования могут использоваться в правоприменительной и образовательной деятельности: для определения допустимых границ применения общедоступных нейронных сетей в уголовном судопроизводстве, разработки методических рекомендаций по составлению юридически грамотных запросов к системам генеративного искусственного интеллекта, а также в целях предупреждения типовых ошибок при обращении к нейронным сетям в ходе профессиональной юридической деятельности.
There were several reasons for writing this article. The first reason is the growing discourse among judicial officials about the incredible prospects of using artificial intelligence (AI) tools in court proceedings and using them to improve the work of both the judicial system in general and any single judge in Russia. This suggests that these statements either are based on some unpublished research that has shown amazing results in the AI effectiveness in legal practice, or, more likely, are a manifestation of the general naively positive opinion about AI. The second reason is the personal curiosity of the author. It seems very tempting to shift some of the work of a professional criminal lawyer to a virtual assistant, especially when their knowledge base and search abilities are promised to far exceed the average human capabilities.
At first glance, the possibility of using AI in various fields, for example, in legal education (Danielyan, 2024) or in checking local draft laws (Ke Wang, 2023), really does not seem a fantastic and remote prospect. Along with this, recent publications have expressed opinions about the limited capabilities of these systems (Callister, 2020). They note the opacity and unpredictability of AI technologies, potential risks to people’s rights and guarantees, such as privacy, the right to non-discrimination, and a fair trial (Buzova, 2024; Vlasova, 2025; Vorozhevich, 2025; Dedov, 2023; Zharova, 2025; Kalyatin, 2024; Karczkhiya, 2024; Kravchenko, 2025; Chebodaeva, 2023; Farinella & Gulyaeva, 2024). Researchers skeptically assess the potential of these technologies in resolving legal conflicts, stating their greater usefulness as a means of documenting and systematizing regulations (Navarro-Dolmestch & Fuentes-Loureiro, 2023). Some authors also reasonably point out the risks associated with the use of generative AI in justice, such as generating incorrect but plausible results (Kirpichev, 2024).
The relevance of the topic is confirmed by the fact that publications on the AI practical application very poorly reflect legal sphere (Avdoshin et al., 2024; Avetisyan, 2024; Andrianov et al., 2024; Baryshnikov, 2022; Betelin, 2024; Eremin & Selenginsky, 2023; Kobrinskii, 2024; Oborotistov & Muraev, 2023; Orlov, 2025; Raikov, 2024; Razumov & Dus, 2024; Rimshin & Kucherenko, 2024; Sayfullin et al., 2023; Sozaeva, 2024). It is most interesting to experimentally study the capabilities of modern publicly available neural networks in solving ordinary legal problems that may arise in a criminal case. It is worth noting that we are interested in exactly those neural networks that anyone can freely use today. If these tools can be used to correctly answer certain legal questions, this will indicate a prospect of developing specialized neural networks for a specific area of law enforcement. Accordingly, this is the thesis that this study aims to verify.
The following publicly available neural networks (further – NNs) (as of November 2025) were used for this study:
– YaGPT (YandexGPT);
– ChatGPT 5;
– Claude Sonnet 4.5;
– GigaChat;
– DeepSeek;
– Perplexity (actually, an aggregator of several neural networks).
These NNs’ functioning is based on the transformer architecture, when a model trained on a huge set of data analyzes the context and generates text. It identifies statistical patterns in the language and builds mathematical connections between words, i.e. simulates understanding through calculations.
These tools employ such methods of automatic text processing in natural language as supervised learning, unsupervised learning, and partially supervised learning (Belov et al., 2020). Large language models are generators of tokens (a minimal unit of text to which a numerical representation is assigned) able to consistently predict them, thereby completing documents. At the same time, despite the current development of large language models, the basic principle of their operation remains the same – it is a tool for completing documents (Berryman & Ziegler, 2025).
Hence, when formulating a request for such an NN, it is necessary to convey the user’s task and the accompanying context so that the model can help find a solution.
The publicly available NNs selected for this study appear in the form of a dialog box in which the user writes a question (request) and receives a written answer to it. Given this mode of operation, the user’s request must meet certain standards to obtain a correct result. The quality of the query correlates with the accuracy (correctness) of the NN’s response.
Currently, there is a wide variety of literature on making queries to NNs (prompt engineering), which describes the basic requirements for them. For example, some authors suggest the following order of making queries to NNs: set the area (describe in detail the desired style or specify the appropriate personality); specify the format (define the rules that the model should follow and the response structure); give examples (add a few examples of the correct solution to the problem); evaluate the quality (find errors, evaluate the answers and determine what affects the response quality); divide the task into subtasks (break down complex tasks into related stages) (Phoenix & Taylor, 2025). One can also find such requirements for a prompt: initially, clearly identify the purpose of the request; think over the desired outcome; determine the format for presenting the outcome; structured the request into paragraphs; use precise wording and specialized terms in the request; explicitly specify the initial conditions and set the level of response detailization; set instructions and rules at the beginning of the request (Kuzmenko, 2025).
Hence, based on the above requirements, the following request was made:
1. Read the question carefully.
2. Answer the question with maximum accuracy.
3. For each key part/thesis of your answer:
– Evaluate confidence (on a 1–100 scale).
– Clearly state whether this is accurate knowledge or an assumption/logical conclusion.
4. If confidence in any part is <70/100, obligatorily suggest an alternative or clearly warn about possible inaccuracy/incompleteness.
5. Formulate three different answers to this question, each with a confidence score as indicated. Choose the most reliable answer.
6. Provide sources, if known.
7. Your role is a judge of a Russian district or city court. You are well-versed in the criminal-procedural legislation of Russia, and you have extensive theoretical and practical experience.
8. In responding to this request, refer to: the Criminal Procedural Code of the Russian Federation; the Criminal Code of the Russian Federation; Resolutions of the Plenum of the Supreme Court of the Russian Federation, including Resolution of the Plenum of the Supreme Court of the Russian Federation No. 14 dated 06/07/2022, and Russian judicial practice on this issue (take practice only from verified sources – https://www.vsrf.ru, www.consultant.ru, https://www.garant.ru, https://sudact.ru, websites of the Russian general jurisdiction courts); always check the relevance of the information.
9. If the information (regulations, online resources, etc.) may be outdated or invalid, obligatorily report it.
10. Carefully study all the contents of the sources, without missing a single point.
11. Make sure that every detail, question or point presented in the materials is taken into account and analyzed.
12. Answer format:
12.1. Give an answer with a detailization level of 100 out of 10.
12.2. When responding, use clear, non-ambiguous language.
12.3. All notes about possible outdated information and verification of sources should be explicit and understandable to the user.
12.4. When responding, reflect the received information as accurately as possible, relying on all sections and details of sources, files and text.
12.5. If the analysis shows that the legal issue does not have an unambiguous answer (i.e., there are varying judicial practices, conflicting legal norms or expert opinions on it), clearly indicate that the issue is debatable, which point of view is preferable and why.
12.6. Do not pass off hypotheses or guesses as an established legal fact.
Using the above request, neural networks were asked to solve the following criminal law problems:
Ivanov, an adult citizen of Russia, was sentenced by the verdict of the Kalininsky District Court of Ufa dated 12.10.2021 under Part 1 of Article 264.1 of the Russian Criminal Code to 1 year of imprisonment on probation for 1 year, with deprivation of the right to engage in activities related to driving vehicles for a period of 1 year and 6 months. The verdict entered into force on 12.11.2021. Ivanov was again detained for driving under alcohol on 01.06.2023. Question: Can Ivanov’s actions be qualified as a crime, and if so, under what article of the Russian Criminal Code will they be qualified?
An adult citizen Petrov was convicted twice.: 1) on 05.11.2023, by the Ivanovo District Court under paragraph “b” of Part 2 of Article 158 of the Russian Criminal Code to 6 months of imprisonment in a general regime correctional colony, released on 05.05.2024 after serving his sentence; 2) on 12.08.2024, by the Ivanovo District Court under Part 1 of Article 161 of the Russian Criminal Code to 1 year of imprisonment to be served in a high-security penal colony. On 10.06.2024, Petrov stole a jacket worth 10,000 rubles. For this crime, he was sentenced on 10.09.2024 by the Ivanovo District Court under paragraph “b” of Part 2 of Article 158 of the Russian Criminal Code to 1 year in prison. Based on Article 70 of the Russian Criminal Code, by the totality of sentences, the unserved part of the sentence of 12.08.2024 was partially attached to the imposed punishment. One year and 6 months of imprisonment were finally imposed, to be served in a high-security penal colony. According to the latest verdict, the court recognized Petrov’s actions as a dangerous recurrence of crimes. Questions: 1) is the final punishment correctly imposed in the verdict of 10.09.2024? 2) is the recidivism of crimes correctly established in this verdict?
Both problems are taken from real judicial practice. The first is based on criminal case No. 1-230/2022 (UID 76RS0013-01-2022-000945-09) in relation to a citizen Chernykh, considered by the Rybinsk City Court of the Yaroslavl region on 18.04.2022 with a verdict of guilty.
The circumstances of the case are as follows: Chernykh was sentenced on 03.07.2018 by the magistrate of the judicial district No. 3 of the Rybinsk Judicial district of the Yaroslavl region under Article 264.1 of the Russian Criminal Code to 11 months of imprisonment, suspended for a probation period of 2 (two) years with deprivation of the right to engage in activities related to driving vehicles for a period of 2 (two) years and 8 (eight) months. The imprisonment was served on 03.07.2020, the deprivation of the right to engage in activities related to driving was served on 13.03.2021. After that, on 02.02.2022 at about 00:55 am, Chernykh in a state of alcoholic intoxication drove a Renault Sandero car near 33 Tolbukhina Street in the city of Rybinsk, Yaroslavl region. The court of first instance qualified these actions as driving a car by a person who is intoxicated and has a criminal record for committing a crime under Article 264.1 of the Russian Criminal Code.
The Court of Cassation did not agree with the verdict and pointed out that Chernykh’s criminal record under the verdict of 03.07.2018 had been extinguished at the time of the new deed commission. This should exclude his criminal liability under Part 2 of Article 264.1 of the Russian Criminal Code, where the presence of a criminal record is a mandatory sign of a crime (Cassation Resolution of the General Jurisdiction Cassation Court No. 2 No. 77-1487/2024 dated 04.06.2024). The Cassation Court decision was based on paragraph 14 of the Resolution No. 14 of the Russian Supreme Court Plenum dated 07.06.2022 “On the practice of courts applying legislation regulating the calculation of the repayment period and the procedure for removing criminal records”.
A similar example is criminal case No. 1-74/2024 (UID 50RS0036-01-2024-000324-74) in relation to a citizen Malofeev, considered by the Pushkin City Court of the Moscow Region on 12.02.2024 with a verdict of guilty, which was also overturned on cassation on a similar basis (Cassation Resolution of the General Jurisdiction Cassation Court No. 1 No. 77-1251/2025 dated 16.04.2025).
Thus, the short answer to the first problem will be as follows: there is no corpus delicti in Ivanov’s actions, since the criminal record under the first sentence was extinguished on 13.05.2023.
Assessment criteria for the first problem are: the absence of corpus delicti is correctly determined and the correct justification is given by expunging the criminal record the next day after serving (executing) the additional sentence. If both criteria are met, the answer is assessed as 100%; if only one criterion is met, the answer is assessed as 50%.
The second problem is based on paragraph 5 of clause 27 of the Resolution of the Russian Supreme Court Plenum dated 29.11.2016 No. 55 “On the court verdict” (stating the need to determine the type of recidivism when assigning a type of correctional institution) and clause 52 of the Resolution of the Russian Supreme Court Plenum dated 22.12.2015 No. 58 “On the practice of assigning criminal punishment by the courts of the Russian Federation” (on applying general rules in the imposition of punishment for a set of crimes to a person who committed another crime before sentencing in the first case).
Based on this, the short answer to the second problem will be as follows: according to Article 70 of the Russian Criminal Code, the final punishment by the totality of sentences was assigned to Petrov incorrectly, since it was necessary to apply Part 5 of Article 69 of the Russian Criminal Code (the imposition of punishment by the totality of crimes). Dangerous recidivism was incorrectly determined, since at the time of the crime commission there was only one criminal record, forming a relapse. Hence, Petrov’s actions are a simple recidivism.
Assessment criteria for the second problem are: the error in the application of Article 70 of the Russian Criminal Code and in determining the type of relapse was correctly identified. If both criteria are met, the answer is assessed as 100%; if only one criterion is met, the answer is assessed as 50%.
As one can see, the solution of the above problems generally depends on the exact definition of the rule to be applied (in the first problem – the rule for calculating the repayment period of a criminal record, in the second problem – the rule for sentencing with multiple crimes) and the correctness of calculations (in the first problem – determining the period of the additional punishment execution and the date of a criminal record repayment; in the second problem – determining the chronology of crimes and sentences). Thus, these problems are based on analyzing precisely defined parameters, and therefore require calculations and exclude the need to formulate evaluative judgments. The problem statements provide comprehensive information that allows one to give a complete and correct answer.
None of the NNs gave a 100% correct answer (Table 1). The closest was Claude Sonnet 4.5, which pointed out that the deed cannot be qualified under Part 2 of Article 264.1 of the Russian Criminal Code, since the criminal record was expunged after the probation period. The assessment for this answer is 50%, because the repayment time of the criminal record was determined incorrectly. Other models did not cope with this problem. DeepSeek gave the least accurate answer, since, instead of expunging a criminal record, it provided judgments about the administrative punishment effect.
Table 1. Answers to Problem 1. Attempt 1
| Model | Brief answer | Assessment (%) |
| Chat GPT 5 | Ivanov’s actions may be qualified under Part 2 of Article 264.1 of the Russian Criminal Code if the criminal record has not yet been expunged | 0 |
| Claude Sonnet 4.5 | Qualification under Part 2 of Article 264.1 of the Russian Criminal Code is impossible, since the criminal record was expunged after the probation period | 50 |
| YaGPT | Ivanov’s actions should be qualified under Part 2 of Article 264.1 of the Russian Criminal Code since the criminal record has not yet been expunged | 0 |
| Perplexity | Ivanov’s actions should be qualified under Part 2 of Article 264.1 of the Russian Criminal Code since the criminal record has not yet been expunged | 0 |
| GigaChat | Ivanov’s actions should be qualified under Part 2 of Article 264.1 of the Russian Criminal Code since the criminal record has not yet been expunged | 0 |
| DeepSeek | Ivanov’s actions should be qualified under Part 2 of Article 264.1 of the Russian Criminal Code since he drove a vehicle in a state of alcohol intoxication during the administrative punishment in the form of deprivation of a special right | 0 |
Apparently, the NNs failed to reach the text of the above-mentioned clause 14 of the Resolution of the Russian Supreme Court Plenum dated 06/07/2022 No. 14 “On the practice of courts applying legislation regulating the calculation of the repayment period and the procedure for removing criminal records”. In this regard, a second attempt was made, during which each model was provided with an accurate quote of the said legal norm. The results changed (Table 2). On the second attempt, Claude Sonnet 4.5, YaGPT and ChatGPT gave a 100% correct answer. They correctly stated that there was no corpus delicti in Ivanov’s actions due to the expungement of his conviction. At the same time, these models recognized the mistake in the first attempt and correctly applied the quote provided to them. Also, the first two models correctly determined the repayment date of the criminal record, and ChatGPT was mistaken in this, indicating a month earlier. This did not affect the answer correctness, so the result was accepted.
Table 2. Answers to Problem 1. Attempt 2
Note: given the above commentary.
| Model | Brief answer | Assessment (%) |
| Chat GPT 5 | There is no corpus delicti in Ivanov’s actions, as his previous conviction was expunged on 13.04.2023 | 100* |
| Claude Sonnet 4.5 | Ivanov’s actions at the time of his arrest on 01.06.2023 cannot be qualified as a crime, since 19 days passed since the expungement of his criminal record. | 100 |
| YaGPT | Ivanov’s criminal record was expunged on 13.05.2023, and his actions, committed on 01.06.2023, cannot be qualified under Part 2 of Article 264.1 of the Russian Criminal Code due to the absence of an outstanding criminal record at the time of the act | 100 |
| Perplexity | Ivanov’s criminal record will be considered expunged only on 13.05./2024, therefore, his actions should be qualified under Part 2 of art. 264.1 of the Russian Criminal Code | 0 |
| GigaChat | Ivanov remained convicted until 12.11.2023, therefore, his actions constitute a crime under Part 1 of art. 264.1 of the Russian Criminal Code | 0 |
| DeepSeek | On 01.06.2023, Ivanov’s criminal record was expunged, the term of administrative punishment in the form of deprivation of the right expired, and his actions should be qualified under Part 2 of Article 264.1 of the Russian Criminal Code | 0 |
This time, the least accurate answer could not be determine, since the answers from Perplexity, GigaChat and DeepSeek were equally incorrect. It is only worth noting that the first two models continued to consider Ivanov to be convicted, that is, they ignored the quote of the legal position provided to them; DeepSeek misunderstood its content, considering that the criminal record repayment had no legal significance in answering the question.
The questions posed in the second problem did not cause difficulties for Claude Sonnet 4.5 and ChatGPT – these models gave 100% correct answers (Table 3).
Table 3. Answers to Problem 2
| Model | Brief answer | Assessment (%) |
| Chat GPT 5 | The final punishment was determined unlawfully, since the Court applied Article 70 of the Russian Criminal Code, although it should have applied Article 69 of the Russian Criminal Code: the crime was committed before the previous sentence was pronounced; dangerous recidivism was determined incorrectly, since Petrov had only one criminal record at the time of the crime | 100 |
| Claude Sonnet 4.5 | The final punishment was imposed incorrectly, as the Court mistakenly applied Article 70 of the Russian Criminal Code (totality of sentences) instead of Article 69 of the Russian Criminal Code (totality of crimes). Establishing a dangerous recidivism is wrong. There is a simple recidivism in Petrov’s actions | 100 |
| YaGPT | The final punishment was set correctly. A dangerous recidivism was set incorrectly, since Petrov had only one criminal record at the time of the crime, so there was no recidivism | 0 |
| Perplexity | The punishment was imposed correctly, and the recidivism was correctly established | 0 |
| GigaChat | The application of Article 70 of the Russian Criminal Code was legitimate, but the type of correctional institution was incorrectly defined. The recognition of a dangerous relapse was unjustified, since there is no relapse at all in Petrov’s actions | 0 |
| DeepSeek | The final punishment was assigned incorrectly in terms of the type of correctional facility. The presence of a dangerous recidivism was incorrectly established, the recidivism should have been recognized as simple | 50 |
Perplexity, YaGPT, DeepSeek and GigaChat did not see a mistake in applying the provisions of Article 70 of the Russian Criminal Code when imposing final punishment. At the same time, the last three models “doubted” the correctness of the relapse type, with YaGPT and GigaChat coming to the wrong conclusion about the absence of recidivism at all. DeepSeek was more accurate, indicating the presence of a simple recidivism, so its answer was assessed as 50% correct. Therefore, Perplexity gave the least accurate answer.
Thus, the most accurate answers to each problem were formulated by Claude Sonnet 4.5 (both problems were solved, all attempts were successful). The next most accurate was Chat GPT 5 (both problems were solved, but the first one with a reservation), followed by YaGPT (one problem was solved on the second attempt) and DeepSeek (the second problem was partially solved).
Since the problems were correctly solved by several NNs at once, the random nature of the results can be excluded. The reasoning of the answers provided by the models also shows that they were obtained through computational processes, rather than random issuance. This demonstrates the real capabilities of the tested NNs, which directly correlate with the amount of data loaded into them and the settings of their computational algorithms.
The above test results make it possible to formulate the following conclusions:
The authors declare that there are no conflicts of interest present.