Using Digital Tools in Government Procurement Analysis: Detecting Suspicious Purchases with Control Indicators
Published at : 29 Dec 2023
Volume : IJtech
Vol 14, No 8 (2023)
DOI : https://doi.org/10.14716/ijtech.v14i8.6851
Mokeeva, T., Yurko, K., 2023. Using Digital Tools in Government Procurement Analysis: Detecting Suspicious Purchases with Control Indicators. International Journal of Technology. Volume 14(8), pp. 1821-1830
| Tatiana Mokeeva | Graduate School of Industrial Economics, Institute of Industrial Management, Economics and Trade, Peter the Great St. Petersburg Polytechnic University, Polytechnicheskaia Street, 29, Saint Petersbur |
| Ksenia Yurko | Graduate School of Industrial Economics, Institute of Industrial Management, Economics and Trade, Peter the Great St. Petersburg Polytechnic University, Polytechnicheskaia Street, 29, Saint Petersbur |
Contract system; Digital technologies; Digitization; Information resources; Public procurement
In the rapidly evolving digital era, technology is becoming increasingly important for government entities. The digital transformation and implementation of modern digital technologies encompass virtually all industries, particularly in the realm of government procurement. Digitization in the procurement system allows for the expansion of market boundaries, attracting a greater number of procurement participants and ensuring principles of openness and transparency in information. Digitization also opens up new, more efficient possibilities for both pre- and post-contract monitoring. Additionally, it enables the expansion of analytics for large volumes of data.
Scientists from around the world are studying the impact of digitization on
government processes. For instance, Berawi et
al. (2021) proposed a blockchain-based data storage system model to
improve the organizational performance of government agencies. This model
leverages blockchain technology to improve transparency, security, and
efficiency in government processes.
In her interview titled
"Public procurement digitalisation: a step forward or two steps
back?", Halonen (2019) argues that considering the current trends in digital
procurement, it is necessary to not only ponder the opportunities that these
technologies bring, but also the problems that arise from increased
transparency and the consequences of this procurement management approach.
Tayler, Langburd, and Wright (2018) highlight the existence
of a digital divide between countries worldwide, particularly in the field of
government procurement. While some countries lack even a portal to publish
information on government contracts, others, such as OECD countries, are
utilizing artificial intelligence and blockchain. For countries adopting these
new technologies, their government procurement processes become more efficient,
and they experience increased business opportunities. This growing divide
underscores the fact that other countries still have to catch up on what they
have missed.
The article "Digital Economy and Technology Development" written
by Zagloel et al. (2021) highlights
the importance of innovative digital technologies in balancing economic
advancement and environmental regeneration. Furthermore, the article emphasizes
that innovation diffusion processes are causing structural changes in different
economic sectors and reshaping economic systems.
Another relevant study by Rytova
et al. (2020) assessed the maturity level of the digital
government of Saint Petersburg. The authors examined various aspects of the
city's digitalization efforts. The findings of this study can provide valuable
insights into the challenges and opportunities associated with digital
government initiatives.
In recent years, several studies have focused on the
development and implementation of digital tools, particularly in government
procurement analysis. These studies highlight the significance of digital tools
in government procurement analysis and emphasize the need for continuous
research and development in this field. The integration of digital technologies
has the potential to revolutionize procurement processes, enabling greater
transparency, efficiency, and effectiveness. However, it is essential to
address the digital divide between countries and ensure that all governments
have access to the necessary tools and resources to leverage the benefits of
digitalization in their procurement practices.
In the article "Development of digitalization in
the procurement contract system" by Holkina
and Shvets (2019), the authors highlight the significance of digital
transformation and the adoption of modern digital technologies in the
procurement system. These advancements enable the market to expand beyond
geographical boundaries, attract more procurement participants, promote healthy
competition, and uphold the principles of openness and transparency in sharing
information about the procurement contract system.
Degtev (2020) explores the topic of digitalization and procurement
management in his study titled "Digital Transformation of Moscow's
Procurement Sphere". In addition, Degtev,
Gladilina, and Labutina (2020) analyze the impact of digitalization on management, emphasizing the need
for legislative changes and updated management tools.
In the works of Prokhorov
and Samoilov (2019), an analysis of the positive effects on competition
in the public procurement market when implementing digital tools is presented.
The authors also argue that the development of competition should be based on
stimulating public control, adjusting antitrust regulation, developing
innovations, and purchasing targeted effects. Mainly, scientific publications
focus on theoretical and normative-legal issues. In addition, the issue of
digitalization of government procurement is addressed in the normative
documents of governments of various countries and international organizations.
According to the European Commission's Procurement
Strategy (European Commission, 2017), the
digitization of procurement can enhance transparency in the internal market,
providing businesses with the opportunity to learn about business prospects,
facilitating access to public tenders and disseminating information about the
awarding of government contracts.
In accordance with the EU public procurement
directives, electronic submission of tender applications became mandatory in
October 2018. Various accompanying regulations and standards have been developed
for this purpose, such as eCertis, the European Single Procurement Document
(ESPD), the European Standard for eInvoicing, and tools to facilitate digital
transformation of procurement at different levels in accordance with Directive 2014/24/EU of the European
Parliament and Council (European Parliament
and Council , 2014).
Undoubtedly, the digitization of public procurement
requires continuous identification and analysis of problem areas in the digital
transformation of these processes. This includes issues related to the
compatibility of software solutions such as eIdentity, eSignature, eDelivery,
and eInvoicing, as well as the establishment of corresponding policies, skills,
and collaboration among stakeholders necessary for the effective implementation
of reforms.
Overall, the literature review on using digital tools
in government procurement analysis provides a comprehensive understanding of
the current state of research and highlights the importance of further
investigation into this area. However, specific research on the utilization of
digital technologies in the analysis of public procurement for detecting
suspicious transactions is not readily available in the public domain.
The contribution and novelty of this article lie in
several aspects. Firstly, it provides a comprehensive analysis of government
procurement data using a wide range of control indicators. By considering 140
control indicators, the authors were able to classify risk groups and provide
detailed characteristics for each group, including information on control
indicators, economic activities, customer levels, and types of trades
associated with each risk group. This level of granularity allows for a more targeted
and efficient detection of suspicious transactions.
Secondly, the article conducts an analysis of the
number of submitted bids, initial maximum contract price, and contract price
reduction for each risk group. This analysis provides insights into the
patterns and trends associated with potentially suspicious procurements,
further enhancing the ability to detect and prevent corrupt practices.
Furthermore, the article goes beyond just data
analysis and adds value by conducting a correlation-regression analysis to
identify the key factors influencing the increase in control indicators
signaling potential violations in procurement procedures. This analysis helps
to identify the underlying causes and drivers of suspicious transactions,
enabling policymakers
and practitioners to address these issues more effectively.
In this study, the authors focused on control
indicators that can be used to identify potentially suspicious procurements.
These control indicators were selected based on a thorough analysis of existing
scientific literature and the legislative framework in the field of government
procurement. The authors also considered information technology and analytical
tools commonly used in this field.
To collect the necessary data for analysis, the
authors selected procurement data from the past two years in the financial
services sector for government customers in the city of St. Petersburg, Russia.
This specific sector and location were chosen to provide a focused and relevant
dataset for the study.
Given the vast amount of information, it was not
feasible to explore all industries. Therefore, one industry was chosen - the
financial services sector. The rationale for selecting this industry is
outlined in the following section of the study, but the decision was also
influenced by the authors' research interests.
The study was conducted in accordance with the main
Federal laws in Russia regarding public procurement - 44-FZ (which regulates
the procurement process for goods, works, and services to meet the needs of the
government and municipalities, including contract formation and execution) and
223-FZ (which regulates procurement by specific types of legal entities).
The time period from 2021 to 2022 was chosen as a
temporary constraint to reflect current statistics (the year 2020 was excluded
due to numerous temporary legislative changes during the COVID-19 period).
Procurement methods were not restricted.
However, the scope of the examined customers was
limited to the delivery location - the city of St. Petersburg. This city was
selected because the study was conducted based on the Department for Combating
Cartels of the Federal Antimonopoly Service in St. Petersburg.
An analysis of the procurement data was conducted
using 140 control indicators. These indicators were carefully chosen to cover a
wide range of potential risk factors and suspicious activities. They included
factors such as the number of submitted bids, the initial maximum contract
price, and contract price reduction.
Based on the analysis of the procurement data, the
authors compiled a classification of risk groups and provided comprehensive
characteristics for each group. This included information on which control
indicators, types of economic activities, levels of customers, and types of
trades were characteristic for each risk group. This classification and
characterization of risk groups can help government agencies and professionals
better understand and target their efforts in identifying suspicious transactions.
In addition to the analysis of the procurement data, the
researchers conducted a correlation-regression analysis to identify the key
factor influencing the increase in the number of control indicators signaling
potential violations in procurement procedures. This analysis helps to identify
the underlying factors that contribute to suspicious transactions and provides
valuable insights for improving the detection and prevention of corruption in
government procurement.
Brief description of the procurement system. In Russia,
starting from January 1, 2019, all purchases, with rare exceptions, have
transitioned to electronic format. Now, any participant can take part in a
tender from anywhere in the world: to participate, they simply need to complete
a free registration on the EIS portal, select the desired procurement, and
participate through electronic trading on the electronic platform.
The procurement system of the Russian Federation, as
it exists today, has been in effect since 2014 and is regulated by the Federal
Laws "On the contract system in the sphere of procurement of goods, works,
services for ensuring state and municipal needs" of April 5, 2013, No.
44-FZ, and "On procurement of goods, works, services by certain types of
legal entities" of July 18, 2011, No. 223-FZ. It is worth noting that
since the introduction of the aforementioned laws, they have been transformed
and amended several times, taking into account the peculiarities of the Russian
economy, as well as the needs of society and businesses, with the aim of
increasing the efficiency of the use of the government contract system.
Federal Law (2013) 44-FZ
outlines the primary types of competitive procedures, namely auctions,
competitions, and requests for quotations. These procedures can now be
conducted electronically. Federal Law (2011)
223-FZ permits the use of alternative procurement methods as established by the
customer in their Procurement Regulations.
Let us examine the principal types of competitive
procedures. An auction entails selecting a winner based on the criterion of
price. The participant who offers the lowest cost for contract execution
emerges as the winner. Auctions, being straightforward, are the most commonly
employed procurement method. For requests for quotations, the price criterion
also determines the winner. The participant who offers the lowest price for
immediate execution secures the victory. However, restrictions on contract value
and annual purchasing volume apply to this procurement method. In the case of a
competition, non-price criteria assume a paramount role, and the victor is
determined by the participant who presents the best contract execution
conditions. Qualification of the participant also significantly influences the
outcome in competitive bidding.
Selecting a specific
sector of the national economy for research is essential
in order to meet the national priorities as stated in Presidential Decree No.
400, which was issued on July 2, 2021, and pertains to the Strategy for
National Security of the Russian Federation. One of these priorities is
ensuring the economic security of the state. In order to achieve this goal, it
is necessary to address tasks such as strengthening the financial system of the
Russian Federation, developing the national infrastructure of financial
markets, including payment infrastructure, reducing dependence on third
countries in this sphere, expanding the practice of settlements with foreign
partners in national currencies, reducing the outflow of financial assets
abroad, and combating illicit financial operations.
Therefore, the financial
services market is crucial to the country's economy. It is important to
effectively identify and prevent violations in procurement procedures for
financial services.
Choice of the industry of the national economy for
research. Ensuring the economic security of the state is one
of the national priorities of the Russian Federation, according to the Decree
of the President of the Russian Federation of July 2, 2021, No. 400
"On the Strategy of National Security of the Russian Federation". One
of these priorities is ensuring the economic security of the state. In order to
achieve this goal, it is necessary to address tasks such as strengthening the
financial system of the Russian Federation, developing the national
infrastructure of financial markets, including payment infrastructure, reducing
dependence on third countries in this sphere, expanding the practice of
settlements with foreign partners in national currencies, reducing the outflow
of financial assets abroad, and combating illicit financial operations.
Therefore, the market for financial services is one of
the most important components of the country's economy. It is crucial to ensure
effective identification and prevention of violations in the procurement
procedures for financial services.
Information technology for the search, analytics, and
monitoring of government procurement. Information
systems in the field of government procurement, which are widely used by both
clients and suppliers, include software products such as Marker-Interfax,
Tenderplan, Bicotender, Konutr.Zakupki, SberA (Sberbank-AST) analytical portal,
and others. These information systems empower participants in government
procurement to efficiently search, analyze, and monitor nearly all existing
tender procedures. They ensure transparency and openness in the implementation
of government procurement.
The Marker-Interfax program is a market procurement
monitoring and analysis system that consolidates and analyses data on
procurement, clients, and suppliers from over 300 trading platforms.
The Tenderplan provides the opportunity to search for
tenders and perform comprehensive analysis of the selected sample, the
customer, the supplier, and the relationships between them. It also supports
collaborative work on the platform and is a cloud-based service.
On the other hand, Bicotender offers the ability to
search for targeted tenders and includes a module for industry analysis,
suppliers, and customers. It provides an API service for integrating the tender
system with CRM systems.
Konutr.Zakupki is an information system specialized in
searching for government and commercial procurement on various platforms. It
allows for the analysis of customers and suppliers.
The SberA analytical portal provides a wide range of
analytical information on government procurements, including statistics, market
analysis, and decision-making support.
These information systems
greatly enhance the work of customers and suppliers involved in government
procurement by enabling them to easily access information and analyze market
conditions. Additionally, they promote transparency and openness in government
procurement and enhance market competitiveness.
In the context of this study, the search and analysis
of information on government procurement were conducted using an automated
program called Marker, which was developed by Interfax. The results of the
analytical study are presented below.
Correlation-regression analysis. With the
aim of identifying factors that influence the increase in the number of control
indicators signaling potential violations in the provision of financial
services procurement procedures, excluding insurance and pension provision
services (OKPD2 - 64), a correlation regression analysis was conducted. The
independent variables selected were the initial maximum contract price (NMC),
type of tender, number of submitted bids, and the customer's field of activity.
Multiple regression models were constructed using the Stata software program.
The initial
hypothesis suggested that the increase in the contract value (NMC) has the
greatest influence on the increase in the number of "suspicion"
indicators in procurement. However, the obtained multiple linear regression
model does not yield satisfactory results. Four selected variables in the model
are statistically significant, but they only explain 13% of the variance in the
number of control indicators. The residual plot, Cook's and Vaisberg's tests,
as well as the White test indicate the heteroscedasticity of the residuals in
the model. Partial residual plots for linearity show a non-linear relationship
between the number of control indicators and the factors under investigation.
Furthermore, the model specification test suggests there are specification
errors, hence rendering the coefficient estimates for the variables
inconsistent. The results of the Shapiro-Wilk and Shapiro-Francia tests
indicate that the residual distribution does not follow a normal distribution.
Consequently,
the linear regression model does not sufficiently describe the relationship
between the number of control indicators and the examined indicators. Building
other multiple regression models, namely logarithmic, linear-logarithmic, and
logarithmic-linear, also yield unsatisfactory results.
Classification analysis. Taking into
account a large number of indicators of "suspiciousness" in
procurement procedures, all analyzed procurements were divided into three risk
groups. The first group included 692 procurements with one to three indicators.
The second group consisted of 158 procurements with four to seven indicators.
The third and most risky group consisted of 14 procurements with eight to ten
indicators.
Next, the
indicators characteristic of each risk group were identified. In all three
groups, the majority of procurement procedures contained indicators such as
"Request for clarification of provisions in the documentation" and
"Some documents are unavailable for search". For the least risky
procurements, which made up the first group, a distinguishing feature was a
high proportion, specifically 10.65% of the total number of procurements in
this group, containing the control indicator "Participant's reduction
exceeds 25%". Additionally, in the first and second groups, there is a
significant proportion of procedures, amounting to 10.65% and 12.57%,
respectively, that have the indicator "Contract savings exceed 25%".
Analysis of the control indicators for the highest-risk procurement procedures
has revealed that in this group of procurements, there is a high proportion of
procedures (11.48% of the total number of third group procurements) that have
the indicator "Contract price increased under the Federal
Law (2011) 44 (Code of Administrative Offenses)", whereas in other
groups, their percentage is significantly lower. For this group, the indicators
"Penalties, fines for the contract with the supplier under the Federal Law (2011) 44" (9.84% of the total
number of third group procurements), "Partial termination by mutual
agreement under the Federal Law (2011)
44" (9.02% of the total number of third group procurements), and
"Existence of non-participating applications " (8.02% of the total
number of third group procurements) are characteristic. Additionally, only in
the third-risk group are there procurements that contain the control indicator
"Participant included in the Register of Unreliable Suppliers (evasion of
contract conclusion)" (4.92% of the total number of third group
procurements).
The research analyzed the economic activities of customers
based on the OKVED classification, which corresponds to each risk group. It was
found that the procurement activities of customers mainly involved in
healthcare services posed the highest risk. This OKVED category was one of the
most frequent across all risk groups, accounting for 27.89% of the total
customers in the first group, 64.56% in the second group, and 14.29% in the
third group. Additionally, it is worth noting that within the first risk group,
39.31% of customers were involved in activities corresponding to the OKVED
category "Provision of electrical energy, gas, steam, and air
conditioning". Furthermore, in the third risk group, which consisted of
customers engaging in the riskiest procurements, a majority of them were
involved in activities corresponding to the OKVED category "Land and
pipeline transportation activities" (85.71% of the total customers in the
third group).
At the next stage, an analysis of the levels of organizations
that act as customers was conducted within each risk group. As a result, it was
found that such organizations are most commonly affiliated with the level of a
regional of the Russian Federation (54.77% of the total number of organizations
in the first risk group, 81.29% of the total number of organizations in the
second group, and 100% of the organizations in the third group). Procurements
with indicators are less frequently carried out by customers belonging to the
municipal level. Further analysis of the types of trading in the studied
procurement procedures allowed us to conclude that the most common method for
these procurements was through electronic auctions (70.81% of the total number
of procurements in the first risk group, 96.20% of the total number of
procurements in the second group, and 100% of the procurements in the third
group). At the same time, 10.84% of the procurements in the least risky first
group were conducted with a sole supplier.
Furthermore, the study also investigated how the contract
price decreased during procurement procedures in each risk group. For further
analysis, data preprocessing was conducted, and instances with missing values
for terms were removed. As a result, procurement in the first risk group mainly
had a reduction ranging from 6% to 25% (38.77% of the total number of
procurement in the first group), while the majority of procurement in the
second group had a decrease ranging from 25% to 50% (52.87% of the total number
of procurement in the second group). On the other hand, the overwhelming
majority of procurement in the third group, which had the highest number of
control indicators, were carried out with a reduction of only up to 2% (64.29%
of the total number of procurement in the third group). The results are
presented in Table 1.
Table 1 Analysis results of procurement risk groups based on price reduction (Compiled by the author)
|
Price reduction |
I group |
II group |
III group | |||
|
number of procurement |
% |
number of procurement |
% |
number of procurement |
% | |
|
up to 2% |
188 |
31.28% |
19 |
12.10% |
9 |
64.29% |
|
up to 6% |
51 |
8.49% |
15 |
9.55% |
3 |
21.43% |
|
up to 25% |
233 |
38.77% |
31 |
19.75% |
2 |
14.29% |
|
up to 50% |
102 |
16.97% |
83 |
52.87% |
0 |
0.00% |
|
up to 75% |
20 |
3.33% |
7 |
4.46% |
0 |
0.00% |
|
more than 75% |
7 |
1.16% |
2 |
1.27% |
0 |
0.00% |
While
analyzing the number of applications submitted for the procurement procedure in
each risk category, it was observed that for the least risky procurements, the
range of applications varied from 0 to 2 (constituting 59.39% of the total
number of applications in the first group). In the second risk category, the
most common range of applications is 3 to 5 (56.96% of the total number of
applications in the second group), and in the third risk category, it is 6 to
16 applications (42.86% of the total number of applications in the third
group). The results are presented in Table 2.
Table 2 Analysis results of procurement risk groups based on the number of submitted applications (Compiled by the author)
|
Number of applications submitted |
I group |
II group |
III group | |||
|
number of procurement |
% |
number of procurement |
% |
number of procurement |
% | |
|
0 |
98 |
14.16% |
- |
- |
- |
- |
|
1 |
110 |
15.90% |
18 |
11.39% |
2 |
14.29% |
|
2 |
203 |
29.34% |
50 |
31.65% |
2 |
14.29% |
|
3 |
142 |
20.52% |
68 |
43.04% |
- |
- |
|
4 |
63 |
9.10% |
10 |
6.33% |
- |
- |
|
5 |
52 |
7.51% |
12 |
7.59% |
4 |
28.57% |
|
6 |
7 |
1.01% |
- |
- |
- |
- |
|
7 |
- |
- |
- |
- |
6 |
42.86% |
|
16 |
17 |
2.46% |
- |
- |
- |
- |
Furthermore,
the analysis of the NMC in each risk category showed that the majority of
procurements in the first and second groups have an NMC ranging from 1 million
to 100 million roubles (49.86% and 44.94% of the total number of procurements
in the first and second groups, respectively) (see Table 3).
In addition, a significant proportion of procurements in these groups have an NMC up to 500 thousand roubles (21.39% and 24.86% of the total number of procurements in the first and second groups, respectively). Meanwhile, in the third risk category, procurements with an NMC ranging from 100 million to 500 million roubles predominate (42.86% of the total number of procurements in the third group), as well as a significant proportion of procurements with an NMC ranging from 500 million to 1 billion roubles (28.57% of the total number of procurements in the third group).
Table 3 Analysis results of procurement risk groups based on the NMC (Compiled by the author)
|
NMC, rub. |
I group |
II group |
III group | |||
|
number of procurement |
% |
number of procurement |
% |
number of procurement |
% | |
|
up to 500 K |
148 |
21.39% |
39 |
24.68% |
0 |
- |
|
up to 1 M |
53 |
7.66% |
21 |
13.29% |
2 |
14.29% |
|
up to 100 M |
345 |
49.86% |
71 |
44.94% |
0 |
- |
|
up to 500 M |
74 |
10.69% |
13 |
8.23% |
6 |
42.86% |
|
up to 1 B |
37 |
5.35% |
4 |
2.53% |
4 |
28.57% |
|
more than 1 B |
35 |
5.06% |
10 |
6.33% |
2 |
14.29% |
Control
authorities in the field of public procurement should pay attention to
procurement procedures that exhibit the following characteristics:
- The presence of control indicators such as "request
for clarification of provisions in the documentation", "Bidder
(winner) offers a reduction exceeding 25%" / "Contract savings exceed
25%", "Some documents are not searchable", "Partial
termination by mutual agreement under Federal Law
(2011) 44-FZ".
- The customer is engaged in activities corresponding to the
OKVED codes "Healthcare activities", "Education",
"Land and pipeline transportation activities", "Electricity,
gas, steam and air conditioning supply".
- The procurement procedure was conducted through electronic
auctions or with a single supplier.
- The procurement process involved 2-3 applications.
In conclusion, this study has demonstrated the relevance and
promise of utilizing digital technologies and tools for analyzing government
procurement through control indicators. The findings indicate that digitization
positively impacts the transparency of procurement procedures and enables more
effective analysis and monitoring. The investigation has provided valuable
insights into potentially suspicious government procurement in the financial
services sector of Saint Petersburg, which can be utilized by regulatory
bodies. While there are some limitations, such as the need to improve the
classification of risk groups and data processing, this study presents
significant opportunities for future research. The approach can be applied to
other regions and sectors, allowing for a more comprehensive understanding of
how digital technologies can enhance government procurement analysis.
Furthermore, the implementation of a machine-learning model for detecting
suspicious procurements and the establishment of an automated system will
further enhance transparency and efficiency in government procurement,
mitigating the risk of corrupt practices. Ongoing research in this field will
contribute to the development and implementation of innovative technologies and
tools that effectively combat potential violations and promote transparency in
government procurement.
The research was
financed as part of the project "Development of a methodology for
instrumental base formation for analysis and modeling of the spatial
socio-economic development of systems based on internal reserves in the context
of digitalization" (FSEG-2023-0008).
Berawi,
M.A., Sari, M., Addiani, F.A.F., Madyaningrum, N. 2021. Developing
Blockchain-based Data Storage System Model to Improve Government Agencies’
Organizational Performance. International
Journal of Technology. Volume 12(5), pp. 1038–1047
Degtev, G.V.,
2020. Digital Transformation of Moscow's Procurement
Sphere. Innovation and Investment, Volume 2, pp. 226–229
Degtev, G.V.,
Gladilina I.P., Labutina N.N., 2020. Digitalization and Procurement Management in the
Process of Achieving the Social and Economic Effects Of Procurement. Innovation
and Investment, Volume 5, pp. 124–127
European
Commission, 2017. Communication from the Commission to the European
Parliament, the Council, the European Economic and Social Committee and the
Committee of the Regions. Available Online at https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM%3A2017%3A572%
3AFIN#document1, Accessed on September 10, 2023
European
Parliament and Council, 2014. Directive 2014/24/Eu of the European Parliament
and Council. Official Journal of the European Union, pp. 65-242. Available Online at https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32014L0024&from=EN, Accessed on September 10, 2023
Federal
Law, 2011. On the Procurement of Goods, Works, and Services by Certain Types of
Legal Entities" dated 18.07.2011 No. 223-FZ (latest version). Consultant
Plus. Available
Online at: https://www.consultant.ru/document/cons_doc_LAW_116964/, Accessed on September 10, 2023
Federal
Law, 2013. On the Contract System in the Procurement of Goods, Works, and
Services for State and Municipal Needs, dated 05.04.2013 No. 44-FZ (latest
version). Consultant Plus. Available Online at: https://www.consultant.ru/document/cons_doc_ LAW_144624/, Accessed on September 10, 2023
Halonen, K.-M., 2019. Public Procurement Digitalisation: A Step Forward
Or Two Steps Back? Available Online at: https://www.howtocrackanut.com/blog/2019/9/3/public-procurement-digit
alisation-kirsi-halonen, Accessed on September 10, 2023
Holkina
N.A., Shvets A.V., 2019. Development of Digitalization of the Contract System
in the Field of Procurement. Financial Markets and Banks, Volume 2, pp. 60–62
Prokhorov
Yu.N, Samoilov M.G., 2019. Digital Transformation and Its Impact on the
Development of Competition in Procurement in the Transition to a Digital
Economy. Financial Markets and Banks, Volume 2, pp. 57–59
Rytova,
E., Verevka, T., Gutman, S., Kuznetsov, S., 2020. Assessing the Maturity Level
of the Digital Government of Saint Petersburg. International Journal of Technology. Volume 11(6), pp. 1081–1090
Tayler, Y., Langburd, N.,
Wright, 2018. The future of public procurement in the era of digitalization. World Bank. Governance for Development. Available Online at: https://blogs.worldbank.org/governance/future-public-procurement-era-digitalization, Accessed on September 10, 2023
Zagloel,
T.Y.M., Surjandari, I., Berawi, M.A., Asvial, M., Harwahyu, R., Suryanegara,
M., Setiawan, E.A., Suwartha, N., Maknun, I.J., 2021. Digital Economy and
Technology Development. International
Journal of Technology. Volume 12(7), pp. 1323–1327