2.1 Public Procurement of Innovation

Public Procurement of Innovation (PPI) is a policy instrument aimed at supporting innovation and enabling SMEs to take part in public procurement [1]. A broader definition refers to public procurement of innovative products, services and processes with the aim to improve offered public services or to optimize socio-economic challenges, that optionally can contain also R&D phase in case it did not take place before [1].

In 2008, review studies showed that usually the PPI process does not include procurement of emerging innovative technology in its early stages, but rather adaptations or improvements of already existing solutions, or improvements of processes and non-technological innovations [19], [48]. Later in 2015, an extensive review of empirical research related to PPI was performed, with the results showing that majority of studies consists of qualitative case studies where the innovation was the result of public procurement. Reviewed cases can be divided into two categories, the first one containing analysis of general policies supporting innovative environment and the second one containing innovations resulting from the public procurement [7], [17].

The core of successful PPI is the so called „intelligent customer that is able to demand, source and identify potential new solutions, moreover is able to specify and manage contracts [25]. PPI is acting as the first buyer of commercial innovations coming to the market [28], [29], [41]. Although, innovation in public sector do not occur every year so the public agencies do not participate in PPI on a regular basis, but only at the end on the current innovation life cycle end [33].

2.2 PPI Principles and Process

Effective pubic procurement should use electronic means and be organized in an open and transparent manner. Based on the previous studies [33], [45] several conditions required for successful public procurement were identified:

1. Expertise in PPI laws and procedures

2. Strategic management of PPI

- To be able to balance objectives of procurement and innovation;

- To take into consideration future needs;

3. Market interaction

- To be able to identify the innovation availability on the market;

- To have sufficient technical knowledge being able to decide what to procure;

- To have sufficient understanding of procurement practices from the customers;

- To avoid routine allocation of resources;

- To have sufficient knowledge of supplier’s competences and to be able to attract the demand side;

4. Risk management

- To be aware and be able to manage technical risks connected to lower than expected performance of the procured products;

5. Coordination and communication

- To be able to coordinate suppliers by coordinated procurement where more than one supplier is involved;

- To have political support;

- To be supported by other institutional actors;

- cross-organizational and cross-country coordination and communication to be able to create demand bundles;

6. Capacity building

The typical PPI process consist of following steps [1], [8], [11]:

1. Identify a challenge - Identification of a main challenge by procuring institution focused on a lack of satisfaction of a human need or an unsolved societal problem. Definition of the need.

2. Definition of functional specification of the proposed solution - Careful translation of the identified challenge into functional specification avoiding technical over specification.

3. Tendering process:

- Opening of the bidding process through a tender.

- Translation of the functional specification into technical specifications by potential suppliers.

- Submission of formal bids by potential suppliers.

- Assessment of tenders and awarding of contracts.

4. Delivery process:

- Product development.

- Production of the product.

- Final delivery to the procurer.

2.3 Pre-Commercial Procurement

The PCP concept was officially introduced by the European Commission in 2006. One of the main triggers of developing a PCP concept was the idea to support development of innovations through public sector and simultaneously to improve the quality and efficiency of public services [12]. PCP was designed to overcome a gap between scientific knowledge and market through creating a public demand [33]. Currently the PCP process should not be limited only to public procurers but should be applied regardless the procurer type.

According to [15], [30], three different PCP process variations can be defined:

• Procurement is undertaken by the public subject for the needs of the same subject.

• Procurement is undertaken by a public agent or partnership for another public user (e.g. the UK SBRI model);

• Procurement is undertaken by a public subject or agent for a set of private clients where there a clear public benefit is present (assimilated to what is called a catalytic procurement).

However, we do not limit the variations of PCP to the three described above. Considering the PCP as a universally applicable process to any procurement of R&D activities, the procurer is not limited only to public sector. Therefore, we propose a fourth PCP variation, where the R&D sourcing subject or R&D procurer is a private subject. PCP process in this case can be applied exactly in the same way as in case of procurer from public sector.

The results or success of PCP is difficult to predict and therefore produces uncertainties and risks. PCP process is applying three aspects to handles these uncertainties [43], [46]:

• Risks and benefits are shared between suppliers and developers.

• Competitive development in phases/iterations is securing the best solution that market is able to offer.

• Separation of development of product and the final procurement of the developed product itself.

These aspects cause the private supplier to share the risk of the R&D phase and the public entity is not obliged to purchase the developed product [1], [23], [36]. However, several authors considered PCP as a type of PPI that consists only of R&D phase and not of procuring a certain product itself [12]. In this point of view, the process differences should not be that significant.

It is possible that PCP would not generate a successful output in the form of a prototype or even usable R&D result and therefore PCP might be complemented by PPI, however PCP cannot be considered just as a particular type of PPI [13]. On the other hand, according to [13], [20] can be the PCP process segmented into three stages:

• Solution identification and definition phase: identification and analysis of the procurer´s need, design generation, evaluation of proposals and best offers selection.

• Prototyping phase: development and adjustments of the suggested product or service prototype.

• Testing phase: solution validation through field tests. During this phase a limited number of prototyped products is produced and in field tested for commercialization of the solution.

Although the public demand is considered as a potential way of fostering innovation, majority of the innovation commercialized are from the private sector [27], [35]. Therefore, PCP can’t be seen just as a policy tool for innovation support but has to be considered as a framework applicable to any type of procurer therefore it has to be defined in a more general approach with the focus on crucial aspects that hinder the success in this process. PCP as the driver for innovation should be applicable to the whole market to strengthen the innovation uptake effect [5].

European Commission conducted a survey during 2010 with the objective to determine the implementation of PCP status, to identify possible difficulties encountered by Member States and to identify further steps that could be taken to facilitate wider implementation of PCP across Europe. Results showed that 14 out of 21 EU countries (Slovakia and Czech Republic included) did perform a few PCP like projects but are missing any conceptual framework. Altogether 11 countries developed a framework for PCP process and 3 countries declared a fully applicable and utilized framework that is already applied in real PCP projects. The lead in PCP framework application belongs to the Nordic countries. In these countries both approaches bottom up and top down are used.

Most of the countries started to build the PCP frameworks from scratch to design national innovation procurement support schemes. Lowering the barriers for PCP application rather than creating a strict legal framework seems to be preferred by most countries. The survey concluded that none of the surveyed countries had a national legal framework that would prevent the roll-out of PCP projects. Regarding the Intellectual Property Rights (IPR) in PCP process their ownership should be left to suppliers allowing them to exploit their developments across markets. The IPR conditions in PCP process significantly vary among EU countries.

3.1 Instrument Development

To reflect the real life situation when considering a PCP process and to assure that all the relevant relations are taken into consideration, a conceptual model based authors experiences and survey results was developed, and the identified relations were statistically tested using empirical data.

In general, the methodology of the research is based on two main approaches:

• Survey about PCP utilization current status, experiences and expectations with the objective to identify crucial problems that have to be solved and

• Empirical data analysis based on data from real procurement process to analyze relations and dependences between basic parameters within electronic procurement process esp. from negotiation phase.

On this base, research objectives of this paper are focused on following research tasks:

• To identify current behavior in PCP processes in Slovakia and Czech Republic, countries with high innovation adoption level for procurement processes

• To identify which phases of procurement process are significant for PCP success and which attributes, related to this phases, are crucial for successful PCP and procurement of innovation

• To identify differences between PCP and traditional procurement determinants

3.2 Data

To fulfil these objectives, a combination of two main data sources was used to prepare and assess the conceptual model components relationship.

First data source was from a qualitative study carried out during the year 2018 on the sample of 19 experienced procurers from Slovak and Czech public and private sector using structured telephone interview approach. Survey overview is provided in Table 1. This survey was focused on identifying the process of PPI and PCP adopted in the Slovak and Czech conditions, to gather information about the problems these companies had to solve in the process, and to gather their expectation for PCP ecosystem improvement.

The structure of the interviews can be summarized into these categories:

• Identification of the need for which an innovative solution should be procured/developed.

• Definition and design description of the potential solution or its procurement

• Market analysis for existing solution identification or supplier identification

• Market assessment and procurement of the innovation / R&D

• Contract execution and monitoring pase

• Implications on improvements of analyzed process

Second data source was a database obtained as an export from electronic reverse auction (eRA) system ProEbiz provided by NAR Marketing, one of the biggest reverse auction SW provider in Central Europe. The data was anonymized and used to study various variables and their mutual relationships and the relation to procurement success. PCP related records and a representative sample of traditional procurement records were extracted and anonymized from the complete sample. The export of the data was carried out with the intention to identify procurement of innovations records and to compare various rules and behavior models between this type of procurement and regular procurement of goods and services. To identify the innovative procurement an approach of key word search based on [24] was used. The data represent auctions carried out in a time window from 2009-2019 roughly counting more than 60 000 auctions.

The sample description of extracted data used for further research is described in Table 2.

We have distinguished two types of procurement types regarding the number of items in one tender to a bulk and separated one. The nature of some PCP procedures and according to survey recommendations the separated type is when each item was negotiated separately and bulk type where all items are negotiated in a bulk as it would be a single item. To analyze the conceptual model relations several variables from the auctions database were used. Table 3 captures the statistics from nominal variables used in the model.

With the relatively low number of experts with PCP experience, results from the interviews were processed into a set of formalized problems or experiences raised by the respondents, with the corresponding frequencies. The most crucial statements perceived by the respondents were ranked on this basis.

As for the data analysis, we used the structural equation modelling (SEM) approach to identify conditional dependences between attributes according to the conceptual model enhanced by descriptive statistics and data mining techniques, esp. regression trees.

3.3 Conceptual Model Development

To describe and to analyze relations between different parameters as factors affecting the success of the procurement process a conceptual model shown in Figure 1 had to be developed. Its components were derived from both data sources as well as crucial factors of innovation procurement success and standard auction parameters used mainly in experimental research studies.

This conceptual scheme contains two different types of attributes:

• Attributes describing the procurement phase (Pricing, Negotiation, Sourcing, Commercialization) with relations to

• Attributes as core elements or exploratory attributes of particular phases determining its efficiency (e.g. for negotiation phase, different type of negotiation is possible to adopt as sealed bid auction vs English auction, number of items used for the negotiation etc.)

3.4 Description of the Conceptual Model Elements and Statistical Testing

Procurement success was decided as an output parameter which was determined by two attributes, savings (as attribute presenting the success of negotiation) and cooperation (as crucial factor for the success of RTD realization, adoption and satisfaction).

Requirements analysis and Pricing as the first phase related to needs, technical parameters and expected value or impact identification. The decision about innovative solution required through its complexity determines volume of procurement and negotiation type. Negotiation or auction type was decided as related to pricing because of two reasons. The first is related to public procurement legal restriction for particular volume what is a common approach in most of countries. The second is related to the existence of impact modelling, e.g. CBA analysis as in some cases procurers judge the pricing only on market competition or offer.

Procurement type is a binomial value that differentiates between government and private procurer. In the conceptual model a relation between the type of procurer and the value of savings is estimated to be positively correlated in favor of private procurer. We use generally perceived opinions that public procurement used to be often inefficient and manipulated.

Negotiation phase of procurement life cycle is determined by several options for contractual condition negotiation and negotiation types applied for achieving a more competitive environment. This phase is considered as crucial in general procurement life cycle and is responsible for contractual condition negotiation in acceptable or best offer on the market and can be sensitive for successful PCP realization. Evaluation of this relation is tested simply by frequency of positive responses ranked from our expert survey. As a core phase can be set up by esp. two basic negotiation parameters: types of auction for the possibility of having more transparent and dynamic pricing environment; and winner selection determination.

Sourcing as a crucial phase for identifying qualified and trusted suppliers or researchers and the identification whether respective innovative solution already exists on the market.

Target price is the estimated value of the purchasing, or the initial bid where no estimated value was available. Volume: the nominal value of the auction, in our case the Winning bid variable from the data set was used. We have merged these two dependent attributes into one only for presenting the relation of volume as input but also output and the expected price as an output parameter of the pricing phase. Target price and its accuracy is logically determined by the pricing method. Inaccuracy of the pricing method can lead to inefficient negotiation especially with low number of potential suppliers. Volume of auction: where generally, higher value is often claimed as motivation for suppliers to attend auction event, although some economic experts argue that according to the marginal utility theory the savings can be lower by increasing auction value [2], [47]. Target price can be also determined through the number of items procured.

Number of items: total number of items negotiated within a single auction negotiation. Although, the number of items doesn’t automatically mean a higher total value, psychologically a higher number of items can be the motivation to compete more in the auction and as such have an impact on savings and indirectly on target price and volume [47].

Auction type: the main negotiation model of the auction, a complex term including sealed bid auction and English type of auction as a dynamic pricing approach. Type of auction can have an impact on auction prices and savings. In this study, we compared sealed bid with dynamic online auctions and assume and indirect impact on saving. Dynamic auction is considered as a more transparent tool, that can attract more participants and thus provide higher savings.

Number of participants: total number of suppliers taking part in the auction. Number of participants is suggested as significant positive factor of auction. Number of participants is derived from the sourcing phase. Higher number of participants should increase the competition and number of price changes that could lead to higher savings.

Number of new participants: means the number of participants that are new to the procurer and have no historical experiences or customer relations. Just as the number of participants, the number of new participants is directly affected by the sourcing carried out before procurement. The number of new participants should help to identify the differences between innovation procurement and regular procurement

Transparency of negotiation: total level of transparency of the given auction process derived from the individual transparency settings. Transparency of negotiation is determined by dynamic auctions settings where different parameters of negotiation can be revealed, e.g. names of competitors, ranking, best price, criteria etc. As mentioned, transparency level can attract more participants and increase competition intensity resulting in auction prolongations or in higher number of price changes during negotiations and thus to higher possible savings. Transparency is derived from negotiation indirectly through the auction type, and can have a direct positively correlated impact on savings.

Number of changes: means total number of offers - price changes during the negotiation phase. The number of price changes is a signal of intensive competitive behavior often resulting in prolongations of e-auctions and finally in better prices and higher savings. The number of changes is affected by the number of participants where a positive relation is expected, and also by target price where similar relation should occur. The sourcing process and negotiation process can affect the number of changes indirectly.

Winner selection method is the type and number of evaluation criteria in a negotiation. In this research, two main types are considered: Price as the only criteria and a multi-criteria setting where other parameters like quality, service conditions, certificates validation or time managerial issues can be set in addition to price. All these setting are defined in the negotiation phase and directly affect the savings and contractual conditions. In the case of price criteria, higher savings are expected.

Final savings: % savings achieved by the auction process calculated in relation the Comparative price or initial offer where comparative price not available.

Cooperation is the crucial factor within the whole life cycle of innovation development and is determined by the communication and co-creation for high satisfaction with innovation adoption.

The relation testing was carried out using the data set gathered from eProcurement SW provider. The sample was divided into two subgroups of innovation procurements and traditional ones. Following, we have tested the causalities between the attributes using a structural equation modelling approach with maximum likelihood estimation. The significant differences in variables of one procurements group against the other procurements group was tested. The direction and size of the change was derived from the mean differences.

To confirm a functional relationship also a decision tree approach was carried our using a chi-square tree with saving as the output variable. The calculations were carried out using R programming language.

4.1 Empirical Data Analysis

Based on the data set gathered from eProcurement SW provider we have analyzed the descriptive statistics of each related attributed, its differences in two subgroups (innovation procurement and traditional procurement records) and tested causalities between the attributes. For relation testing, structural equation modelling approach was selected due to a wide usage for conditional relationships testing and visualization of estimated path with dependence validation for each relation. As the estimation method, maximum likelihood estimation was applied.

To identify statistically significant relations among studied attributes, the SEM methodology estimated using Maximum Likelihood estimation was applied, taking into account all possible relations. Due to the conceptual model not anticipating any unobserved variables, these were not included in the model. The calculations were performed using the R project for statistical computing.

The SEM was applied on the whole sample with both subgroups, as these would be examined more deeply later.

Table 6 provides an information about results for SEM application where only significant causalities with p=.000 are presented with information about the causality direction.

Although in our assumptions the higher volume could be motivational for participants to provide higher savings, the result from SEM show an opposite causality. It seems, the logic behind this result lies more on the economics of scale so volume has negative relation to savings. On the other hand, the negative impact of volume on number of items shows the fragmentation of tenders in smaller financial volumes in procurement.

The strongest attribute with the highest impact on savings was identified as the number of price changes, which is logical esp. in dynamic pricing environments where competitive bidding decreases the price. This result is determined by higher number of English auction type procurements present in the sample.

Another logical causality was revealed between the number of participants and the number of price changes, which again can be caused by the dynamic pricing environment and higher activity of auction participants. Interesting issue is that SEM didn’t reveal any other determinants of number of price changes, despite the fact that it could be expected that also new participants can push on participants to be more active or number of items, which should affect the number of price changes in negotiation. To get a deeper insight, we have applied the regression tree analysis with CHAID algorithm for number of changes as an output. Results from the decision tree are presented in Figure 3 and reveal more conditional causalities with relatively high significance.

These results also show additional causalities with number of price changes as the main determinants of savings. Number of items shows a clear positive relation to number of price changes with additional information about procurement with one item where number of new participants seems to be very important to increasing the number of price changes. This information can be important in relation to the fragmentation of the contract and as support for the recommendations from survey, where dividing the procurement of innovation into smaller parts can have additional benefits.

Number of new participants is, according to SEM results, determined by number of participants and auction type. The most significant finding is, that more participants are in a sealed bid tender with lowest level of transparency then in a traditional English auction with higher transparency. This result would be suitable for validation in other countries and other cultures. It can be explained as ongoing fears from transparent environments or more speculative opportunities in sealed bid auctions related to the perception of unfair practices in Slovakia esp. in the public procurement environment.

The impact of auction type on transparency is deterministic, especially taking in to account the fact, that English auctions are generally used with higher functional possibilities of negotiation and attribute visibility than Request for Quotation (RFQ) models.

Generally, the PCP process is defined as relatively specific in most success cases and policy documents. A comparison of selected statistics is provided in Figure 4. We will test these differences within the conceptual model for separate model elements and for causal relations between attributes. The objective is to identify which settings, components of procurement phases are specific only for PCP process and vice versa. For the purpose of identification if differences between PCP and non PCP tenders exist within the proposed conceptual model, the Independent sample Mann Whitney U Test was performed with results presented in Table 7.

According to these tests, PCP procurement differs in distributions of some variables in relation to categories of Innovation type. In the following figures, the sample is first divided into two subgroups - innovative for PCP and standard for traditional procurement (common product and services). Then individual causal relationships are tested separately for both subgroups by dividing them according to their mean (using one variable) into below and above average subsamples, followed by testing if the means and distribution of the tested variable (different from the one used to divide the groups) differs for these subgroups. Means of the tested variable for both subgroups are then used to determine the direction of the relation, if the difference of means is statistically significant.

According to the Mann Whitney test, we see that the impact of procurement type on Volume is not statistically significant in the case of innovation procurement. Standard procuring records show significant effect of procurement type on volume. The same holds for the number of items vs volume. These results are presented in Figure 5. In relation to these results we can discuss what is the acceptable level of statistical significance. Small sample of innovation procurement is the main limitation of research, but in this case we can at least with lower validity of p = 0.139 say that it seems, the innovation procurement differs from traditional one as a higher number of items has a positive impact on volume, which is not present in traditional procurement.

The effect of auction type on the two participant attributes (results in Figure 6) doesn’t show statistical significance, which can be caused by small extreme values in above average groups.

The number of participants is an attribute with a relatively easy to understand background, as it can be assumed that in the PCP case, the potential suppliers with requested qualification, research and innovation competencies in respective fields of innovative solutions are not as common on the market as standard seller of common goods and services. One interesting question is, if new participants, which are considered as positive factor for increasing supply chain and procurement efficiency, are crucial also for PCP. According to distribution comparison and Mann Whitney U test presented in Figure 7, a difference exists in the impact of the number of participants on the number of new participants and on the number of price changes. Statistically significant and positive relation is presented only on the non PCP sample. The PCP sample show high p-values not revealing this impact. This result supports the thesis, that research and innovation suppliers are not so common not only through p-value of the test itself, but also through the differences in range of this attribute in the two groups. Number of price changes is a crucial factor for achieving significant savings in a competitive environment, which can be very helpful in the case of low skilled innovator (innovation procurer) who is not able to estimate the expected value and the cost acceptable to pay for the delivery. In this case, a high number of price changes represents competition which should help achieve real market prices. On the other hand, real market price doesn’t have to necessarily be acceptable in the case of lower expected benefit from innovation adoption. In the PCP group we see a significantly smaller range of number of price changes (see Figure 7) that can indicate low efficiency of sourcing processes.

The effect of volume on the number of items seems disputable. Some researchers and statistical philosophers argue that there cannot be a standard or conventional level of p-value. As value 0.1 is often acceptable in many sociological researches, 0.11 could be acceptable. In this case, we can assume that the difference in these relations between the two groups exists and shows positive impact of volume on number of items in the case of PCP, as seen in Figure 8. The distribution shows significant differences in PCP group between below and above average what can be the evidence supporting some recommendations, that PCP, esp. in higher volume, should be divided into smaller parts to reduce risks and within this research also increase savings.

Regarding volume, we have found that for the whole sample volume has negative impact on savings but differences in volume exist between the two groups, as seen in Figure 8. In this case we see that differences in relations were not confirmed. It seems, that the reason for this could be the small number of extreme values in volumes in the PCP records.

Number of price changes was identified as the most crucial factor in SEM analysis on the whole sample. In this test, we see in Figure 9, that both samples show a positive impact on savings.

4.2 Discussion

Development of a complex conceptual model with the corresponding relation testing for procurements where innovation is procured is according to our literature review rare and we were not able to find a relevant study that would deal with PCP or PPI in a similar way as in this paper. Especially not papers comparing the PCP to traditional procurement type based on empirical data. However, there are several studies that are focused on the impact of partial factors on the traditional procurement efficiency mostly through the various factors impact on savings [1], [2], [18], [21], [25], [30], [38], [39], [40].

During the relation testing on empirical data several assumed relations were confirmed by statistical tests. An assumed positive effect of volume on savings that was described by [37], [39] was confirmed only for the traditional procurement type and not in the case of PCP where higher volumes have had a negative effect on savings.

Assumed positive effect of number of items on the achieved savings as discovered by [1], [18], [25], was not confirmed in PCP nor the traditional procurements.

On the other hand, the number of participants had in both cases a positive effect on the savings through increased number of price changes. This relation was confirmed by several studies [21], [25], [30]. The effect of a new participant during the procurement was also tested. According to [38] a positive effect on saving could be expected, but in case of our data this relation could not be supported.

As stated in [38] transparency can foster savings in an electronic procurement when auction is used. A positive effect of rank order visibility or visibility of non-price factors [36], [40] could not be confirmed during this study as the transparency setting were not significant neither in PCP nor the traditional procurements.

The developed conceptual model (presented in Figure 10) clarifies the relations inside a PCP process and identifies the most significant differences compared to a traditional procurement. The model can be used as a guide to understand the complexity of such a process. From the practical point of view, a PCP or PPI issuer can identify the bottle necks of the savings generation and adjust the PCP process according to that. On the other hand, the results can be used to justify differences of PCP procurements compared to traditional ones.

This research has its own limitation because of the small number of PCP conducted in the Slovak and Czech Republic and problems of clearly identifying PCP in the private sector, therefore a more detailed data export and further analysis is suitable.