1 Introduction
As part of the tools of the new millennium to generate competitiveness in organizations and foster the use of information and communication technologies ICT, electronic commerce or e-commerce emerged. This type of trade has been increasing over time and more and more organizations are generating these communication channels with the customer to enhance trade relations [85]. Overall, e-commerce is changing the nature of trade worldwide; however, this revolution is developing at different rates in different areas of the planet [208]. At present, its most relevant influence is taking place in Asia, USA and Northern Europe [20], [144], [206].
In addition to its outstanding advance in developed countries, e-commerce is being used more and more in emerging countries where the level of its acceptance is rapidilly increasing. Due to the changes occurring in the way shopping is being carried out worldwide, commerce has passed from happening through direct contact between people to taking place through intangible means and a variety of alternatives, offered based on the customer’s needs, in order to obtain goods. As a result, organizations that have traditionally offered products for sale through traditional channels, are finding better alternatives to make handsome profits through virtual ones.
Understanding how e-commerce technology adoption behaves in the social and economic contexts in which it has not been studied, is of paramount importance to contribute to future research in this field, as well as to provide the business sector interested with channels different from the traditional ones to enter new markets. Nonetheless, given the socio-cultural differences existing among developed and emerging nations, and due to every process of adoption responds to different cultural, social, and political dynamics, the study of technology acceptance in diverse contexts becomes challenging. Consequently, this study intends to examine the approaches, topics, and variables that have emerged in the exploration of e-commerce technology adoption in emerging countries in order identify the existing factors that influence such adoption in different economic, social, and cultural contexts.
Decisions about adopting technologies like e-commerce are influenced by aspects such as the relative novelty of these types of innovations and the greater uncertainty associated to the implementation of new technologies at large. In this regard, several studies have inquired about the factors that influence e-commerce application and found that they vary in terms of object of study, as well as the social, cultural, and economic contexts where this technology is adopted. Such variety of influencing factors has hindered establishing generic models to permit an adequate evaluation of the issue. Accordingly, this research study sought to examine and discuss trends and developments in the analysis of the factors involved in the adoption of e-commerce, led by two main research questions: What are the tendencies in the study of e-commerce adoption factors? What are the most researched topics in this field of knowledge?
The causes that motivate people and enterprises to find out the factors that move the business sector to adopt electronic commerce are diverse. In this regard, [11] claims that the most important benefit that promotes e-commerce adoption in the business sector is the generation of customer trust, while its most significant barrier is lack of electronic payment systems. On the other hand, [13] state that e-commerce has become a very important opportunity among small and medium-sized enterprises (SMEs). SMEs executives have realized that e-commerce allows them to introduce innovation in the market and increase their competitive advantage in a more efficient way.
Regarding pragmatic motivations, an adecuated identification of the factors involved in the adoption of e-commerce at the time of implementing it within a given enterprise context permits incrementing the productivity and profits of corporations and facilitates the generation of strategies consistent with the needs of the implementation environment. Consequently, awareness of the factors that influence e-commerce adoption will make its implementation easier and more likely to be successful for an enterprise. Carrying out research about this matter is thus very important as it would enable the assignation of resources on the factors and variables identified as a priority at the occasion of implementing e-commerce [97], [129], [132].
Regarding the method, bibliometrics have been chosen because it allows a holistic view of the researched context of a specific theme [154]; providing significant data about topics, authors, journals, countries and the most productive and influencing institutions; and supplying information about current trends in the issue and its impact on a given context [58]. Therefore, bibliometric studies are considered to offer valuable starting points for new research on emergent themes [67]. Considering the qualities of bibliometrics, we have chosen this research method to study the factors of e-commerce acceptance. We considered that it would enable suitable access to the research context of the topic, permitting the identification of the main approaches, and the authors that have made the most outstanding contributions in the generation of new models and methodologies to study it.
A bibliometric study was thus utilized in this analysis as a tool to facilitate the identification of tendencies in the adoption of e-commerce. Several indicators for the analyzed field were defined and a dynamic of keywords that permitted the identification and selection of 408 journal articles was developed. The inquiry led to the revision of the main publications on the topic, with the intention of having a holistic perspective of the factors that are most frequently associated to the adoption of e-commerce. We expect that such perspective will serve as a referent for new studies on the evaluation of adoption factors, giving access to a richer number of theoretical sources, permitting an integral vision of the results from previous studies, and allowing the construction of better research designs that could focus on the issue with greater theoretical end empirical validity.
The paper is structured as follows: Drawing on the work of various authors, the first section presents a conceptual framework that elaborates on the most important concepts related to electronic commerce, points out some milestones in its historical development, highlights its importance, and exposes the factors that promote or inhibit its adoption. In a second section, the methodology used for bibliometric analysis is described. In a third section, the obtained results are laid out along with their interpretation. Finally, conclusions and limitations of the study are presented followed by a research agenda.
2 Conceptual Background
This section presents the conceptual framework that supports the bibliometric analysis used in the research methodology. This framework introduces the conceptualization of e-commerce, presents some milestones of development, and highlights its importance in strengthening supplier-customer relations in today's organizations.
2.1 Electronic Commerce (E-Commerce)
Electronic commerce has been defined as a useful tool to reduce the economic gap between countries [149] and as a channel to deliver products and services through the Internet. However, although it has become a source of competitive advantage due to the possibilities offered to both buyers and sellers, its use, especially in emerging economies, is limited [52]. The main advantage mentioned by the authors in the use of electronic commerce relates to the opportunity that it provides for companies to expand their number of customers [18] and its potential to become a medium to boost micro, small and medium sized enterprises--which represent a source of economic growth for many countries [70], [102]. In the United States, for example, it has become a transactional medium used for business by nearly 40% of the third of the population [55]. In addition, e-commerce can transform business management and its potential affects sectors as diverse as telecommunications, finance, industry, trade, education, health and government [203].
According to several authors, e-commerce comprises any transaction made by means of electronic devices, which allows conducting business and transforms internal and external relationships to create value and exploit the market opportunities influenced by new rules of a connected economy. It allows the incorporation of all transactions of information, products, services or payment via electronic networks, and integrates solutions to any transaction via Internet--which is understood as the use of digital communication networks to facilitate the purchase and sale of any product or service [15], [41], [44], [46], [62], [93], [149], [151], [185], [186], [192], [195], [200], [201].
As a product of the convergence of information and communication technologies (ICT), e-commerce can be defined from many perspectives. In [135] the following definitions are set out:
1. From the perspective of communication, e-commerce is the delivery of information, products, services or payment made through telephone lines, computer networks or similar means.
2. From the perspective of business process, e-commerce is the application of technology to achieve automation of transactions or workflow.
3. From the perspective of service, e-commerce is a tool to achieve the objectives of the organization and the customer, and to properly manage costs significantly improving delivery times.
4. From an online perspective, e-commerce provides the ability to view and buy products and information from the Internet or other online services.
Since 1997, when the White House launched the Framework for Global Electronic Commerce, a perception of the need to implement suitable policies for the development and growth of digital systems necessary to strengthen the free exchange of goods, services and information via Internet emerged. In this regard, the referred framework sought to strengthen e-commerce through four general principles that aimed at adopting an unregulated approach that would prevent undue restrictions to electronic commerce. The private sector would ideally lead the process recognizing the qualities of the Internet to develop it on the basis of a global context [43]. Despite the fact that almost 20 years have passed since the adoption of the regulatory framework, electronic commerce has not yet developed as envisaged within the global context. As a consequence, developing countries have basically failed to adequately adopt e-commerce. However, research has been conducted to study the factors that enable and promote its adoption [19], [21], [27], [52], [76], [86], [93], [96], [99], [119], [149], [176], [179], [204], [207]. The existent literature has allowed conceptualizing e-commerce and typifying it according to some characteristics. Table 1 groups the categories and applications of e-commerce defined by different authors in the literature:
2.2 Adoption Factors of E-Commerce
E-Commerce is defined as a channel for any type of transaction, including offering products and services through the Internet, which, despite its advantages, has not yet managed to persuade a greater proportion of users to carry out transactions through this means of shopping [23]. In this regard, several studies have shown that the rate of consumer adoption for e-commerce is low in emerging economies and in micro, small and medium sized enterprises (SMEs) compared to developed countries [10], [18], [54], [70], [73], [99], [141]. Therefore, it has been considered important to conduct studies that examine the trends and developments in the identification of factors that encourage the adoption of e-commerce.
Academic and research studies have sought to explain the phenomenon of adoption of e-commerce with the goal of properly encouraging its use from various fields. For example, through the theory of diffusion of innovation, [65] an integrated model of the adoption of e-commerce for small enterprises in emerging economies was developed. Through this model, it was possible to identify determinant factors for the adoption of e-commerce involving perceived usefulness, perceived compatibility, perceived risks, perceived ease, and perceived complexity, which are defined in Table 2.
By the same token, [45] utilized the notion of orchestrating resources to study the process of adoption of e-commerce; findings from this study suggest that the prevailing competitive environment of each phase results in a capacity of specific coordination that develops through actions focused on resources. Another approach examined the dynamic competition between traditional retail companies and retailers on the Internet, using dynamic games theory [161], [101]. Other researchers have used the descriptive research design with statistical techniques to identify the same factors. As an example, a regression model was used in [21] to anticipate the adoption of e-commerce. In that study, all variables except the business experience of the company were found to have a significant contribution to explain the phenomenon.
Regarding the focus of the papers analyzed, a study of issues such as gender to identify its impact on the adoption of e-commerce showed that it slightly influences its implementation in SMEs. [39]. A qualitative approach on the same subject showed that both organizational and environmental contexts and attributes of innovation influence the adoption of e-commerce [19]. Other studies on e-commerce investigated issues such as consumer behavior, analyzing the factors that influence the type of products / services purchased online [152] or cultural values of the individual behind the buyer's decision [38]. Finally, study [61] considered the performance of companies operating exclusively on the Internet, called e-companies or dotcoms, analyzing factors that are intrinsic to the Web site, such as accessibility, web presence and utility.
On the other hand, [158] identified critical success factors for e-commerce including: adding value, focusing on a niche, expanding, maintaining flexibility and segmenting geographically, adopting the right technology to handle a critical perception, providing exceptional service, creating an effective connectivity, and understanding the culture of the Internet. Meanwhile, [50] studied internal and external organizational factors that may impact the use of Internet among SMEs, finding aspects that enable organizations to be self-efficient through technologies related to marketing use and perceived competitive pressure. The study carried out by [49] analyzed factors that prompt the adoption of e-commerce, examining the influence of perceived usefulness by consumers, and proposing the addition of five sources of income associated with the Internet: price advantage, variety of assortment, value of products, perceived convenience and access to information. In addition, [9] studied behavior in e-commerce adoption finding a significant correlation between perceived compatibility, perceived complexity, perceived observability, perceived uncertainty, among other factors analyzed.
Electronic commerce continues to be an emerging distribution channel even in developed countries. As a result, varied studies have been conducted to identify influencing factors in the adoption of e-commerce, and its implications for big contexts, first level economies, and large size enterprises. According to [168], a study conducted in Kuwait presents a model of online payment trust that reveals the impact of four external factors: personal innovation, tendency to trust, familiarity, and presence of third party seals over the intention of using an online payment through the mediation of three endogenous variable factors: pleasure, risk, and costumer trust. Costumer trust and pleasure are two important factors in the adoption of online payment. By the same token, [133] examine the factors that affect e-commerce adoption in the Arab states. Five countries are represented in this study: Saudi Arabia, Qatar, Kuwait, United Arab Emirates, and Yemen. Findings reveal that knowledge is the most important factor that influences the adoption of e-commerce mediating between costumer perception about the risk and their confidence to adopt it.
In fact, for large enterprises in the First World countries [165] analyzed the combined influence of environmental, technological, and organizational factors that determine e-commerce adoption. The study provides proof that competitive pressure discourages e-commerce adoption in the long term. The findings of this study also reveal that the development of consumption patterns, the technology availability of the market forces, the global reach of the enterprise, and its competences in innovation, continuously favor e-commerce adoption. Meanwhile, [59] showed that organizational, technical, and environmental barriers, as well as technology external factors have affected two initial variables of the Technology Acceptance Model (TAM) that includes perceived usefulness and facility. Likewise, the relations found in the TAM were confirmed including the impact of usefulness and perceived facility over attitude, the impact of attitude over intention, and the impact of intention over actual use.
The study conducted by [175] demonstrates that in developed countries security and privacy are the main aspects that encourage enterprises to adopt e-commerce channels, and consequently consumers to develop confidence in the digital environment. Electronic commerce will be useful in the current management practice, in the design of business policies and strategies, and in the exchange of information between organization executives and leaders. E-commerce nets are playing a fundamental role in online business and customers are more concerned about the ethical issues of electronic commerce, including security, privacy, and trust. The vast majority of public sectors
According to [205], in first world nations a better access to modern ICTs and the adoption of e-commerce applications stimulate bilateral commercial flows at various levels. The study points out efficient use of ICTs equipped with high speed Internet and safe servers as a crucial milestone to unblock e-commerce potential in developing countries. The public sector of depeloping countries acknowledges the importance of e-commerce for business to reach success, which highlights the importance of creating a supporting environment for the adoption and success of electronic commerce in the long term. In the same vein, [103] demonstrate that the public sector should particularly consider its commitment with e-commerce implementation, the generation of policies to regulate it, and the technological resources needed to implement it, as the main factors that mediate between e-commerce adequated use and its successful adoption.
3 Materials and Methods
The methodology for identifying factors of adoption of electronic commerce in this investigation consisted of a bibliometric analysis. Bibliometrics is conceived as the science that studies the trend of an area of knowledge or specific discipline through the computation and analysis of the facets of the written [157] communication and allow us to study science through aspects of quantity, quality, and scientific progress that are to become indicators that would enable to describe the behavior of the analyzed field [172]. However, to perform this type of analysis one must consider that full access to scientific production is unattainable [17], reason for which reliable and rigorous sources must be regarded to ensure the relevance and significance of the knowledge to which one has access, being the choice of the database from which the information will be obtained, the fundamental starting point to begin this process [138].
It is important to highlight that similar studies in the same area of interest have been conducted; for example, the one carried out by [83], which explored worldwide tendencies of e-commerce during the 1996-2012 period. Some important results from this study evidenced working papers, and articles as the main type of documents in which the matter was reported. Besides that, the three main conferences on the topic analyzed reached a participation of the 10,6% of the total of working papers and the first 20 publications participated with a 31,1% of the total of journal articles. Finally, Computer Science Information Systems, Business, and Informatics Theory Methods were the three most popular topic categories found.
In a similar study conducted by [124], a bibliometric analysis of online shopping was carried out. This analysis was done in the Web of Science (WOS) data base during the period 2000-2014, generating 772 related articles, which were analyzed under the criteria of productivity, sustainability, and impact. Within their main conclusions, the authors state that online shopping is becoming increasingly popular worldwide and highlight Journal of Business Research, Marketing Science, Information and Management, and Psychology and Marketing as the top journals publishing articles under the three evaluation criteria used to carry out the analysis. However, as there is no single prominent journal on the subject studied, research publications on the matter are fragmented, and researchers publish their work on a variety of journals in the fields of technology, marketing, operations, or management, depending on the content of the paper and the authors’ preference. Finally, the authors point out Izak Benbasat as the top author of papers written from the perspectives of productivity and sustainability, since he wrote 10 articles in 7 different years within the period studied.
Bibliometric analysis has thus been proven a methodology that permits a quantitative identification of tendencies and behavior on a specific area of study through a variety of indicators, depending on the factors of interest identified for a particular research study [198]. The bibliometric approach is mainly based on the search of quantitative characteristics of the topic studied: features of research publications, such as article titles, key words and phrases; authors, including their institutional afiliation, coauthors and reputation; and books and journals, including titles, topics and country of origin [75], permitting the identification of patterns useful for research advance and scientific development [148]. In this article, a bibliometric analysis of the factors involved in the adoption of e-commerce was made from the data of academic publications reported by the SCOPUS database. SCOPUS provides tools to manage information, and meets other criteria such as the amount of citations and accessibility, which make it one of the most used databases in this type of analysis in the literature [77]. Besides its accessibility, SCOPUS was selected in this study due to being considered a valid, reliable, and timely source of academic information, which makes it the largest database regarding citations and abstracts from peer-reviewed literature and highquality sources in the Web [83], [124]. On the other hand, the researchers intended to address a data base different from the one used in the studies mentioned (WOS). In addition, high levels of correlation in their reported bibliometric indicators have been found between Scopus and WOS, which make the two data bases statistically comparable and stable. Thus, the indicators of production and scientific quality would equally support findings generated by either of the two data bases [16].
Subsequently, a search equation was defined per the object of bibliometric analysis; i.e., the factors involved in the adoption of e-commerce. For this purpose, key terms equivalent to electronic commerce (e-commerce - electronic commerce) were considered as search criteria along with stems equivalent to the term adoption (appropriat * - adopt *). Those key terms were searched in the title and keyword fields, with no restriction of time for a broader horizon of analysis (taking as cutoff date, items recorded in Scopus until April 1, 2017). Tests with 7 equation options, which differred from each other in their key words, distance between the terms, and the search scope (between title, abstract, key words and full text), were carried out. The analysis of results and consensus among the participating researchers determined the following search equation: TITLE (e commerce W/5 adopt* ) OR ( e commerce W/5 appropriat* ) OR KEY ( electronic W/5 commerce W/5 adopt* ) OR ( electronic W/5 commerce W/5 appropriat*), which was the one that reported the highest affinity with the studies conducted on the theme. Including and excluding criteria were completely based on the search equation since all the downloaded articles resulting from the search were analyzed. The equation yielded a total of 408 records, which were processed and interpreted using Microsoft Excel and Data Analyzer (Thomson Reuter’s software) to identify indicators of quantity and quality. In addition, free software CYTOSCPE was used for calculating indicators of structure and network topology maps of authors.
Among the indicators calculated in this study, there were statistics about activity (amount) that permitted visualizing the state of the art, within which the number and distribution of publications, productivity, dispersion of publications, and collaboration in publications are included. Likewise, there were impact indicators (quality), used to assess the incidence of authors, papers, and / or journals in a specific area or discipline [28], [53]. Finally, there were structure indicators that allowed measuring connections between different authors and publications [170].
Some of the indicators permitted to evidence the factors involved in the adoption of e-commerce in terms of quantity, indicating items with higher impact such as the years with the highest production, the most representative journals, and the most prolific authors in the area. The measurement of publication quality through impact (in terms of number of citations) is widely accepted, mainly based on what the Frascati Manual and the Organisation for Economic Co-operation and Development (OECD) propose for the measurement of Research and Development (R&D) activities, and the analysis of research systems and scientific specialization both of researchers and countries [53], [95], [130], [140], [163]. Based on the previously mentioned, the number of citations of an article has been taken as a point of reference for the calculation of quality indicators. By the same token, quality indicators facilitated to explain authors’ impact, and reasons for journals and articles to be the most outstanding and cited [95], [140], [163]. Structure indicators allowed to observe the behavior of working nets on the theme [70], [210]. Finally, trends in the study of the explored field were analyzed through the observation of the behavior of the key words, which permitted identifying increasing, decreasing and emergent fields in the area [198].
4 Findings
The results of the analysis are presented in this section based on the indicators proposed in the methodology. The information has accordingly been grouped in four categories: quantity indicators, quality indicators, indicators of structure, and research trends in e-commerce adoption factors.
4.1 Quantity Indicators
Based on the obtained search equation, the bibliometric quantity indicators calculated for the topic of Electronic Commerce are laid out in this subsection of the findings.
In Figure 1, the indicators show a growing interest on the topic since 2001, 2006 being one of the most productive years with a number of 36 published articles. In the following years of 2007 to 2009, the number of publications decreases steadily to present a new increase in 2010, this being the year of the highest increment with 39 published articles. From 2011 to 2016, a steady decline in the number of publications has been showing, which may be an indication that the identification of factors of adoption of e-commerce had its process boom between 2001 and 2006 in the theoretical realm and retaking known models of technology acceptance. Later, we have seen that publications have focused more on identifying adaptations of adoption models according to the particular socio-cultural, economic and demographic contexts. In conclusion, we observe that research tendency in the topic has focused on more specific alternatives, opting for the evaluation of factors in particular business dynamics (small and medium-sized business). Such evaluation has been based on the use of quantitative models in agreement with the Diffusion of Innovation Theory, in which inquiry about the validity of the factors of those models and their pertinence in the successful adoption of e-commerce has been conducted.
To determine the journals with major impact in the field, it is necessary to identify the ones that publish the 80% of research studies on the topic [117]. However, regarding the topic of this article, that percentage is disseminated among 62.07% of the journals (87 journals) that publish on the subject; therefore, it is not possible to present a traditional graphic grouping the journals with the highest production. It was thus considered appropriate to take the 10 main journals that publish on the topic as a reference, selected on the basis of the number of articles that address e-commerce adoption factors (the number of citations from these studies has not been considered yet). Accordingly, Figure 2 presents the ten journals with the largest number of publications related to the field of study.
IFIP Advances in Information and Communication Technology, is the top journal with 7 articles published. This publication compiles the disclosures made in the framework of the International Conference on Electronic Business (ICEB), where technology issues involving e-commerce are discussed, ranging from platforms and Web services, to knowledge models for e-commerce for both the environment and the company, as well as peculiarities of the different applications of this technology [118]. The second journal, with 6 articles, is the Industrial Management and Data Systems, in which papers about the social, cultural, organizational and cognitive impacts of e-commerce technology and advances in organizations worldwide are published [120]. The third journal with most publication is the Journal of Internet Commerce, with 6 articles, which "promotes greater access to high quality presentations at conferences and workshops on computer fields" [121]. The fourth publication, is the Electronic Commerce Research and Applications Journal [57], [122] with 5 articles. This journal aims to create and disseminate knowledge about e-commerce, considering it is a multidisciplinary area that should be developed in cooperation with other existing fields, such as Information Systems and Technology; Marketing, Finance and/or Supply Chain Management; Strategy and Management; Public policy; Telecommunications and Computer Science; and Legal Studies. In fifth place is the Indian Journal of Science and Technology with 5 articles and the following four publications have 4 articles each: Advanced Materials Research; Advanced Science Letters; Journal of Small Business and Enterprise Development; and finally, Mediterranean Journal of Social Sciences.
To continue the analysis of the cumulative percentage of publications per journal, Figure 3 permits to conclude that there is no Pareto Law, due to 62.07% of the journals published 80% of the items. Therefore, we proceeded to separate the percentage of journals per quartile, obtaining the following results: (1) 9.20% of the journals published 25% of the articles; (2) showing a significant increase in the proportion, 24.14% of the journals published 50% of the articles; and (3) the remaining 52.9% of the journals published 75% of the articles. This shows that on the subject of adoption factors of e-commerce, knowledge is dispersed among several journals related to the area of Information Technology and Communication, and there are no publications with a significant concentration of studies on the field.
Regarding the analysis of researchers with production in the field, Figure 4 displays the list of the ten researchers who have most productions in this subject area. MacGregor, R.C., who has 9 publications, appears on the top. Robert MacGregor is an associate professor in the School of Informatics and Computer Science at the University of Wollongong in Australia. His research expertise is in the area of information technology (IT) and electronic commerce (E-commerce) in small and medium sized enterprises (SMEs). He is the author of articles and conference publications examining the use and adoption of ICT in SMEs series. MacGregor is also the founding editor of the Australasian Journal of Information Systems and was president of the Australian Information Systems Conference in 1992 [124]. The following authors in the list are Vrazalic, L., with 8 publications; Govindaraju, R., Kartiwi, M. and Zhu, L., with 5 publications each; Kurnia, S., Scupola, A. and Thatcher, S.M.B., with 4 articles each; and the last two authors with 3 publications each are Al-Somali, S.A. and Chandra, D.R. (Figure 4).
In connection with the number of publications by journal, the cumulative amount of publications of the top ten authors (see Figure 5) shows that there is no Pareto Law, since 63.46% of the authors published 80% of the articles. Given this situation, we proceeded to separate the cumulative percentage of publications for the first ten authors, by degree of relevance, obtaining that: (1) 12.18% of the authors published 25% of the articles; (2) 34.62% of the authors published 50% of the articles; and (3) 59.0% of the authors published 75% of all articles. The information collected reveals an equitable distribution of articles per researcher; i.e., in terms of number of publications no author stands out.
By applying natural logarithm to each of the axes, the law of productivity was obtained for journals (Figure 6) and for authors (Figure 7). This law postulates that productivity does not follow a linear but logarithmic distribution; it can thus be inferred that "the more published articles an author has, the easier it seems for them to produce and publish" [124]. In conducting this analysis, the authors of a given set of publications are distributed in three levels of productivity: the small producers (log n = 0), the media producers (1> log n> 0) and the large producers (log n> 1) [125]. Figure 7 shows the squared value of the regression coefficient of the adjustment on cumulative number of articles against the natural logarithm of journals, being greater than 0.9, demonstrating a proper fit for the proposed model for productivity.
The law of productivity was also verified for authors in the field. Figure 7 presents the results obtained with respect to this law. The graphic shows a determination coefficient greater than 0.8 confirming a suitable adjustment for the proposed model for productivity. As mentioned above, the authors analyzed have an average level of productivity.
Finally, it is worth noting that 48% of the records obtained from the search equation correspond to articles and 40% are papers presented at conferences. The participation of other types of documents on Electronic Commerce is 7% (see Figure 8). Accordingly, one could say that the dissemination of academic and research findings in this field is mainly concentrated on publications in refereed journals and participation in conferences that allow greater interaction and associativity between authors.
In addition, classifying these publications by area of knowledge shows that most of them are located in the area of Computer Science (50.0%) followed by the area of Business, Management and Accounting (38.9%), Engineering (18.1%), Social Science (15.9%), and Decision Science (13.4%). This shows that the main interest has focused on developing better computer systems (software, apps, etc.) that allow greater acceptance of e-commerce in contrast to the area of Business and Management, which pursues a greater ownership of technology towards improving profitability levels and the use of such means for companies to sell their products and for consumers to make their purchases.
4.2 Quality Indicators
This section presents the bibliometric indicators for quality, calculated for the factors of adoption of e-commerce, according to the defined search equation. As mentioned above, quality indicators are closely related to the publication impact; in other words, the number of its citations, [53], [69], [95], [130], [140], [163], [198], [210].
Based on quality indicators, it was possible to find the ten authors with most citations on the subject (see Figure 8). Within those ten are Almeida V.A.F. et al., who tops the list with 168 citations. Their publication dates to 1999, and it is an investigation that proposes a methodology for characterizing and generating e-commerce workload models. In the paper, the authors first introduce a state transition graph called Customer Behavior Model Graph (CBMG), used to describe the behavior of groups of customers who exhibit similar navigational patterns. Following, a set of useful metrics, analytically derived from the analysis of the CBMG is presented. Next, they define a workload model and show the steps required to obtain its parameters. Finally, they propose a clustering algorithm to characterize workloads of e-commerce sites in terms of CBMGs [125], [128].
The consecutive authors in the graph with the highest number of citations are the researchers Kevin Zhu and Weiyin Hong, who have 151 citations each, of an article whose methodology was based on multi-nominal logistic regression, demonstrating that technology integration, web functionalities, web spending, and partner usage were significant adoption predictors. The model showed that these variables could successfully differentiate non-adopters from adopters. Further, the migration model demonstrated that web functionalities, web spending, and integration of externally oriented inter-organizational systems tend to be the most influential drivers in firms’ migration toward e-commerce, while firm size, partner usage, electronic data interchange usage, and perceived obstacles were found to negatively affect e-commerce migration [82], [126]. This study provided adequate tools for the implementation of strategies that would adopt Internet services in a cultural environment different from that of developed economies.
The following researcher within the top ten in the list is Licker P.S., with 148 citations recorded. This article from 2005 shows that the effects of network externalities influence the based on perceived organizational e-readiness (POER) and perceived environmental e-readiness (PEER) [127], [134]. The quality indicator does not have any similarities with that of quantity, because none of the 10 authors with the highest number of citations is part of the 10 authors with the highest number of publications (compare Figures 4 and 8).
Regarding impact, the journal with the highest number of citations is International Journal of Electronic Commerce (IJEC) [90], with 148 citations. The IJEC is a quarterly journal devoted exclusively to electronic commerce, which aims to provide an integrated view of the field by presenting approaches to multiple disciplines. It does so through the publication of topics such as the effects of e-commerce market, the transformation of the organization with e-commerce, business models in e-business, management and supply chain collaborative commerce, the co-creation in e-commerce, social network analysis, and e-commerce in the globalization of business, among others [136].
Information and Management, with 78 citations, stays in second place. This journal serves managers, professionals, database administrators, and senior executives of organizations, which design, implement and manage Information Systems Applications. The major aims of the journal are (1) to collect and disseminate information on new and advanced developments in the field of applied information systems; (2) to provide material for training and education in administrative data systems; (3) to encourage further progress in information systems methodology and applications; and (4) to cover the range of information system development and usage in their use of managerial policies, strategies, and activities, for business, public administration, and international organizations [135].
Below in the list is the European Journal of Purchasing and Supply Management with 72 citations. This journal fulfilled its 30th anniversary making contribution to research. It provides integrated information systems and how these can be developed and used effectively, to manage information and knowledge that may lead to more effective results in the organizations’ vision [137].
Out of the 20 journals with the highest number of citations in the topic, 62.50% pertain to Scopus Q1, 16.67% to Q2, 8.33% to Q3, and about 12.50% are not ranked in the quartile due to being Proceedings (see figure 9). This fact evidences that the journals with the highest quality are the ones that report a higher number of citations in the topic. Besides that, among the thematic areas published in these journals, two approaches stand out: (1) a Business Management approach, with the following subtopics declared in Scopus: Business and International Management, Management of Technology and Innovation, Strategy and Management; and (2) a Systems Approach with the following subtopics declared in Scopus: Computer Networks and Communications, Information Systems and Management, Human-Computer Interaction, and Information Systems, which evidences the interest in both fields of knowledge about publications related to e-commerce adoption factors. Finally, the bibliometric analysis identified that the topic of Electronic Commerce has been studied since its inception in 1996, when 3 citations were recorded. In the years 1998 and 1999, the thematic one obtained a significant increase in the number of citations, with 47 in 1998 to reach the highest number of citations, 104, in 1999. The article that achieved this amount of citations is the aforementioned Methodology for Workload Characterization of E-commerce Sites showing that the most outstanding authors of the thematic have contributed the findings with greatest value in the field [125] (see figure 10).
4.3 Indicators of Structure
Structure indicators measure the connectivity between publications [138], the authors, and areas of knowledge; these are often associated with the construction and analysis of social networks, which consist of nodes (vertices) and links. In the case of a bibliometric analysis, nodes are authors (people who research and publish their results) and links represent co-authorships. Thus, from the analysis of these networks, it is possible to identify outstanding researchers in the field and define the dynamics with which they work together [139]. The results of the indicators of the network of authors are presented in Table 3.
The analysis of structure indicators arose 863 authors in the field, and evidence that, on average, each author has published with 2 other authors (2,125). Additionally, the network density was found to be close to zero (0.002), which shows that the strength of connections between the authors of the entire network is very low. This could indicate that the dispersion of production is significant. In addition, the group subnets formed between its nodes (authors) is also significant since they have a degree of clustering of 0.622. This does not however imply connectivity since heterogeneity (0.671) was found to be greater than said grouping--which implies that some subnets tend to look like a star. This shape means that there is a central node in the subnet connected to the others, which can be a bridge for connections between authors while their disappearance could cause the isolation of nodes with which it has links; i.e., an author is the one who serves as a connection with others to work the different subject area knowledge.
The typical explanation about cohesion for the effect of network position on knowledge outcomes is that centrality provides individuals with timelier access to richer and more diverse information, increasing the extent to which they learn from their network and their potential to synthesize and recombine this information into novel ideas [153]. At the other extreme of the spectrum, the cohesion perspective views ties as pipes through which information and knowledge flow and characterizes network position in terms of centrality; that is to say, the extent to which an individual is well connected, both directly and indirectly, to others in the network.
Networks are not merely a representational form of social relations. Instead, research in network theory posits that the structure of relationships affects the opportunities and constraints of individual action and their outcomes. Networks affect opportunities for action [68]; additionally, the collective actors as well as the relations that they sustain are the building blocks of social networks.
Table 1 also shows the cumulative structure indicators in two periods (1996-2005 and 1996-2017), with the purpose of analyzing the evolution of the network of authors over time. There, there is an increase in the number of authors (number of nodes), number of independent authors (number of isolated nodes), and number of isolated subnets (number of isolated components). However, the growth rate of the nodes is much larger than that of the isolated nodes and connected components. Therefore, there is a decrease in the dispersion of isolated subnets and independent publications (variations of component indicators connected by nodes and components isolated by nodes respectively). Evidently, authors who are adhering to the field, do so by means of already formed networks.
In this sense, the subnets have been increasing in size and decreasing in density, which is reflected in the evolution of the degree of clustering and the diameter of the network. In spite of this, a slight increase of the density of the network reflects that the new authors are related to some few authors of the subnet to which they were connected, and as a consequence the expected characteristic distance has not undergone variations.
Another characteristic evidenced in this network of authors is that there is only one link of the shortest path that connects all the nodes of a network (1,210) and that in turn, the maximum distance between any pair of nodes of the network is equivalent to 4, which corresponds to a typology of networks of internal knowledge management, used to maximize the application of individual knowledge to the objectives of the organization. These networks evolve through thematic mapping of experience within the organization, and environments conducive to knowledge sharing [143].
On the other hand, the network is increasingly decentralized and more heterogeneous; that is, there are no common central authors for the entire field, but there are no common central authors for the subnetworks. There is a possibility that these subnetworks are related to the disaggregation of the field of knowledge and therefore may represent the fragmentation of knowledge. Finally, the study examined search conceptualize knowledge networks as consisting of nodes that serve as repositories of knowledge and agents that search for, adopt, transmit, and create knowledge. For this reason, nodes are simultaneously sources and recipients of information and knowledge. Likewise, such relationships constitute a means by which nodes search for information and knowledge; a medium through which information and knowledge diffuse and flow.
4.4 Research Trends in E-Commerce Adoption Factors
Keyword analysis is used to identify the topics studied in the field that are generating greater publication, i.e. have a greater research tendency. Figure 11, presents the keyword dynamics, which is constructed by performing a normalization of keyword search criterion, linking concepts with the same meaning, and making a thesaurus of keywords recovered in the search. Afterwards, a count of the words is performed in order to spot the most important ones. This analysis is important because they are the words used by the authors to characterize their investigations, which turns them into a source of primary information.
In Figure 11, the most dynamic topics in the analyzed period are evident. Accordingly, SMEs becomes the keyword that has the highest recurrence and therefore the context in which e-commerce is being researched the most. Such research shows that e-commerce is being implemented in the SMEs, through the use of information and communications technologies (ICT) to facilitate its access [21], [98], [110], [120], [125], [126], [147]. In this regard, it is clear that promoting e-commerce in the sectors of small and medium sized enterprises has become a national development policy in some countries [40]. Similarly, in Latin America, there have been studies that have identified factors that reflect greater impact on the adoption of electronic commerce, namely the company size and their level of maturity in the implementation of ICT [54]. On the other hand, [112] discusses the importance of the use of e-commerce for the development of small and medium sized enterprises as it constitutes a source of competitive advantage. Other studies examine the effectiveness and impact of government policies to support the use of e-commerce and propose some recommendations [106]. The study of [159] concludes that the rate of adoption of e-commerce in SMEs is directly related to the rate of adoption of ICT. Meanwhile, [147] presents evidence that SMEs are more prone to the adoption of electronic commerce, and have more financial and technological resources as well as greater external pressure to be part of such trade.
The second recurrent keyword in Figure 11 is adoption, which evidently refers to the adoption of e-commerce. It indicates that research has explored the effort that is being made in small and medium-sized companies in emerging countries to incorporate e-commerce to improve performance [94], [116], [120], [123], [125], [147]. Among the analyzed research, various factors of adoption of e-commerce are mentioned, and those that have to do with organizational and financial issues stand [4], [7], [26], [27], [72], [141], [196]; for example, [81] states that the Organization for Information Ecology (OIE) is an important factor for the adoption of e-commerce and the paper proposes a model that integrates this vision to previously known factors.
With respect to B2B e-Commerce, studies show that foreign aid is crucial to increase the levels of its adoption, and that SMEs whose managers completed higher education are more likely to adopt it [112]. On the other hand, [125] examines, through the TOE Framework (Technology-Organization-Environment), the factors that allow the adoption of B2B e-commerce in Europe; as a result, the proposed model found seven factors including the development of technology, technology integration, the size of the company, the obstacles, the level of education, competitive pressures, and socio-business collaboration. The study also explained that the exponential growth of the Internet population encourages the adoption rate of e-commerce in different parts of the world. It is noteworthy that the adoption of B2B e-commerce in an emerging country was simulated with a model in which the size of the company, the business experience, the experience of Internet use, the amount of computer equipment, and the income and percentage gain of the company were considered as explanatory variables. This model showed that all variables except the business experience of the organization have a significant contribution to explain the adoption of electronic commerce by SMEs; in fact, it revealed that the number of employees and business experience of the company is negatively related to the adoption of electronic commerce [21].
With regards to Technology Acceptance Models, the Technology Acceptance Model (TAM) as well as the Theory of Diffusion of Innovation (IDT) and the TOE framework have been used in order to explain the adoption factors in small and medium sized enterprises in emerging countries [14], [19], [37], [64], [125], [176]. On the other hand, the study of e-business is specifically addressed by [184], showing the trends in research in this field; findings of this study also indicate that this field of research is multidisciplinary and is evolving to integrate with the supply chain as well as with specialized technologies in line [35]. The TOE framework identifies three contexts that can influence the use of a technological innovation in the field of organizational psychology [181]: technological context, organizational context, and environmental context [3], [84], [111], [113], [115], [191]. The TOE framework has been successfully used to understand the key factors of information systems that affect adoption [84].
Likewise, [47] introduced the Technology Acceptance Model to explain acceptance in the use of information technologies for the first time. This model is based on the Rational Action Theory (RAT), a psychological theory that intends to explain behavior based on intention. According to the RAT theory beliefs influence attitudes, which lead intentions, generating conduct [8]. The Technology Acceptance Model adopted this relation and applied it to the acceptance of information systems by users [1]. The TAM includes two primary predictors: the easiness of use and the perception of usefulness. Behavior intention acts as an independent variable, which in RAT is supposed to be tightly linked to actual behavior. This way, the TAM establishes the causal relations between perceived usefulness, perceived easiness of use, attitude towards use, and actual use of technology [105]. Finally, [145] argues that the TAM can explain user behavior through a broad spectrum of end user computing technologies.
Regarding the Supply Chain Management research has been conducted on its current and future state, focusing on the use of ICT, e-commerce and supply chains, and specifically on the type of technology being adopted, adoption factors, benefits, and barriers affecting the use of e-commerce as a way of innovation for logistics service providers [164]. In addition, [101], [162] conducted studies where B2B interactions were investigated in the supply chain showing that this tool improves its management. Finally, [25], [171] conclude that although the rate of e-commerce among providers of logistics services is small, those which have adopted e-commerce have seen a high level of benefits and a contribution to the development of e-commerce within the company in general terms.
Figure 12 displays the fields of research in e-commerce that are in growth, decrease, and increase; this analysis is based on the periodical behavior of keywords, allowing to identify trends for both the topics with a growing and less interest of researchers, and making evident the new topics of interest in the area studied. According to Figure 12, the research fields that are growing for the selected search are small and medium sized enterprises, adoption, developing countries, model of technology acceptance, e-business, information and communications technology (ICT), management of the supply chain, Theory of Diffusion of Innovations, ICT adoption models, and mobile commerce. These results point to the identification of e-commerce adoption factors in SMEs as an option to improve their position and competitiveness in the market. In addition, the graph pinpoints models that are being used to examine the factors that motivate the adoption of e-commerce such as Theory of Diffusion of Innovations and Technology Acceptance Models, and statistical analysis through Structural Equation Models. Regarding the fields in decline, the graph shows that these are electronic markets, consumer behavior, social theory, economic analysis, electronic data interchange, ICT adoption, optimization of mobile services, and tourism industry. Finally, the nascent fields are Technology-Organization-Environment framework (TOE) and adoption of systems.
Based on the identified themes, further research on the matter should focus in differentiating influencing factors in the adoption of e-commerce, based on the economical and social context of users, as in the case of variables such as private security and perceived risk, which usually differ according to the context. In addition, findings suggest researching about differences that can occur when population divisions such as gender, intention of using this channel for the shopping/buying of products, and the influence of culture in the adoption of e-commerce are made [97]. In the same vein, research studies should be linked to theoretical models of innovation adoption and diffusion, with the aim of having stronger statistical support in the analysis of variables. An approach based on the structuration theory, which focuses on studying social phenomena with the intention of understanding the interrelation between individuals and society [66] is one that could be useful in further research. On the other hand, theories of a stronger explanatory nature could be utilized to explain the role of culture in the generation of individuals’ perception, which would lead to a better comprehension of the relationship between information privacy and culture [132]. Finally, one of the flaws in research in this field is the lack of studies with a longitudinal standpoint that could provide a better perspective in the process of adoption of electronic commerce from a followup of determined moments along the process.
Based on the previous graphic, Table 4 groups adoption factors of electronic commerce through the analysis of the articles containing the key term adoption. The title of the article has been included to provide the focus and context of the article and facilitate constrast between the identified adoption factors. Following, we will provide an agenda intended to guide future research and provide recommendations about the important factors to be considered in the adoption of e-commerce:
Table 4 displays a series of factors that, according to the authors mentioned, enable and encourage e-commerce adoption. With the intention of higlighting recurrent e-commerce adoption factors, Figure 13 shows the ones identified by the authors consulted in chronological order:
The figure 13 allows to observe that during the last decade the studies about the identification of e-commerce adoption factors have been prolific and demonstrated their evolution around diverse organization characteristics. It is worth emphasizing that in the early years (2006 and 2007) varied factors involving both organizational and technical aspects were identified. Research in the field has gradually advanced, going from studies that explored the technical factors to studies based on technology adoption and acceptance models (TAM), as well as on consumer behavior models (TPB) (2007-2010). Research has as well included studies intending to explain the identification of these factors through an articulated model (TOE), and the ones that have identified the importance of a suitable governmental structure as a mediating factor that impels e-commerce adoption, especially in developing countries (2010-2016). Based on Table 4 and Figure 13, we next generate a research agenda intending to guide the study of adoption factors in e-commerce in order to demolish barriers around this technology to access the benefits that it promises.
5 Research Agenda
Based on Table 4 and Figure 13, we present a research agenda that provides recommendations about the factors that should be taken into account in the adoption of e-commerce. This agenda is intended as a guideline for future investigation.
1. In first place, key aspects identified in the adoption of e-commerce are the role of the organization managers, and its organizational culture and structure. In this regard, organizations that count on financial and technical resources and have a flexible structure are more likely to taking risks and adopting e-commerce as part of their strategy.
2. Regarding the factors that stimulate e-commerce adoption in small and medium sized enterprises, findings point to the importance of considering technological opportunism and readiness, owner experience with computers, support within the organization, relative advantage, and compatibility [182]. In addition, factors related to the national context have been found to have a significant impact in the adoption of e-commerce in certain countries. In countries such as Malesia and Indonesia, for example, e-commerce has been affected by perceived relative advantage, perceived compatibility, manager/owner's knowledge and expertise, management characteristics and external change agents.
3. Some results point out the importance of considering gender factors influencing e-commerce adoption. In this regard, the influence of social norms is stronger in the female group while the influence of enjoyment is stronger in the male group. Regarding B2B e-commerce adoption, some findings indicate that education level along with exposition to Internet use become significant predictors to explain the rate of adoption of this new technology.
4. Regarding research methodologies to study e-commerce adoption factors, statistical analysis, structural equations, as well as models such as TAM and theories like DOI, findings of some studies indicate that marketing and competition benefits, essential benefits that support strategy and development, and business efficiency benefits all positively contribute to decisions that lead to an advanced level of e-commerce being adopted.
5. Finally, future work to be considered includes a meta-analysis or a systematic review of literature about the adoption of e-commerce in SMEs in emerging economies, in order to propose a model for it in accordance with the economic and social context, and the limitations of SMEs in such economies. On the other hand, a comparative analysis among the most used models in the evaluation of the adoption of e-commerce (Technology Acceptance Model; Theory of Planned Behavior; and more recently the technological framework model, Technology-Environmental-Organizational Framework, is recommended to put forward a unified integrative model that compiles the findings in this research field.
We would like to note the importance of making e-commerce a value generating tool in small and medium sized enterprises, considering that such enterprises constitute the 95% of the productive apparatus in developing countries. This could be achieved and consolidated through the strengthening of organizational processes, building flexible structures that permit an adequated level of risk in order to introduce this type of technology. In addition, since e-commerce adoption is a social process, the study of the factors that allow it in the market of small and medium sized enterprises should integrate qualitative and quantitative research methodologies.
6 Conclusions
The analysis evidenced that the study of factors of adoption of e-commerce is developing significantly in small and medium sized enterprises, due to them being most prone to the implementation of this commercial channel. Current studies are being conducted in a focused way in emerging economies, because it is in these countries where SMEs make up a large proportion of the business sector. With regards to the slow adoption of e-commerce in small businesses, they are attributed to various barriers that relate to perception of adoption and economic obstacles to invest in technology. A decisive factor in adopting e-commerce is perceived ease, this being a predictor of behavior for consumer purchases via the Internet. On the other hand, increased acceptance in the adoption of technology by consumers can be positively affected by the use of smart phones, because of its flexibility and ease of use, in addition to the installed capacity of banks to support credit card transactions, while still being complex, given that purchasing power becomes an important factor to prevent consumers from making transactions online.
It was possible to observe through the bibliometric analysis that great importance on the research field of e-commerce lies currently on the analysis of the processes of adoption of e-commerce in small and medium-sized companies in emerging countries, using robust models as the TAM, the Theory of Planned Behavior (TPB), and more recently, the TOE. On the other hand, statistical analysis through structural equations stands as the main form of analysis of information reported by research on the factors influencing the adoption of e-commerce; this is important because it constitutes a statistically robust tool to shape these factors and their role in the use of e-commerce. Finally, a strong interest is emerging in implementing e-commerce in the management of the supply chain through logistics providers, using it as an alternative that has resulted in benefits and high levels of productivity in companies.
With respect to the factors that favor the adoption of e-commerce, the following were identified: prior use of Information and Communication Technology (ICT) in the company personnel, the existence of national policies in support of e-commerce, the size of businesses, the level of maturity in the implementation of ICT in organizations, the provision of financial and technological resources for the implementation process of e-commerce, external pressure to be part of this type of trade, and possibilities for external assistance (advice or support) to implement e-commerce platforms.
Regarding the classification of e-commerce by type of transaction (see Table 1), the analysis indicates that the mode reporting more research is business-to-business (B2B) in emerging economies, which is linked to the growth in Internet users in emerging countries. This evidence the need for the business sector in emerging economies to increasingly adopt E-commerce among their strategies for market expansion. Among the factors that influence this process are the growth of the company, business history, recognition of the company by consumers, and the marketing strategies they use to reach potential customers.
It was observed that decreasing research fields (electronic markets, consumer behavior, social theory, economic analysis) focus on macro variables in the context of adoption of e-commerce. The decreasing interest of researchers on those aspects of the study of the adoption of e-commerce may be explained by the fact that research has (1) focused and specialized in particular factors that determine their use, and (2) oriented towards the perception and satisfaction of the business sector and consumers with the variables involved in its adoption. Among those variables are perceived confidence, attitude towards the use, and perceived ease of use, perceived usefulness, safety, and innovativeness.
The bibliometric analysis results show a growth in interest in the subject since 2001, 2006 being one of the most productive years with 38 published articles. With respect to publications, the Proceedings of the International Conference on Electronic Business (ICEB) was found to be of significant interest, compiling disclosures made within the framework of the conference, where technology issues involving E-commerce are discussed. That includes platforms, web services, knowledge models for e-commerce for the environment and for the company, as well as the peculiarities of the different applications of this technology.
The analysis of the law of productivity concludes that there is no Pareto Law, since 71.48% of the journals published 80% of the items, so we proceeded to separate the percentage of journal by quartile, obtaining that 8.20% of journals published 25% of the articles; presenting a marked increase in the proportion, 28.52% of the journals published 50% of the articles; and finally, the statistics showed that 64.45% of the journals published 75% of the articles. This indicates that knowledge is scattered in various related scientific journals in the area of Information Technology and Communication and there are no academic journals that have a significant concentration of studies on the topic of e-commerce adoption factors.
The author with highest productivity was found to be Robert MacGregor, Associate Professor in the School of Informatics and Computer Science at the University of Wollongong in Australia. His research expertise covers the area of information technology (IT) and electronic commerce (e-commerce) in small and medium enterprises (SMEs). Regarding the impact of publications on the subject, Chwelos, P (2001) tops the list with 489 citations; the paper presents results of an investigation of a parsimonious model that postulates three factors as determinants of adoption of electronic data interchange (EDI): provision, perceived benefits and external pressure. The authors conclude that external pressure and readiness are considerably more important than perceived benefits. The journal with the greatest impact is Information Systems Research (ISR), with 164 citations. ISR is an international journal that focuses on theories, developments and research on information systems in fields such as organizations, economy, and society in general.
7 Limitations
Among the main limitations of bibliometrics is the lack of a solid theoretical background supporting the construction of bibliometric analyses, which implies difficulties to generate strong enough indicators and leaves the inclusion and interpretation of a large part of the indicators to the criteria of researchers. On the other hand, dependence on a particular data base causes the risk of leaving aside journals which might be too recent to have been indexed or journals that have decided to get indexed in a different data base, which reduces the scope of information to be accessed.
The definition of systematic search equations, requires the analysis of information structures in data bases, which hinders the search in unpublished papers or papers published in lower quality journals or events, as well as in grey literature in the field of knowledge. Consequently, access to knowledge generated through other scientific dissemination media is left aside. A similar situation occurs with the language since main scientific data bases include a larger proportion of journals in English and restrict the possibility to access findings of research published in other languages.