Back to Journals » Risk Management and Healthcare Policy » Volume 13

Risk Management in Executive Levels of Healthcare Organizations: Insights from a Scoping Review (2018)

Authors Ferdosi M, Rezayatmand R, Molavi Taleghani Y 

Received 19 September 2019

Accepted for publication 27 February 2020

Published 19 March 2020 Volume 2020:13 Pages 215—243

DOI https://doi.org/10.2147/RMHP.S231712

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Kent Rondeau



Masoud Ferdosi,1 Reza Rezayatmand,2 Yasamin Molavi Taleghani3

1Health Management and Economics Research Center, Department of Health Services Management, School of Management and Medical Information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran; 2Health Management and Economics Research Center, Isfahan University of Medical Sciences, Isfahan, Iran; 3Department of Health Services Management, School of Management and Medical Information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran

Correspondence: Yasamin Molavi Taleghani
Isfahan University of Medical Sciences, School of Management and Medical Informatics, Health Management and Economics Research Center, Hezar Jarib Street, Second Floor, Isfahan, Iran
Tel +98 912 7233347
Email [email protected]

Background: This study attempted to present a framework and appropriate techniques for implementing risk management (RM) in executive levels of healthcare organizations (HCOs) and grasping new future research opportunities in this field.
Methods: A scoping review was conducted of all English language studies, from January 2000 to October 2018 in the main bibliographic databases. Review selection and characterization were performed by two independent reviewers using pretested forms.
Results: Following a keyword search and an assessment of fit for this review, 37 studies were analyzed. Based on the findings and considering the ISO31000 model, a comprehensive yet simple framework of risk management is developed for the executive levels of HCOs. It includes five main phases: establishing the context, risk assessment, risk treatment, monitoring and review, and communication and consultation. A set of tools and techniques were also suggested for use at each phase. Also, the status of risk management in the executive levels of HCOs was determined based on the proposed framework.
Conclusion: The framework can be used as a training tool to guide in effective risk assessment as well as a tool to assess non-clinical risks of healthcare organizations. Managers of healthcare organizations who seek to ensure high quality should use a range of risk management methods and tools in their organizations, based on their need, and not assume that each tool is comprehensive.

Keywords: organization risk management, scoping review, risk analysis, health care, executive levels

Introduction

Given the World Health Report (2000), the significance of healthcare organizations(HCOs) has grown in global health discourse.1 However, in the last decade, HCOs have faced two contradictions: first, healthcare costs have increased due to population aging, the introduction of advanced technologies, and increased medical errors.2,3 On the other hand, HCOs have become more complicated due to such factors as efficient customers, biomedical developments, the complexity of services and an increasing number of healthcare users.2,3 Therefore, demand for healthcare is significantly higher than the human capacity and resources available in healthcare departments.4 Corresponding to these limits, three interventional approaches have been developed at various levels of the HCOs: (i) quality management, (ii) risk management, and (iii) patient safety.5

In particular, risk management (RM) is a process-oriented method providing a structured framework for identifying, assessing, and reducing risk at appropriate times for HCOs.6 RM approach protects healthcare providers against unfavorable incidents.7 This way, RM plays a major role in shrinking uncertainties and enhancing rich opportunities for different areas of the health system.8 Development of RM helps HCOs and providers to reduce damage due to the probable occurrence of defective processes through identifying error, rooting, and strategy development.9 Implementing RM in HCOs improves allocation of health resources,10 process management, decision-making, reduced organizational losses,11 patient safety,11 continuous quality improvement,2 customer satisfaction,2 organizational performance,12 hospital reputation,11 and better community creation.2

A general framework for RM needs to be identified before implementing the risk process. This framework determines the strategy of organization for identifying risk, risk assessment, and risk reduction.13 This strategy outlines how the RM process should be implemented in the organization. It determines the resources that are needed, the key roles and responsibilities for that, the ways risk needs to be identified. It shows how the decision-making process looks like while using those strategies.13 The available evidence suggests that despite the existence of a large number of RM techniques, a few of them have been employed so far in the HCOs.1416

Risk management is one of the emerging areas in management systems; there are several reports that have provided an overview of risk management inHCOs; however, it is difficult to find studies that have systematically synthesized risk management models at the executive levels of healthcare organizations.1719 This sector is far behind the rest of the industry in terms of using these techniques. Nowadays, there is a consensus in the healthcare sectors that the knowledge, experience, and expertise of other industries in RM can improve the quality of services provided in the healthcare sectors.3 Therefore, reviewing the selection of RM techniques seems indispensable. These instruments need to be tailored to the complexities of the healthcare system and the causes affecting incidents in this sector.20,21

The organizational structure of the healthcare system has been classified into executive, administrative and operational, each of which is exposed to some risks.22 This limited study aims to identify those risks that happen in executive levels. The study would not consider those risks that may happen in the operational levels of healthcare organizations and can be considered as a clinical risk. Mention should be made that the executive levels of healthcare organizations are the headquarters and deputies of the HCOs that provides counseling and control over healthcare delivery units.22 Therefore, the aim of this review is to scope published different organizational RM models, identify the strengths and weaknesses of each model, and this way, propose a framework for implementing RM in the executive levels of HCOs.

The applied purpose of this study was to integrate existing research on the various areas of RM cycle (risk identification, risk assessment, & risk management) and ultimately provide a centralized knowledge base for future research in the executive levels of HCOs. It is of note that the executive levels of HCOs are the headquarters and deputies of the HCOs that provides counseling and control over healthcare delivery units.

Methods

The methodological framework of the scope review described below was guided by such methodologies, which have been published elsewhere.23,24

Scoping Review Question

The first phase was represented by the definition of the scope of the study in compliance with the objectives and the underlying research hypotheses.

Based on preliminary studies, the research questions developed for scoping review are as follows:

  • RQ1: How are organizational risks identified and categorized within the executive levels of HCOs?
  • RQ2: What is the proposed framework for organizational risk management in the executive levels of HCOs? Also, what is the status of risk management in the executive levels of HCOs based on the proposed framework?
  • RQ3: What techniques and tools are available for implementing organizational risk management in the executive levels of HCOs?
  • Inclusion and Exclusion Criteria

    To obtain and include relevant and important documents to concentrate on, a series of inclusion and exclusion criteria should be defined. The selection of the studies was done according to the following inclusion criteria:

    (i) Studies on organizational RM and assessment techniques and framework in healthcare organizations or related organizations appropriate for imitation in the healthcare organization; (ii) articles in English; (iii) 2000 to October 2018.

    The following studies were excluded: (i) in the format of letters, editorials, news, professional commentaries, and reviews; (ii) without available abstracts or full text or references; (v) Models that cannot be imitated in healthcare organizations; (vi) Published in languages other than English.

    Identifying Locating Sources and Relevant Articles

    This study was conducted in October 2018 through consulting such databases as Pub Med, ISI, Emerald, Scopus, IEEE, Springer, ProQuest, Cochrane, and Wiley from 2000 to May 2018. The search strategy was the same for all the databases.

    The identification of the keywords related to the subjects and the objectives of the study are as follows: initially, keywords were identified by the authors through a brainstorming process. The identified keywords were refined and validated by a team composed of two university academic members and two healthcare managers. The search strategy was formulated using Boolean operators. The formula was searched in the field of title and abstract in online databases. The search strings used are shown in Table 1, a search for each research question was performed. Also, the search was repeated two times with the following search string. In addition, the references were retrieved from the studies included in the first iteration. The keywords of references that matched with the search keywords were chosen.

    Table 1 Search Strings for Research Questions and Studies

    Study Selection and Data Abstraction

    The two authors (YMT and MF) independently performed level 1 (titles and abstracts) and level 2 (full article texts) screening forms. All screening and extraction were completed in duplicate. Disagreements were discussed between the two reviewers and a third-party reviewer (R R) was contacted if disagreements could not be resolved. After independent reading of the full texts, the content analyzed and selected the articles that answer the respective research questions. Study quality was not assessed during the scoping review as the objective of a scoping review is to identify gaps in the literature and highlight future areas for systematic review.23,24 The required information extracted based on the research questions and placed in the designed templates.

    Results

    Three thousand five hundred and seventy-four studies were screened, excluded 761 duplicates, 1556 on title review, 1081 on abstract review and 144 in a full-text review. In total, leaving 37 papers (32 papers first iteration on the database and five studies from hand searching) search for critical appraisal. Table 2 shows the flowchart for the study selection.

    Table 2 Paper Selection Process

    Characteristics of Articles Reviewed

    Bibliographical information about the 36 articles included in this review can be obtained from Table 3.

    Table 3 Bibliographical Sources of the Studies Included in the Literature Review

    According to Table 3, 11 articles (14.3%) were used to answer the first research question, 30 articles (38.9%) were used to answer questions 2, and finally, 36 articles (46.8%) were used to answer research question 3. (Total papers >36 because each paper may be classified into two or more study types, or may address two or more review questions.) Also, it could be recognized that all but four articles were published in 2009 or later, this is due to the complexity of environment and type of services provided by organizations and, consequently, use of the RM and risk assessment process as a tool for reducing errors and incidents in recent years.

    As can be seen in Table 3, based on the setting of the studies, Europe had the most study with (59.5%) of the authors affiliated with European universities and institutions. Asia was the next one with (21.6%) of the studies, followed by America (13.5%), Oceania (2.7%), and Africa with 2.7%. Also, most of the studies examined in developed countries. Thus, at this point, we can already identify a need for more research into risk management in developing countries.

    As for design, 2(5.4%) studies were empirical quantitative, 5 (13.5%) empirical qualitative, 12 (32.4%) conceptual/theoretical and 18 (48.7%) mix method.

    How are Organizational Risks Identified and Categorized Within Executive Levels of Healthcare Organizations?

    Risk identification is usually a necessary condition for later risk management.25 Given dynamic and complex healthcare organizations, different risk sources can trigger hazardous situations, potentially harming the organization.36 It is therefore essential to consider as many risk sources as possible within a classification to help participants familiarize themselves with the given system and potential risk sources.36 Although the study strategy did not focus on risk types of healthcare organizations (see methods), the reviewed studies placed significant emphasis on identifying and discussing a variety of typical risks in similar organizations with healthcare organizations.

    According to the results of Simsekler et al, risk identification Framework (RID Framework) used to identify risks of the health organizations.36 The risk identification framework includes a spectrum of inputs (System familiarization), processes (Identification of risks), and outputs (Presentation of the risks) in its structure.36

    Results of the studies, a functional framework for identifying and classifying risks in executive levels of HCOs are presented in Table 4.

    Table 4 Identification and Classification of Risks in Executive Levels of Healthcare Organization

    According to Table 4, risk sources are classified into two categories (internal and external), and risk identification tools classified into two categories (retrospective-prospective and intra-organizational – inter-organizational).

    Which Organization RM Framework and Techniques are Used in Executive Levels of Healthcare Organizations?

    A stringent risk management process may enable executive levels of HCOs to cope with the risks presented in the previous section. Once risks have been identified, a number of techniques and actions can be selected to address them.

    Various models have been used by organizations to assess and manage risk, the results are which are shown in Table 5. Based on the findings in Table 5, the risk management framework that are applicable to the executive levels of HCOs are classified into basic models and combined models. In addition, risk management models are divided by cost, time, and complexity. The approaches of risk management models are also divided into qualitative or quantitative, systemic or individual, retrospective or retrospective, and holistic or partial.

    Table 5 Characteristics of Organization RM and Risk Analysis Techniques

    According to the studies’ results, a simple and comprehensive framework for RM in executive levels of HCOs was suggested. The proposed framework of the present study consists of five phases that its main phases are adapted from the ISO13000 framework. The following is a suggested framework and techniques that can be used to implement risk management processes in executive levels of HCOs. Finally, in Table 5 examines the extent to which risk management based on the key phases of the proposed framework is established in healthcare organizations.

    1. Establishing the context,
    2. Risk assessment (risk identification, risk analysis, and risk evaluation),
    3. Risk treatment (strategy determination, designing measures and decision-making, planning, and implementation),
    4. Communication and consultation, and
    5. Monitoring and reviews.

    In the following, RM framework and techniques in executive levels of HCOs for each organization were mentioned.

    Establishing the Context (Initiation and Preparations)

    The first phase in the risk management process is establishing the context. The context establishment primarily paves the way for the organizational nature of the company such as the project objective and management style or organization culture. In this step, issues such as healthcare organization background, who should conduct the RM process, Identify interested parties, formulate problems, set the objective(s) of RM and Select appropriate methods for RM are reviewed.43,59

    The organizational RM team should be multidisciplinary and comprised of various specializations, in particular, managers, process owner experts, and RM experts (consultants and facilitators).25,33 Also, the number of team members depends on the complexity of organizational issues.33,40,43

    Risk Assessment

    The second phase in the risk management process is risk assessment, which involves measuring or estimating the potential frequency of losses and the potential impact of a risk on the organizations' health care. Subsequently, the risks can be ranked according to its importance for the HCOs. In general, the following three steps (risk identification, risk analysis, and risk evaluation) proposed for risk assessment in executive levels of HCOs:

    Risk Identification

    Describing the Process and System Definition

    According to the results, there were several methods for outlining risky processes that executive levels of HCOs can use depending on their needs: Textual system description,8,41,53,59 activity breakdown structure (ABS),8 radar charts,34 flow charts,3,25,28,30,38,45,50,56,62 process diagrams,34,38,45,56,58 system diagram,8,34,62 integration definition (IDEF),35 and hierarchical task analysis Diagram (HTA) or task diagram,26,28,35,42,57,62 communication diagram,56,62 information diagram,35,56,62,63 organizational diagram,35,56,62,63 stakeholder diagrams,56 swim lane activity diagram,56 state transition diagram,56 sequence diagram,56 and data flow diagram.56

    In general, process description tools are divided into two categories of descriptive tools and process tools. Radar charts, also called Kiviat diagrams, were built in order to visualize initial and residual risks for each kind process.34 ABS is process-oriented instead of being product-oriented, moreover, this method lacks time dimension.8 Also, a task diagram is used for describing the hierarchy of operations and plans, system mapping for how data is transmitted through activities, Information diagrams for describing information hierarchies, organizational diagrams for describing organizational roles hierarchy and Communication diagrams for displaying information flows between individuals and Business processes and IDEF for linking between inputs and outputs in organizational activities and resources, and Sequence diagrams for interacting information between stakeholders.

    According to Cagliano et al, the flow chart included the name or code of both process phase and activity at issue, actors performing the activity; inputs (information, materials, preliminary actions, orders, etc.); a detailed description of operations required by the activity; duration and frequency; controls to monitor activity progress; tools necessary to perform both the activity and related controls and outputs (other activities, information, and data).8 Moreover, in Parand et al’s study, activities in flow chart classified based on action, retrieval, checking, selection and information, and communication.28 In general, as the describing the process be stronger, the results of the risk assessment can be more effective.

    According to Simsekler et al36 and Jun et al.56 Studies, specific types of diagrams were selected by stakeholders as more useful than others in identifying different sources of risks within the given system. In general, employees’ perception, the ease of use and usefulness are the main variables for choosing the most optimal system modeling tool.

    Risk Identification

    After drawing the process flowchart, at this stage, organizational risks or organizational process risks are determined. The applied frameworks for identifying risks in executive levels of HCOs presented in Table 4.

    Cause Identification

    Based on some risk assessment models, the effective causes and the root causes of the errors are identified at this stage. Based on the Eindhoven model, the classes of causes error classified into two main categories of latent errors (technical and organizational) and active errors (human errors and other factors).25 Furthermore, based on the results of some studies, the causes of errors classified in the Institutional context factors, organizational and management factors, work environment factors, team factors, communication factors, individual (staff) factors, training and education factors, equipment factors, task factors, and patient factors.35,36 In addition, based on the results of some studies, the Ishikawa cause-effect diagram can be used to determine the sources of errors.37,45,48

    Risk Analysis

    At this stage, it is possible to estimate the risk, qualitatively, semi-qualitatively or quantitatively according to the probability of the risk. The following steps considered for risk analysis in executive levels of HCOs.

    Risk Estimation (Severity and Consequences and Likelihood Estimation)

    At this stage, it is possible to risk estimation according to the probability and severity of risk. There are numerous qualitative, semi-quantitative and quantitative methods that try to estimate individual components of risk for a result to better reflect the reality.

    Using verbal descriptors (low, medium, or high),26 risk weights,25,34,38,49,59,61 encoding,30,40,52,60,61 scoring tables,2527,30,32,37 Bayesian methods,46 Monte Carlo method,46,60 and historical data,49 suggested for estimating the severity and probability of risk in executive levels of HCOs.

    In quantitative risk estimation methods (Monte Carlo and Bayesian), activities find a probabilistic form and a distribution function is specified for them.46,60 In qualitative risk estimation methods, risks are prioritized based on their potential impacts on project objectives based on qualitative variables. Qualitative methods of risk estimation can either lead to further analysis in quantitative risk estimation or directly to risk response planning.30,60

    Interview with experts,32,53 questionnaire design,32,61 Delphi method or expert,60 and focus group,38,44,46,49-51,53 identified an applied method for risk estimation in executive levels of HCOs.

    Risk Presentation

    Present-estimated risks based on risk presentation formats, included a single number index (e.g. 1/100,000),27,37 use failure space vs success space,54 fuzzy numbers scales,30,32,40,41,52,61 tables (e.g. sizes or bands of fatalities are 1–10, 11–100, and 101–1000),30,40 risk matrix,25,33,43,52,53,57 graphs or diagrams (e.g. Frequency-Number (F-N) curve),35,46 and maps (e.g. risk contour plot).45

    In sensitivity analysis, the management index (Risk Index x Sensitivity) provided further ranking for those risks that have equivalent Risk Indexes. Given its scope, this analysis may not necessarily constitute an integrated step of risk analysis.49

    Conclusion

    Synthesize information about the main risk elements included risks and their causes and contributing causes, frequency or probability, consequences due to risk, and estimated risks.49

    Risk Evaluation

    Risk evaluation is the process of comparing the results of the risk analysis with the risk evaluation criteria defined during the context establishment to determine whether the cyber-risks are acceptable. In this step, the following steps considered for risk evaluation in executive levels of HCOs.

    Select Risk Evaluation Criteria

    There was a wide range of qualitative and quantitative risk criteria or standards for evaluation of various types of errors in executive levels of HCOs. Selection of risk criteria may also depend on the results of the risk analysis and how risks are estimated.60

    Compare Estimated Risks Against the Risk Criteria and Prioritize or Rank Risks

    This step concerned with making decisions about prioritization and comparison of risks to be managed, based on the outcomes of risk analysis.27

    A simple method for risk filtering was a Pareto analysis.26,30,58,60 Moreover, in some studies, decision tree,25,28,49,57 priority matrix,25,30,35 criticality matrix,34,44 Criticality scale,34,38,49,60 and risk prioritization grid used to determine acceptable and unacceptable risks.27 Furthermore, simple additive weighting (SAW),32 and hazard totem pole (HTP)60 methods can be used as practical and quantitative methods for risk evaluation. SAW was a simple and most applicable multi-attribute decision method which is known as a weighted linear combination or scoring technique.32

    Risk Treatment

    This phase involved defining and implementing actions for mitigating the determined risk level and verifying that the residual risk level is acceptable.27

    Determine Organization RM Strategies

    The four common organization RM strategies options:

    1. Avoid: elimination involves elimination of risks at the source.
    2. Reduce: The strategy of risk reduction involves reduction, but not a complete elimination, of the frequency of occurrence of undesirable risks and/or the severity of their consequences.53,60

    These comprise two fundamental approaches to risk reduction, which were:

    • Prevention
    • Mitigation: Reduce the occurrence probability of the risk or the impact of the risk.
      1. SHARE (spread or transfers): sharing the risk to another entity and/or function. Risk sharing is carried out in different ways, including risk sharing by insurance and contract, risk transfer and physical transfer.
      2. Accept: Risk can be retained in cases where it cannot be avoided or transferred.25,44,45,53,60

    Moreover, theory of problem-solving by an inventive method,25 Generating Options for Active Risk Control (GO-ARC) Technique64 and dynamic systems development method (DSDM)50 used to redesign the process and improve strategies.

    In the GO-ARC Technique, risk control options are divided into 5 categories (elimination, design controls, administrative controls, detection/situational awareness, and preparedness). The first three consist of the 3-tiered hierarchy of risk controls. The remaining two, detection/situational awareness and preparedness help users consider risk controls to reduce the severity of harm or prevent harm in the midst of an on-going systems breakdown; they are aimed at promoting resilience, as opposed to focusing solely on preventing systems breakdowns in the first place. In general, GO-ARC improves the trend of producing risk control options. Use of the Generating Options for Active Risk Control (GO-ARC) Technique can lead to more robust risk control options.

    On the other hand, the DSDM framework is complicated to become a general framework for solving task problems. At DSDM, the primary effort is to provide software that is good enough to meet the needs of the business and that it can progress to the next iteration.50

    Additionally, the SWOT matrix with four strategy areas, SO (maxi-maxi) and ST (maxi-mini) and WO (mini-maxi) and WT (mini-mini), was used to determine strategies and corrective actions.31

    RM Measures and Decision-Making

    RM strategies and measures were often difficult to compare and evaluate executive levels of HCOs. The best decision is the one that yields the greatest expected value. The interventions prioritized according to two criteria of their ability to reduce the root causes (interventional power) and perception of their implementation based on what is anticipated (reliability of intervention).26,30

    The best performance measures can be selected based on criteria such as safety, profitability, quality, efficiency, effectiveness, time, cost, available resources, performance, environmental conditions, and satisfaction.41,42,45,46,59 In one study, AHP/ANP and BOCR (benefits, opportunities, costs, and risks) used to select the best RM strategies.41

    Planning and Implementation

    Finally, a plan also defined risk ownership, roles and responsibilities, and time frames to implement mitigation strategies.45 Risk governance structure was a useful tool for risk assessment planning. In this method, the roles and responsibilities of each employee determined in the RM plans.39,40,45 Moreover, using the pilot study method43,59 and simulation,41,49 suggested before the implementation in a wide range.

    These steps are typically performed as iterative cycles that controlled and triggered by two continuously running activities: risk review and monitoring, communication, and consultation.

    Communication and Consultation

    Communication and consultation with internal and external stakeholders needed to keep them informed of process outputs and let them provide inputs.27

    Risk-related information should be shared based on appropriate access levels in the exchange organization or between decision-makers and other stakeholders. These should address the issues related to risk itself, its causes, its consequences (if there is information about them), and the measures taken to deal with it.

    Communication and consulting with project stakeholders can be a key factor in a favorable execution of risk management and in achieving better results. In practice, regular reporting is of important components of communication that helps senior managers identify the risks they are faced with. Summary reports prepared from risks, in fact reflect the status of the responding guidelines and the trend index of risk occurrence.59

    Work sessions,29,59 intranet-based calendars,59 reports and gatherings,59 wiki page,45 and PMBOOK software,46 are suggested as tools for information exchange in executive levels of HCOs.

    Monitoring and Review: (Re-Assessment – a Continuous and Cyclic Process)

    Effective risk management requires a reporting and reviewing structure in order to ensure that risks are effectively identified and evaluated and responses and controls are in a timely manner. In this phase, policies and following of standards should be regularly verified and the performance of standards should be reviewed to identify improvement opportunities.27

    Various methods such as risk compliance readiness template,45 risk project update template,45 data management system,60 variance analysis,46 risk reassessment,46 Wiki page as collaborative workspace,45 control chart,43 trend analysis,46 risk auditing,39,46 visual process control,43 and communication plan43 recognized to monitor and evaluate the effective and efficient RM cycle in executive levels of HCOs.

    By conducting continuous monitoring and reviewing of risk, it is ensured that new risks are being identified and managed, and executive programs are effectively implemented and developed.46

    Discussion

    Given different and dynamic nature of organizations, various frameworks and techniques are used in managing and accessing organization risks. Therefore, recognizing organization RM framework is an important step in RM in executive levels of HCOs. In this study, based on a review of studies, frameworks and tools that can be used to implement organizational risk management in the executive level of HCOs are proposed.

    According to the first question of this study, healthcare organizations may be faced with risks that may prevent the mission and achievement of the organization’s objectives, so at the first step of risk management, risk resources should be identified with optimal tools.17 In the present study, using an innovative approach, a framework for identifying and classifying risks in the executive levels of HCOs was proposed. The proposed framework included three steps of input, process, and output.

    Input phases considered a spectrum of inputs to help increase understanding of the system, and awareness of potential organization risks that can occur in complex and changeable healthcare systems.36 Input phases consist of (Risk Sources,8,36 Nature of Hazards,36 and Time).36 At the process stage, the tools that can be used as intra- or inter-organization and retrospective-prospective in the executive levels of healthcare organizations are determined.55 Finally, in the presence of the risk stage (output stage), the identified risks were clearly registered in executive levels of HCOs.8

    Using this framework is a helpful guide for managers to identify potential error in the executive levels of HCOs. Based on the results of the study by Pott et al57 and Similker et al,17 different approaches should be used to identify risks in organizations, and data from different resources should be integrated to gain a general view into the risks of a system.

    We have no standard answer as to which one of the risk identification tools is a more optimal tool. Each tool is used to identify a range of risks, so the best approach to identify all risks is to integrate retrospective and prospective analysis to understand a broader scope of the risks.

    Based on the results of the studies, organizational risks,8,26,31,45,59 technological supports,8,31,34,40,45,60 and information and communication,8,31,34,40,55,59 were identified as the most important resources of risk in most studies, so treatment of these risks is of high importance in the executive levels of HCOs.

    In today’s world, when being faced with healthcare organization risks, managers have realized the need to develop a risk management framework at the organization level. According to the second and third questions of this study provides a state of the art based on the review of studies and it tried to propose a framework for risk management and techniques applicable to each of the stages of risk management and risk assessment in executive levels of HCOs. The term “framework” has a broader scope than the term “technique.” The risk management framework includes guidelines for analyzing, assessing, and managing risks in healthcare organizations. In contrast, management, and risk assessment techniques considered as analytical tools for analyzing data and risk information.

    In general, the risk management framework has required stability, but there is no strong and complete risk assessment and risk management techniques that can be applied completely for risk management in organizations, and managers of healthcare organizations must make the decisions necessary to determine the optimal tool for risk management and assessment at each time and based on specific conditions and position of the organization. Therefore, Table 5 presents limitations, strengths and weaknesses and factors influencing the selection of each of the models for risk management and risk assessment in executive levels of HCOs. Therefore, the content of this table can help risk analysts, healthcare managers and other stakeholders to make rational decisions about identifying risk management and risk assessment models in executive levels of HCOs.

    According to the results of the studies, there was a wide range of well-known and successful tools for single and combined risk assessment and a hierarchy of risk analysis models suggested for executive levels of HCOs.

    Hierarchy of risk analysis and risk assessment models divided:

    High-level tools: At this level, risk assessment tools cover a wide range of risk scenarios and provide various information for the organization based on risk scenarios. However, such tools should not be used when the details need to be emphasized in risk assessment. Some risk assessment tools employed at this level are All the combined models presented in Table 5 for analysis and risk assessment,30,35,38,40,42,43,45,50,52 Six Sigma,43,45 IRMAS,59 CREA (Clinical Risk and Error Analysis).35

    Mid-level tools: Implementing risk assessment tools at this level makes it possible to provide the modest information and details for the organization considering risk scenarios. Some risk assessment tools employed at this level are Health failure mode and effect analysis (HFMEA),25,42,50 HFMEA/FMEA/FMECA,8,25,26,28,30,37,38,49 root cause analysis (RCA),38,43,50 bow-tie model,48,51 hazard and operability analysis (HAZOP).35

    Low-level tools: At this level, risk assessment tools evaluate the limited range of risk scenarios, but with more details for the organization. Some risk assessment tools employed at this level are: Preliminary risk analysis method (PRA),34 fault tree analysis (FTA),54 change risk assessment model (CRAMS),46 change analysis (CHA),46 human reliability assessment (HRA),8 Pareto analysis (PA),26,30 relative ranking/risk indexing (RI),32,60 5 whys technique,8,36 hazard checklists (HCl),35 change analysis (CA),28 strategic risk analysis (SRA).31

    Optimal implementation of the risk management process is nothing but the adoption of the most appropriate techniques and tools available in each phase. However, there is no strong and complete risk assessment and risk management techniques that can be applied completely for risk management in organizations, and managers of healthcare organizations must make the decisions necessary to determine the optimal tool for risk management and assessment at each time and based on scope of risk analysis, legal requirements, results/information needed data, resources and time available, complexity and size of risk analysis and type of activity or system and concerning issues. As a general rule, the best risk management tool is to overcome the participants’ mental judgment.

    Most of the models extracted from the results of the study were somewhat similar and presented the same components. The three main factors that were found in all risk management models included measurement, management, and monitoring. Therefore, based on the results of the studies and the nature of healthcare organizations, the risk management process had one primary phase and four main phases. In the primary phase, the objectives and prerequisites for risk management are set out for execution. The main phases are as follows: Risk assessment (identifying potential risks, determining the likelihood and consequence of the identified risk and determining the level of the risk), risk treatment (how to reduce the impact of unacceptable risks and selecting appropriate responses to them), monitoring and reviewing (effectiveness of measures) and the latest activity of the process of communication and consultation with the stakeholders on the trend have been carried out.

    The proposed framework of this study is very similar to the iso13000 framework, with the difference that more details are provided in the framework of the present study. The ISO13000 approach describes the organization’s risk management in a comprehensive, strategic, and holistic way.45

    Also, the model developed in the present study has several specific features compared with the previous models: 1) In the present research it was tried that the research literature be integrated in the field of risk management and provide a framework that is more comprehensive; 2) According to the search strategy, all risk management frameworks of healthcare organizations and organizations adaptable with healthcare organizations were examined and there was no particular dependence on the specific industry and from this perspective, they have more advantages compared to some frameworks that were established regarding a specific industry; 3) The proposed framework is provided based on the internal and external flows dominant on healthcare organization. Managers of healthcare organizations today need a structured and coherent approach to identify, analyze, and manage risk across a range of intra- and inter-organizational activities; 4) With the establishment of the proposed model in the organization, the basic assumptions dominant on healthcare organizations are examined in specific time periods and, if necessary, continuous improvement in healthcare organizations is done in a dynamic cycle.

    Regarding the status of healthcare organizations in establishing each of the main phases of the proposed risk management framework, studies have identified and evaluated the risk, and the treatment phase and risk monitoring were neglected in most studies. However, risk management should be done throughout the life of the organization. New risks need to be identified and managed at every stage of the organization’s life. Also, based on Table 5, most studies were not done at the phase of risk assessment, process mapping, and cause identification. While many system mapping approaches have been widely used in various industries, healthcare organizations have only used a limited number of them to process mapping.62 Each process mapping tool has a specific application, and managers and professionals should use the most useful of them to identify sources of risk in healthcare organizations. The most important phase, guiding the risk management process, and determines the main policies in risk management is the phase of planning and setting objectives, which is done incompletely in most studies. Risk managers should pay great attention to risk planning; obviously, if this is not done in a fully transparent manner, the execution of risk management will be subject to some uncertainty.43,46

    Based on the results of Table 5, in most studies (89.6% of studies), risk management attitude was prospective and in few studies, each of prospective and retrospective risk management approaches was emphasized. Whereas, based on the results of the Kessele-Habraken et al study, the integration of prospective and retrospective analysis is important in improving the safety and optimization of organizational processes.58

    As we proposed, information about incidents and their retrospectively reported frequencies could be used as a reference point in the prospective analyses, which might facilitate frontline staff in the risk assessment. Conversely, prospectively developed failure scenarios could be used as guideline for retrospective.

    Further Research Avenues and Limits

    In this study, a framework for the execution of risk management in the executive levels of HCOs was proposed. Like any other management framework, successful implementation of the organization RM framework in executive levels of HCOs necessitate organizational commitment, establishing a stimulating culture, accurate planning, stakeholder engagement, strong and effective management, and use of available resources to implement the stages. Based on the results, it can be suggested that studies of risk management are increasing over time; however, there are still new cases that need further investigation and researches, some of which are mentioned below.

    1. Studies evaluating the effectiveness of risk management frameworks were very scarce and the effectiveness of risk management models should be examined in the future.
    2. The amount of outcome studies was not significant with respect to the investigated period (2000–2018). The outcome of most studies was also partial and lacks the necessary comprehensiveness. In most studies, the identification and assessment of risk were dealt with, and the phases of risk treatment and monitoring was neglected. Future studies, therefore, need to be implemented with a holistic view of the risk management process in healthcare organizations.
    3. In most studies, the sample size was very small, and risk management was performed at a micro level in the healthcare organization and organizations adaptable with the terms of healthcare. Therefore, the risk management needs to become dominant in a more comprehensive way and in larger-scales in the healthcare organization.
    4. Based on the results, various tools have been identified to achieve the risk management framework at different phases. The variety of the materials collected, together with the limited evidence for each topic, make it difficult to come to general conclusions, so it is necessary to conduct a cost-benefit analysis of risk assessment techniques.
    5. In this study, risk sources have been identified theoretically and for staff areas of healthcare organizations and some risks may not have been identified, although maybe a significant threat to the health system. Therefore, we cannot claim that this framework can be extended to other organizations in the health system.
    6. The volumes of the most studies of risk management in healthcare organizations are related to risk assessment, so it is recommended that all future phases of risk management in healthcare organizations be established.
    7. For some phases of organization risk management, there were only conceptual studies; therefore, a feasibility study is needed to effectively implement various phases of RM in organizations.
    8. Development of the organization RM framework for other areas of healthcare, development of advanced technological solutions to facilitate risk assessment, development of tools or criteria for effective and efficient implementation of organization RM frameworks, managers’ perceptions of organization RM frameworks are factors which should be considered for further research.

    One limitation of this study was that the number of findings in the systemic review was dependent on the selection of keywords and input/output criteria. Therefore, more models can be extracted for organizational risk management. Also, non-English studies were not included and there may, therefore, be a bias towards inclusion of studies performed in English-speaking countries. In addition, articles were exclusively selected from journals, hence, other parts of literature, such as books, book sections, and gray literature were excluded from the process as journal articles are readily available in journal databases and are usually used as a mean of scientific communication.

    Despite these limitations, this study has several strengths. First, all models of risk management and evaluation in healthcare organizations and organizations that could be modeled for the executive levels of the HCOs were examined in this study. Second, this paper contributes to the field of risk management research in healthcare. Third, the tools and techniques for risk assessment and management that are applicable to staff areas of healthcare organizations are mentioned.

    Conclusion

    Based on the findings and considering the ISO31000 model, a comprehensive yet simple framework for risk management is developed for the executive levels of HCOs. It includes five main phases: establishing the context, risk assessment (risk identification, risk analysis, and risk evaluation), risk treatment (strategy determination, designing corrective actions, planning, and implementation), Monitoring, and review, and communication and consultation.

    Tools and techniques were also suggested for use at each phase of the proposed risk management framework. These techniques have been selected to best apply to non-clinical risks in healthcare organizations. Managers of healthcare organizations who seek to ensure high quality should use a range of risk management methods and tools in their organizations, based on their need, and not assume that each tool are comprehensive.

    Acknowledgments

    We would like to thank all the staff members who assisted with our research.

    Disclosure

    The authors report no conflicts of interest in this work.

    References

    1. Sekhri Feachem N, Afshar A, Pruett C, Avanceña AL. Mapping healthcare systems: a policy relevant analytic tool. Int Health. 2017;9(4):252–262. doi:10.1093/inthealth/ihx005

    2. Crema M, Verbano C. Investigating the connections between health lean management and clinical risk management: insights from a systematic literature review. Int J Health Care Qual Assur. 2015;28(8):791–811. doi:10.1108/IJHCQA-03-2015-0029

    3. Cagliano AC, Grimaldi S, Rafele C. Choosing project risk management techniques. A theoretical framework. J Risk Res. 2015;18(2):232–248. doi:10.1080/13669877.2014.896398

    4. Alhassan RK, Nketiah-amponsah E, Spieker N, et al. Effect of community engagement interventions on patient safety and risk reduction efforts in primary health facilities: evidence from Ghana. PLoS One. 2015;10(11):1–20. doi:10.1371/journal.pone.0142389

    5. Franca M. Quality, risk management and patient safety: the challenge of effective integration. World Hosp Health Serv. 2008;44(4):21.

    6. Levett JM, Fasone JM, Smith AL, et al. Enterprise Risk Management in Healthcare. Surgical Patient Care. Springer; 2017:67–86.

    7. Rubbio I, Bruccoleri M, Perrone G. Introducing “Healthcare Resilience” in Clinical Risk Management. Italy: University of Palermo; 2016.

    8. Cagliano AC, Grimaldi S, Rafele C. A systemic methodology for risk management in healthcare sector. Saf Sci. 2011;49(5):695–708. doi:10.1016/j.ssci.2011.01.006

    9. El-jardali F, Jaafar M, Dimassi H, Jamal D, Hamdan R. The current state of patient safety culture in Lebanese hospitals: a study at baseline. Int J Qual Health Care. 2010;22(5):386–395. doi:10.1093/intqhc/mzq047

    10. Santelices E, Muñiz P, Arriagada L, Delgado M, Rojas J. Adjusted clinical groups as a risk assessment model for healthcare resource allocation. Rev Med Chil. 2014;142(2):153–160. doi:10.4067/S0034-98872014000200002

    11. Møller AH, Hansen L, Jensen MS, Ehlers LH. A cost-effectiveness analysis of reducing ventilator-associated pneumonia at a Danish ICU with ventilator bundle. J Med Econ. 2012;15(2):285–292. doi:10.3111/13696998.2011.647175

    12. Thomya W, Saenchaiyathon K. The effects of organizational culture and enterprise risk management on organizational performance: A conceptual framework. Int Business Manage. 2015;9(2):158–163.

    13. Standardization IOF. Information technology–Security techniques–Information security management systems–requirements(First Edition). British BS ISO/IEC27005; 2005; 34.

    14. Lau CY. Quality improvement tools and processes. Neurosurg Clin N Am. 2015;26(2):177–187. doi:10.1016/j.nec.2014.11.016

    15. Eugene Fibuch M, Arif Ahmed B. The role of failure mode and effects analysis in health care. Physician Exec. 2014;40(4):28.

    16. Qazi A, Quigley J, Dickson A, editors. Supply chain risk management: systematic literature review and a conceptual framework for capturing interdependencies between risks. Industrial Engineering and Operations Management (IEOM), 2015 International Conference on; 2015: IEEE.

    17. Simsekler M, Card AJ, Ward JR, Clarkson PJ. Trust-level risk identification guidance in the NHS East of England. Int J Risk Saf Med. 2015;27(2):67–76. doi:10.3233/JRS-150651

    18. Ward J, Clarkson P, Buckle P, Berman J, Lim R, Jun G. Prospective Hazard Analysis: Tailoring Prospective Methods to a Healthcare Context. UK: University of Cambridge University of Surrey; 2010.

    19. Card AJ, Ward JR, Clarkson PJ. Trust‐level risk evaluation and risk control guidance in the NHS east of England. Risk Analysis. 2014;34(8):1469–1481. doi:10.1111/risa.2014.34.issue-8

    20. Card AJ. The Active Risk Control (ARC) toolkit: a new approach to designing risk control interventions. J Healthcare Risk Manage. 2014;33(4):5–14. doi:10.1002/jhrm.21137

    21. Card AJ, Klein VR. A new frontier in healthcare risk management: working to reduce avoidable patient suffering. J Healthcare Risk Manage. 2016;35(3):31–37. doi:10.1002/jhrm.21207

    22. McConnell CR. The Effective Health Care Supervisor. Jones & Bartlett Publishers; 2011.

    23. Arksey H, O’malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. doi:10.1080/1364557032000119616

    24. Levac D, Colquhoun H, O’brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):69. doi:10.1186/1748-5908-5-69

    25. Taleghani YM, Rezaei F, Sheikhbardsiri H. Risk assessment of the emergency processes: healthcare failure mode and effect analysis. World j Emerg Med. 2016;7(2):97–105. doi:10.5847/wjem.j.1920-8642.2016.02.003

    26. Gervais B, D’arcy DM. Quality risk analysis in a cGMP environment: multiple models for comprehensive failure mode identification during the computer system lifecycle. Drug Dev Ind Pharm. 2014;40(1):46–60. doi:10.3109/03639045.2012.744417

    27. Bernardini G, Paganelli F, Manetti M, Fantechi A, Iadanza E. SYRMA: a tool for a system approach to risk management in mission critical systems. Int J Business Inf Syst. 2013;13(1):21–44. doi:10.1504/IJBIS.2013.054166

    28. Parand A, Faiella G, Franklin BD, et al. A prospective risk assessment of informal carers’ medication administration errors within the domiciliary setting. Ergonomics. 2018;61(1):104–121. doi:10.1080/00140139.2017.1330491

    29. Sendlhofer G, Brunner G, Tax C, et al. Systematic implementation of clinical risk management in a large university hospital: the impact of risk managers. Wien Klin Wochenschr. 2015;127(1–2):1–11. doi:10.1007/s00508-014-0620-7

    30. Lopez F, Bartolo CD, Piazza T, et al. A quality risk management model approach for cell therapy manufacturing. Risk Anal. 2010;30(12):1857–1871. doi:10.1111/j.1539-6924.2010.01465.x

    31. Emblemsvåg J, Endre Kjølstad L. Strategic risk analysis–a field version. Manage Decision. 2002;40(9):842–852. doi:10.1108/00251740210441063

    32. Jaberidoost M, Olfat L, Hosseini A, et al. Pharmaceutical supply chain risk assessment in Iran using analytic hierarchy process (AHP) and simple additive weighting (SAW) methods. J Pharm Policy Pract. 2015;8(1):9. doi:10.1186/s40545-015-0029-3

    33. Wierenga PC, Lie-a-huen L, de Rooij SE, Klazinga NS, Guchelaar H-J, Smorenburg SM. Application of the bow-tie model in medication safety risk analysis. Drug Saf. 2009;32(8):663–673. doi:10.2165/00002018-200932080-00005

    34. Niel-lainé J, Martelli N, Bonan B, et al. Interest of the preliminary risk analysis method in a central sterile supply department. BMJ Qual Saf. 2011;20:698–773. doi:10.1136/bmjqs.2010.048074

    35. Trucco P, Cavallin M. A quantitative approach to clinical risk assessment: the CREA method. Saf Sci. 2006;44(6):491–513. doi:10.1016/j.ssci.2006.01.003

    36. Simsekler ME, Ward JR, Clarkson PJ. Design for patient safety: a systems-based risk identification framework. Ergonomics. 2018;61(8):1–19. doi:10.1080/00140139.2018.1437224

    37. Bonnabry P, Cingria L, Sadeghipour F, Ing H, Fonzo-christe C, Pfister R. Use of a systematic risk analysis method to improve safety in the production of paediatric parenteral nutrition solutions. BMJ Qual Saf. 2005;14(2):93–98. doi:10.1136/qshc.2003.007914

    38. Rezaei F, Yarmohammadian MH, Haghshenas A, Fallah A, Ferdosi M. Revised risk priority number in failure mode and effects analysis model from the perspective of healthcare system. Int J Prev Med. 2018;9.

    39. Domański J. Risk categories and risk management processes in nonprofit organizations. Found Manage. 2016;8(1):227–242. doi:10.1515/fman-2016-0018

    40. Ramkumar M. A modified ANP and fuzzy inference system based approach for risk assessment of in-house and third party e-procurement systems. Strategic Outsourcing. 2016;9(2):159–188. doi:10.1108/SO-12-2015-0030

    41. Beauchamp-Akatova E, Curran R. From initial risk assessments to system risk management. J Model Manage. 2013;8(3):262–289. doi:10.1108/JM2-01-2011-0008

    42. Faiella G, Parand A, Franklin BD, et al. Expanding healthcare failure mode and effect analysis: a composite proactive risk analysis approach. Reliab Eng Sys Safety. 2018;169:117–126. doi:10.1016/j.ress.2017.08.003

    43. Usman Tariq M. A Six Sigma based risk management framework for handling undesired effects associated with delays in project completion. Int J Lean Six Sigma. 2013;4(3):265–279. doi:10.1108/IJLSS-05-2013-0028

    44. Famiyeh S, Adaku E, Kissi-mensah L, Amoatey CT. Risk management for a tailings re-mining project in Ghana. Int J Managing Proj Business. 2015;8(2):241–255. doi:10.1108/IJMPB-04-2014-0033

    45. Choo BS-Y, Goh JC-L. Pragmatic adaptation of the ISO 31000: 2009 enterprise risk management framework in a high-tech organization using Six Sigma. Int J Account Inf Manage. 2015;23(4):364–382. doi:10.1108/IJAIM-12-2014-0079

    46. Apostolopoulos C, Halikias G, Maroukian K, Tsaramirsis G. Facilitating organisational decision making: a change risk assessment model case study. J Model Manage. 2016;11(2):694–721. doi:10.1108/JM2-05-2014-0035

    47. Delcea C, Ioana-alexandra B. Fostering risk management in healthcare units using grey systems theory. Grey Syst. 2016;6(2):216–232. doi:10.1108/GS-12-2015-0078

    48. Abdi Z, Ravaghi H, Abbasi M, Delgoshaei B, Esfandiari S. Application of Bow-tie methodology to improve patient safety. Int J Health Care Qual Assur. 2016;29(4):425–440. doi:10.1108/IJHCQA-10-2015-0121

    49. Chu K-C, Hung L-P. Adaptive failure identification for healthcare risk analysis and its application on E-healthcare. J Appl Math. 2014;2014:1–17. doi:10.1155/2014/865241

    50. Prijatelj V, Rajkovič V, Šušteršič O. A model for risk assessment in health care using a health care failure method and effect analysis. Slovenian J Public Health. 2013;52(4):316–331. doi:10.2478/sjph-2013-0032

    51. Kerckhoffs MC, van der Sluijs AF, Binnekade JM, Dongelmans DA. Improving patient safety in the ICU by prospective identification of missing safety barriers using the bow-tie prospective risk analysis model. J Patient Saf. 2013;9(3):154–159. doi:10.1097/PTS.0b013e318288a476

    52. Vahidnia S, Tanrıöver ÖÖ, Askerzade I. An early phase software project risk assessment support method for emergent software organizations. Int J Advan Comp Sci Appl. 2017;8(5):105–118. doi:10.14569/IJACSA.2017.080514

    53. Leung F, Isaacs F. Risk management in public sector research: approach and lessons learned at a national research organization. R&d Manage. 2008;38(5):510–519. doi:10.1111/radm.2008.38.issue-5

    54. Zeng Y, Skibniewski MJ. Risk assessment for enterprise resource planning (ERP) system implementations: a fault tree analysis approach. Enterprise Inf Syst. 2013;7(3):332–353. doi:10.1080/17517575.2012.690049

    55. Simsekler ME, Card AJ, Ruggeri K, Ward JR, Clarkson PJ. A comparison of the methods used to support risk identification for patient safety in one UK NHS foundation trust. Clin Risk. 2015;21(2–3):37–46. doi:10.1177/1356262215580224

    56. Jun GT, Ward J, Clarkson PJ. Systems modelling approaches to the design of safe healthcare delivery: ease of use and usefulness perceived by healthcare workers. Ergonomics. 2010;53(7):829–847. doi:10.1080/00140139.2010.489653

    57. Potts HW, Anderson JE, Colligan L, Leach P, Davis S, Berman J. Assessing the validity of prospective hazard analysis methods: a comparison of two techniques. BMC Health Serv Res. 2014;14(1):41. doi:10.1186/1472-6963-14-41

    58. Kessels-Habraken M, Van der Schaaf T, De Jonge J, Rutte C, Kerkvliet K. Integration of prospective and retrospective methods for risk analysis in hospitals. Int J Qual Health Care. 2009;21(6):427–432. doi:10.1093/intqhc/mzp043

    59. Kayis B, Zhou M, Savci S, et al. IRMAS–development of a risk management tool for collaborative multi-site, multi-partner new product development projects. J Manufactur Technol Manage. 2007;18(4):387–414. doi:10.1108/17410380710743770

    60. Tummala R, Schoenherr T. Assessing and managing risks using the supply chain risk management process (SCRMP). Supply Chain Manage. 2011;16(6):474–483. doi:10.1108/13598541111171165

    61. Rathore R, Thakkar JJ, Jha JK. A quantitative risk assessment methodology and evaluation of food supply chain. Int J Logistics Manage. 2017;28(4):1272–1293. doi:10.1108/IJLM-08-2016-0198

    62. Simsekler MCE, Ward JR, Clarkson PJ. Evaluation of system mapping approaches in identifying patient safety risks. Int J Qual Health Care. 2018;30(3):227–233. doi:10.1093/intqhc/mzx176

    63. Kumar S,J, Himes K,P, Kritzer C. Risk assessment and operational approaches to managing risk in global supply chains. J Manufactur Technol Manage. 2014;25(6):873–890. doi:10.1108/JMTM-04-2012-0044

    64. Card AJ, Simsekler MCE, Clark M, Ward JR, Clarkson PJ. Use of the Generating Options for Active Risk Control (GO-ARC) technique can lead to more robust risk control options. Int J Risk Saf Med. 2014;26(4):199–211. doi:10.3233/JRS-140636

    Creative Commons License © 2020 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.