CGK4PM: Towards a Methodology for the Systematic Generation of Clinical Guideline Process Models and the Utilization of Conformance Checking

Abstract

In the context of improving clinical treatments and certifying clinics, guideline-compliant care has become more important. However, verifying the compliance of treatment procedures with Clinical Guidelines remains difficult, as guidelines are mostly available in non-computer interpretable form and previous computer-interpretable approaches neglect the process perspective with its potential to gain medical insight. In this paper, we present our transformation framework CGK4PM, which addresses the procedural nature of treatment processes and which guides the transformation of clinical explicit and implicit guideline knowledge into process models. The procedural representation enables the use of process mining techniques such as conformance checking to verify guideline compliance and the opportunity to gain insights from complex clinical treatment processes. In collaboration with physicians from Münster University Hospital, the practical applicability of the framework is demonstrated in a case study by transforming the guideline for the treatment of malignant melanoma. The case study findings demonstrate the need for structured and guided transformation and highlight the difficulties in developing a guideline-based process model.

1. Introduction

Clinical Guidelines (CGs) are systematically developed statements that reflect the current state of medical knowledge to support physicians and patients in the decision-making process for appropriate medical care in specific clinical situations [1]. They may include, e.g., precise instructions about what diagnostic or screening examinations to conduct and when, or when to recommend what treatment, according to the state of the science. Thus, CGs are an important tool for ensuring appropriate treatment and supporting the introduction of new scientific evidence into clinical care [2,3]. However, CGs are generally published only in PDF, print, or searchable web page formats, and not in computer-interpretable formats.

In order to make use of guideline knowledge in medical information systems, computer-interpretable Guidelines (CIGs) have been developed [4]. This transformation enables intelligent processing, e.g., to build decision support systems through computer-aided reasoning and execution of these formalized models [5]. Thus, this could support physicians in their daily clinical treatment, avoid unintended guideline-inconsistent treatments [4] and enable knowledge gain through retrospective analyses of deviations. Although many CIG approaches were presented, no holistic standard for representation and verification of guideline compliance in clinical treatment has yet become established.

With the increased use of clinical pathways and awareness of the complexity of clinical treatment processes, the process perspective is gaining medical importance. This procedural perspective is explicitly addressed by process mining, an emerging process-oriented research field that deals with the data-driven representation and analysis of processes and bridges the gap between business process modeling (BPM) and data-mining [6]. The basis for the application of process mining are event logs. These contain data about real activities performed and can be used to implement different types of process mining techniques. One type is conformance checking, which is the focus of this paper. Here, real process executions are compared with an a priori model and thus a comparison of reality and model is made [7]. In the context of CGs, conformance checking can be used to verify the compliance of clinical treatment processes to guidelines [8].

The procedural nature of clinical treatments and guidelines makes the application of process mining algorithms promising [9]. In particular, multi-perspective algorithms seem suitable to capture the data and time perspective of the medical context. Thus, process mining techniques could be utilized to provide further analytical insight to medical personnel. They could enable the verification of guideline compliance, which is not only relevant for appropriate treatment but also, e.g., for the certification of oncology centers (https://www.krebsgesellschaft.de/gcs/german-cancer-society/certification.html, accessed on 20 June 2022). Furthermore, the ability to analyze treatment processes furthers medical research [10,11] and forms a basis for the development of decision-support systems that support physicians in practice [12,13].

However, the use of process mining to verify guideline conformance requires a representation of guideline knowledge as a process model. Due to the high dynamics, complexity and multidisciplinarity of medical processes [14] as well as the prerequisite of deep, implicit domain-specific knowledge for guideline interpretation, the transformation of guideline knowledge into a computer-interpretable representation is a non-trivial process [8,15]. In addition, misinterpretations carry the risk of incorrect treatment and thus potential dangers for the health of patients.

Based on the presented initial situation, we hypothesize that medical processes can be completely represented in computer-interpretable process models. For this purpose, a structured approach can support the efficient transfer of guideline knowledge into a computer-interpretable form and ensure correctness. Therefore, we present CGK4PM (Clinical Guideline Knowledge for Process Mining) as a first proposal for guided generation of procedural representations of guideline knowledge. The framework is an effective technique to generate an accurate guideline-based process model, which enables to unify treatment data with evidence-based and tacit knowledge. The transparent development and formalization process ensures comparability of results, reduction of transformation error-proneness, and provides a foundation for the application of process mining algorithms.

The remainder of the paper is organized as follows. In Section 2, the CGK4PM framework is first introduced. Next, we show in a case study the practical application of the framework in Section 3. Subsequently, Section 4 discusses the current state of the framework and the results of the case study. Finally, in Section 5, the paper is summarized, a conclusion based on the results is drawn, and future research directions are provided.

2. Materials and Methods

The development of process models for CGs is a form of knowledge engineering from text and expert knowledge combined with process modeling. Therefore, frameworks of knowledge acquisition and process modeling as well as experiences from these fields should be used as a foundation.

The framework should combine these areas and support knowledge acquisition from the guidelines. Among other things, it is intended to support the selection of guidelines and modeling languages and to address the potential impact of errors in medical models by reducing the risks of failures in the modeling process in a structured manner. In addition, the approach should integrate a well-founded and detailed quality assurance and evaluation so that the output is a correct, procedural and a process mining ready representation of the guidelines.

In the course of the literature research, no framework could be found that fulfills all requirements. This is especially due to the multidisciplinarity of the medical domain and the need for the correctness of guideline knowledge interpretation and modeling to avoid risks for patients [15]. Therefore, implicit expert knowledge for interpretation and for consolidation of possibly competing statements from several guidelines is required. Since misinterpretations and modeling failures can have severe consequences, the absolute correctness of the transferred knowledge is paramount compared to many other domains, even taking into account many boundary conditions and special cases. For this reason, we decided to use a framework which fulfills the fundamental requirements but is agile enough to be further specified and to adapt other problem-specific frameworks. Finally, we chose the stages of knowledge acquisition from Buchanan et al. [16]. It guides in five steps through the acquisition process, taking into account the possibility of correcting modeling errors after testing and thus addresses the safety of the transformed information. The steps were further adapted according to the problem definition and enhanced by approaches from the fields of process modeling, process mining and knowledge engineering. The resulting framework CGK4PM is intended to serve as an initial proposal for the structured acquisition of explicit and tacit guideline knowledge, the transformation into a procedural knowledge representation and thus the utilization of the knowledge for process mining application. In the following, the framework and its different stages are presented.

2.1. The CGK4PM Framework

We have adapted Buchanan’s framework [16] to the domain-specific requirements of medicine and the technical requirements of process mining and process modeling (see Figure 1). The adapted identification stage comprises the composition of the project team consisting of domain experts and process engineers, the selection of the medical guidelines to be transformed, and the generation of sample reference patients for the validation of the model later on. The following conceptual stage is executed in alternation with the formalization stage and consists of workshops with the stakeholders to create a conceptual procedural guideline representation. The formalization stage includes, on the one hand, the translation of the conceptualization into an initial formal representation of the guideline knowledge and, on the other hand, a review and reflection process designed to identify and correct modeling errors. Afterwards, the implementation stage follows, which now consists of the process modeling language (PML) selection for representing the guideline knowledge and the computer-interpretable implementation of it. The testing stage is aligned with established procedures and metrics to verify and validate the resulting procedural representation.

2.1.1. Identification Stage

In the first stage, (1) the project is set up, (2) the team is assembled, and (3) the guidelines are selected. The selected guidelines and the two sets of reference patients represent the output of this stage. The steps of the basic framework were adapted and specialized as follows:

(1)

Composing team. This activity involves the identification of participants and roles. Previous case studies [17] and the Process Mining Project Methodology (PM²) [18] recommend team members with diverse backgrounds [15,19]. Based on this fact and the fact that process engineers have little knowledge of the medical domain and medical staff typically have limited experience with process engineering and modeling [15,20], we define the following roles that need to be filled: domain experts (usually physicians of the domain in question) and process engineers (who have experience in transforming the knowledge into a procedural representation and to prepare it for the application of process mining).

(2)

Selecting clinical guidelines. The guideline selection is based on [21,22] and addresses the fact that there are different guidelines from different publishers on the same topic. In addition, depending on the guidelines and national requirements, there may be one aggregated guideline that addresses a specific topic or many guidelines that together form the evidence-based body of knowledge for the treatment of a disease. Therefore, the selection of the guidelines to be implemented is an important step that can significantly influence the quality of the result. Suggested decision criteria include validity and level of the guidelines’ evidence, the applicability in a clinical context, institutional recommendations, local practice variations, and the ease of operationalization [22]. When using multiple guidelines, it is important to cross-check them for conflicting recommendations [23]. In the following, the set of selected guidelines is denoted by G.

(3)

Generating sample reference patients. It is important that the acquired knowledge is checked for consistency and correctness in relation to the guidelines at all stages of the framework. For this purpose, reference patients are presented. Reference patients are synthetically generated patients with guideline-compliant or non-compliant treatment courses. Therefore, a set $C_G$ of compliant and a set $N_G$ of non-compliant reference patients for the set of guidelines G is generated by the stakeholders. It is recommended to describe patients that cover both special and standard cases and that cover as many possible pathways in treatment. At the same time, it must be considered that the chosen amount of reference patients is still manageable for manual evaluation.

2.1.2. Conceptualization Stage

The conceptualization stage and the formalization stage build on each other and are carried out sequentially and iteratively until the domain experts reach a consensus on the correctness of the formalized knowledge. Core elements of the conceptualization stage are conceptual modeling workshops, in which domain experts contribute their expertise [24] and discuss conflicting guideline interpretations [15]. The main goal of this stage is to collect the concepts formulated in the guidelines or by the domain experts and to transform them into a conceptual representation using the brown paper method [25].

In accordance with object-oriented software design [26], the result of the conceptualization stage is a simplified, readable procedural representation and does not necessarily have to conform to a particular standard. Since understanding the process is crucial to any process analysis [27], the conceptual representation by itself should also be intuitive and easy to understand for domain experts. The input of the stage is the set of selected guidelines G and the current state of the semi-formal representation (see Section 2.1.3) including the reviews, resulting from the alternating execution of the conceptualization and formalization stage. The output is the conceptual representation, consisting of new and revised concepts and used for the formalization, as well as the selected focus segment (see initial workshop) for the next workshop. The framework distinguishes between the initial workshop and the follow-up workshops:

• The initial workshop includes a segmentation of the guidelines into n segments. The segmentation process shall facilitate addressing and focusing on a specific segment in future workshops. The resulting number and size of segments depends on the complexity of the guidelines and the duration of the workshops. Confirming the state of the science, however, it was found that shorter workshops in which smaller segments were discussed produced better results [28]. Once the segmentation is complete and a focus segment has been selected, the workshop can proceed with the second step of the follow-up workshop.
• For the follow-up workshops a three-step approach is recommended: (1) discussion of the reviews (see Section 2.1.3), (2) discussion of the previously defined focus segment, derivation and collection of concepts from it, and conceptual modeling of these using the brown paper method, (3) definition of a new focus segment for the next workshop. It is important that the relation between guideline statements and the concepts can be established, e.g., through annotation of the concepts or documentation. This enables efficient maintenance of the model in case of guideline updates. During the workshop, it is the process engineers’ job to guide the discussions as a moderator and outline the discussions’ outcomes in the conceptual model.

Because conflicting interpretations and views of the guideline statements can occur, workshops with space for discussion are a core element [15]. In addition, based on the formulation of CGs, the strength of consensus in decisions should be documented.

2.1.3. Formalization Stage

Since computer-interpretable models are usually not easy to read, the results of the conceptualization phase, i.e., the collection of concepts, are first converted into a semi-formal representation by the process engineers and then checked for correctness by the domain experts.

Semi-Formal Modeling. In order to avoid misunderstandings and to detect errors, as well as to derive requirements for the PML later on, an intermediate result in the form of a semi-formal process representation of the concepts collected so far is created in this step. For this purpose, the collected concepts from the conceptualization phase are put into a global context with all previously collected concepts. Thus, the representation is done as a simple, readable model consisting of edges, nodes, conditions and comments. The implementation of new concepts therefore results in the deletion, editing or addition of edges, nodes, conditions or comments. For the development of the semi-formal representation, the use of the seven process modeling guidelines (7PMG) is recommended. The 7PMG are a set of recommendations for building a process model from scratch and for improving existing process models [29]. To ensure traceability of the changes made, all changes are recorded in a changelog and each new iteration results in a new version of the semi-formal representation. The output is not only a revised model, but also a list of questions for the domain experts with all the ambiguities that arose during modeling.

Review and Reflection. The review ensures that the conceptual model has been correctly understood by the process engineers and transferred into the semi-formal representation. The input is the semi-formal representation and a list of questions that arose during the modeling process. Based on the representation, the design and modeling decisions are reviewed by the domain experts. This includes checking the semi-formal representation for factual accuracy, identifying errors, collecting discussion points and commenting on the questionnaire in preparation for the next workshop.

Important tools in this activity are the guidelines and the reference patients. The Output of this stage is a semi-structured documentation of the review as a basis for the next workshop. The following questions can be used as guiding questions for this stage:

• Do the changes to the semi-formal representation exactly match the results from the last workshop?
• Does the representation correctly reflect the guidelines content of the modeled segment?
• Does the representation cover the set $C_G$ of patients defined in the planning stage? Furthermore, would the representation classify patient set $N_G$ as non-compliant?
• Are there any suggested changes to the representation, and in particular to the section under consideration?

Experience showed that merging clinical everyday life with the guidelines to a procedural representation is challenging, and that questioning the previous decisions again contributes decisively to the representation’s quality. Therefore, we recommend integrating Shiffman et al.’s [21] Deabstraction and Disambiguation process to improve the review result. The deabstraction refers to recommendations not specifically formulated in the guidelines, which the domain experts need to specify. Disambiguation is the process of establishing a single semantic interpretation for a recommendation when values of decision variables are not mutually exclusive [21]. Generally, the domain experts are the only stakeholders who are able to identify and document such problems. The process engineers help formalize and resolve identified ambiguities in the next workshop.

2.1.4. Implementation Stage

In this step, the semantically correct and complete semi-formal representation evaluated by the domain experts in the previous formalization phase is transformed into a computer-interpretable model based on a PML. Generally, PMLs are used in the field of BPM to represent, analyze and improve business processes within organizations [30]. Thus, these process models are abstract representations of business processes [31]. To describe the behavior of business processes, different PMLs have been developed, including BPMN, Petri nets, Declare and Causal Nets. Among other things, the modeling languages differ in syntax, expressive power, and mathematical foundation and are therefore better suited for certain environments and problems. Furthermore, PMLs can be distinguished between imperative and declarative languages [32]. Imperative, and therefore procedural modeling languages describe the allowed sequence of activities explicitly and thus define how a process needs to be executed [33]. Declarative modeling languages offer more flexibility in the execution of processes than procedural languages. Hence, declarative models can be executed in all possible ways as long as the behavior is not explicitly forbidden [34].

The process of the implementation stage is divided into the selection of the process modeling language and the transformation of the semi-formal representation into the computer-interpretable model. The selection of the PML is based on the approach for process modeling language selection of Kelemen et al. [35]. For this purpose, the five-step procedure was adapted for the process model representation.

• Criteria definition. The criteria definition is based on Luo & Tung [36] and divides PML requirements into modeling objectives. The identified objectives are desired characteristics that can be the degree of formalization of the language, the scalability of the language via the possibility of multilevel modeling, the ease of use of modeling [36], the intelligibility of the PML, software support, the portability of the format and the spread of the language [35]. Additional objectives include requirements for specific process mining perspectives (e.g., data, control flow, time, resource, case) [32,37,38] and the availability of required process mining algorithms. Each requirement is categorized as a mandatory requirement or an optional requirement. Optional requirements are weighted according to the importance of the requirement. For determining the weights, the weighted score method is used [39,40]. The development of the requirements is conducted in workshops with the stakeholders and documented on the basis of the IEEE 830-1998 standard [41].
• Candidate subset. Based on the current literature, a subset of candidate PMLs is identified. In particular, the experience of the process engineers can be drawn upon. For a basic pre-selection of modeling languages, we recommend considering the quality properties that provide a good basis for the evaluation of modeling techniques [42,43]:
  • Expressiveness. The degree to which a given modeling technique is capable of denoting the models of any number and kinds of application domains.
  • Arbitrariness. The degree of freedom one has when modeling one and the same domain.
  • Suitability. The degree to which a given modeling technique is specifically tailored for a specific kind of application domain.
• Candidate review. Candidates are reviewed in detail. For this purpose, each PML is examined to determine whether it meets the specified requirements. In addition to Boolean values, free text answers can also be given in case of ambiguity. The review results are documented in PML profiles.
• PML comparison. The PMLs are compared on the basis of the candidate review. For this purpose, the PML profiles are aggregated in a table. The overview is used to identify all PMLs that fulfill all mandatory requirements.
• PML selection. Finally, the PML is selected based on the results of the PML comparison. If several PMLs fulfill all mandatory requirements, the optional requirements and their weightings are used for decision-making.

After the PML is selected, the process engineers use it to transfer the semi-formal representation into a computer-interpretable model. The challenge is the correct semantic transfer of the previously designed medical process into logic procedural formulations. Therefore, it is recommended to divide the semi-formal representation into meaningful segments to facilitate the transfer and reduce the complexity of the modeling [29]. Thus, the process modeling is less error-prone [44]. For example, the first segment could be the admission process of a patient to the hospital. The segmentation may be different from the segmentation in Section 2.1.2. For each segment, the modeling process is divided into three steps:

(1)

Preparation— Identification of the segment’s requirements (e.g., in terms of data) and specification of the segment’s dependencies between segment activities and with predecessor and successor segments (e.g., edge conditions).

(2)

Modeling—Translation of the semi-formal representation into the computer- interpretable model.

(3)

Check—Verification of the modeled segment to ensure that it actually reflects the intended medical procedure and that all contingencies and possibilities have been integrated by the semi-formal representation.

Finally, the segments are aggregated into a single model by adding edges between the segments.

2.1.5. Testing Stage

In this step, the correctness of the implemented model is checked. A distinction is made between verification and validation, while verification examines the syntactic and semantic nature of a model, validation examines the suitability of the model with respect to its intended use [45,46]. Verification can typically be automated using algorithms, whereas validation is tied to human judgment and expertise and therefore must involve stakeholders and the process specification [46,47]. The relationship of validation and verification is described as building on each other in the context of process models [48]. Hence, if the verification fails, the detected errors must be corrected before the validation can be started. This reduces the number of validation cycles [46].

• Verification: As the complexity of the models often leads to deadlocks or lack of synchronization, verification is important in process modeling [46]. Therefore, the focus of verification is on identifying undesirable behaviors in the process model and syntactic errors [49]. Since the framework provides flexibility regarding the PML choice and verification approaches depend on the PML used, no specific approaches are described here. Following [48] the evaluation step is aborted if it fails and the model has to be revised in the implementation step. Only if the verification is successful, the model is validated.
• Validation: Validation deals with the correct translation of the guidelines recommendations into a process model. The validation is done in two steps. In the first step, the validation is carried out against the sample data set of reference patients. Using a conformance checking algorithm, these can be automatically validated against the process model. This is carried out by the process engineers and is intended to detect initial errors. In the second step, the model is tested using a facilitated model walkthrough, a method for checking the adequacy in representing the actual process [50]. In such walkthroughs, the process model is presented to the domain experts by the moderator. Based on this, the validity of the model is examined collaboratively and stakeholders point out errors. The moderator summarizes the results of the walkthrough and mediates the discussion between the stakeholders. Usually, several iterations are needed to reach a common result [51]. If errors are found, they have to be fixed by a backward step in the framework and verified and validated again.

3. Results

In this chapter, we describe how CGK4PM was applied in a specific case study. The case study was conducted together with the Skin Tumor Center of the University Hospital Münster. In particular, the University Hospital provided support through the participation of domain experts in the workshops, the selection of appropriate guideline sections, and the evaluation. In the following, we discuss the activities performed in each phase of the framework and the analysis results.

Identification Stage. The goal was to determine the guideline compliance of the diagnosis and initial treatment of patients with malignant melanoma. The project team consisted of three domain experts from Münster University Hospital as well as one process engineer from DFKI and one from University of Trier. The domain experts selected the guideline on diagnosis, therapy and follow-up of melanoma (https://www.awmf.org/leitlinien/detail/ll/032-024OL.html (Retrieved 16 July 2021) of the AWMF (The Association of the Scientific Medical Societies in Germany (AWMF), established in 1962 and located in Frankfurt am Main, is the umbrella organization of more than 150 German medical societies), which already contains aggregated evidence-based treatment recommendations. The focus of the use case should be on acute initial treatment and staging of patients and should cover a majority of patients. Therefore, chapters 4 (diagnostics and therapy in primary care) and 6 (diagnostics and therapy for locoregional metastasis) of the guideline were selected, which the domain experts estimated 70% of patients would undergo. The reference patients were informally described by the domain experts and transferred to an event log by the process engineers. In total, 10 compliant and 10 non-compliant patients were modeled.

Conceptualization Stage. In the initial workshop, the selected guideline section was segmented in eight segments: Diagnosis of Melanoma, Re-Excision, Sentinel Lymph Node Biopsy, Other Diagnostic Measures, Staging up to IIB, Staging IIC and III, Lymphadenectomy, and Adjuvant Therapy (see Figure 2). Each segment consists of a set of activities that are related in a certain way. Figure 3 outlines the activity breakdown of the Diagnosis of Melanoma segment. Therefore, a total of eight workshops were held to dedicate one workshop for each segment. During the workshops, the discussion of the review took a considerable amount of time. However, the results of the discussions led to substantial improvements in terms of quality. It has also been shown that the moderation of process engineers in the conceptualization phase is a critical factor for effective knowledge acquisition.

Formalization Stage. Based on the input from the workshops, a total of 11 versions of the semi-formal representations were designed. In three iterations, only the last focus segment of the representation needed to be adjusted, while in the other eight cases many cross-segment changes were necessary. This is due to the complexity of the domain, the knowledge gained over the workshops, and the strong interdependence of the segments. The process representation was based on the representation of event-driven process chain (EPC) models [52] and was supplemented by comments on specific sections and extended conditions. An extract of the model is shown in Figure 4. During the transformation of the knowledge concepts gathered in the conceptualization stage into an EPC, the 7PMG were considered to reduce errors and complexity. All changes made were recorded in a changelog and attached to the new version of the semi-formal representation to maintain traceability. During the transformation process, domain-specific questions arose that had to be answered by the domain experts. These were sent to the domain experts for review together with the new semi-formal representation. It became apparent that it was important to ask the questions in great detail and to support them partially graphically with representation suggestions, as otherwise misunderstandings could have arisen between the domain experts and the process engineers. The semi-formal representation and especially the changes were reviewed by the domain experts in each iteration. The comments collected in a semi-structured review protocol were then given to the workshop moderator as input for the next workshop. After 11 iterations, there was a consensus that the model was correct and complete and ready to be implemented.

Implementation Stage. The criteria definition resulted in 24 requirements for the PML whereof nine are regarding the control flow perspective, 10 are regarding the data perspective, and two are regarding the time perspective. Another requirement was process mining related and required that the PML must have conformance checking algorithms available for implementation. The last two requirements were assigned to the meta perspective, which describes domain specific information that could not be assigned to any other perspective.

Based on the criteria definition and literature research, data Petri nets (DPNs) [53,54] and MP-Declare [55] were identified as suitable PMLs and reviewed. As a result, both PMLs seemed to be appropriate for modeling. However, DPNs were chosen for modeling since the imperative modeling nature is more fitting to represent the guideline’s content, which mainly consists of explicit treatment recommendations. Furthermore, DPNs allow the global management of variables, which allows a more realistic representation of the domain regarding data management. The DPN was modeled in ProM [56] considering 7PMG [29]. An extract of the DPN model is shown in Figure 5. The segmentation of the sections during the transition from the conceptual to the computer-interpretable model is analogous to the segmentation of the conceptualization stage.

Testing Stage. Verification was performed by the process engineers. The ProM plug-in Analyze with Woflan was used to check the soundness and thus the safeness, proper completion, option to complete and liveness. Since DPNs cannot be processed with Analyze with Woflan, the DPN was first converted into a workflow net, a subclass of Petri nets. Three verification iterations were performed, as errors in the model were identified in the first two iterations and had to be fixed in the implementation stage.

The validation was performed in two steps, in the first step the compliance of the created reference patients was checked using the Multi-perspective Process Explorer [57]. In the first iteration of the validation, two compliant patients were classified as non-compliant and all non-compliant patients were identified as non-compliant. The result of the conformance check was then discussed in a facilitated model walkthrough. During the review, the domain experts identified the semantic errors. In addition, it was noted that patients who move from basic staging to advanced staging are not covered by the model, although this is guideline-compliant. This was discussed in another workshop in the conceptualization stage and subsequently implemented in the implementation stage. A further verification and validation, including another model walkthrough, did not identify any new errors.

4. Discussion

In this section, we discuss the results, findings, and open challenges we faced during the case study, along with some best practices for using CGK4PM. Fundamental challenges in modeling guideline knowledge are the complexity and dynamics of the medical field. This was also reflected in many aspects of the project. Since guidelines mostly give rule-based if-then recommendations, without putting them into an overall procedural context, it became apparent that the transformation towards a process representation requires a high degree of expert knowledge, especially to include the needed tacit knowledge. Furthermore, the guidelines do not fully describe the treatment processes of patients, creating process gaps that need to be filled by expert knowledge. Depending on the composition of the domain experts in the team, different outcomes may result in terms of quality, but also in terms of the design regarding treatment pathways. The high degree of variability in medical treatment processes was also challenging, which further complicated the correct and complete representation.

Overall, however, the knowledge acquisition and process modeling effort is relatively high. The initial creation and iterative revision of the semi-formal representation requires a lot of processing and coordination time from domain experts and process engineers, since the semantically correct implementation must be ensured and errors in the medical context usually imply serious consequences. Another aspect is the maintenance of the model, e.g., when a new version of the modeled guideline with new findings is published. This is addressed in the conceptual phase. The annotations of the concepts or documentation created there allows the model to be updated efficiently. CGK4PM enables modeling to be as correct as possible, but at the same time requires a lot of time for modeling and reconciliation. Currently, CGK4PM focuses on the transformation of explicit guideline knowledge, supported by implicit knowledge of domain experts to interpret the guideline, into an explicit, computer-interpretable knowledge representation. Furthermore, the framework could also be used to transform all other explicit knowledge representations, such as clinic-specific guidelines or recommendations for the treatment of rare diseases, into a computer-interpretable model.

In the case study, the guideline was successfully transformed into a computer-based, interpretable process model using CGK4PM. All treatment contingencies and all specifications of the guideline could first be summarized in a semi-formal knowledge representation and then transformed into a computer-interpretable process model. The close collaboration between domain experts and process engineers proved to be absolutely necessary, as the close exchange helped to identify errors early on and prevented incorrect modeling. The analysis and review iterations prescribed in the framework proved useful and required. This approach successfully corrected errors and filled in missing treatment paths during implementation. Furthermore, it proved to be good practice to have a moderator at the workshops to mediate between process engineers and domain experts to ensure an effective knowledge transfer. It became apparent that it was useful to use a process engineer as a moderator because the engineer was not too deeply involved in the medical discussion and could direct the discussion results into a process engineering perspective. The early consideration of procedurally important aspects simplified and accelerated the later modeling process. Critical reflection on the results of previous workshops led to a significant increase in the quality of the model, as well as a higher level of maturity.

Other framework-independent challenges could also be identified that require further addressing. These are mostly due to the fact that the model is a mapping of the guideline and thus covers all cases described there. It should be noted that a violation of the guideline or guideline model does not necessarily imply negative causality. For example, the deviation from guideline specifications can have many reasons such as the patient refusing treatment or cannot afford the treatment. Furthermore, in a retrospective analysis, the repeated identification of a deviation at a certain point in the treatment can lead to a new gain in knowledge. Namely, if it turns out that the deviation was justified and led to a better recovery of the patient. However, if an attempt were made to represent all contingencies in the model, a dilemma would emerge from a process modeling perspective. Especially for pure process models, variability quickly leads to highly complex models. If all path variants were represented, then the process model grows to a so-called spaghetti process [38] with a size that is no longer manageable and incomprehensible, so that the modeling as well as the administration and maintenance of the model mean great effort. However, if path variants are neglected, this would mean that the model does not cover affected patients and that their guideline compliance cannot be checked. One possibility to tackle this problem could be a mixed-paradigm process modeling approach, as recently presented by [58], which integrates the strengths of procedural and declarative representations such as Petri nets and Declare. As stated, this modeling approach is particularly interesting as it enables the compact capturing of complex behavior [58].

5. Conclusions

In this paper, we have presented a first methodological approach to acquire the textual knowledge of CGs and tacit knowledge regarding the guidelines from domain experts and transfer it into computer-interpretable models. This approach ensures correct transfer of CGs into a process model through an iterative process of reviews and assessments. This enables the use of process mining algorithms, e.g., conformance checking algorithms, to compare patient data with the created model to determine compliance or non-compliance with CGs in treatment.

As a proof of concept, we conducted a case study in which we transformed the CG for malignant melanoma into a data Petri net. During the implementation of the case study, we were able to test the practicality of the framework and confirm its applicability, and showed the importance of a structured approach involving domain experts and process engineers. The current state of the framework provides a solid starting point for the definition of a standard that enables the reliable transformation of guideline knowledge into a procedural representation, and thus makes it possible to work with process mining methods in the context of CGs.

Although the case study generated promising results, we plan to further refine the CGK4PM framework to improve the transformation results. In the identification stage, the creation of reference patients must be carried out by domain experts and the assistance of the process engineers. It is planned to extend the framework to provide more assistance in the creation process, making it easier to generate high-quality sets of compliant and non-compliant patients. We found the iterations between conceptualization and formalization stage to be extremely time-consuming and intend to develop a more optimized and efficient procedure by conducting more case studies. The criteria definition in the implementation stage is non-trivial and needs thorough analysis to provide a useful foundation for the PML selection. Therefore, we plan to implement more guidance and templates for reuse. The testing stage in its current state describes the verification and validation process in a rather generic way. To raise the outcomes’ quality and ensure thorough testing procedures, we want to extend the stage and provide detailed approaches for imperative and declarative PMLs. The approach also offers basically the possibility of transferring several guidelines (for the same or for different clinical pictures) into one model. This will be further investigated in a future case study.