Next Article in Journal
Preliminary Effectiveness of mHealth App-Based Pelvic Floor Muscle Training among Pregnant Women to Improve Their Exercise Adherence: A Pilot Randomised Control Trial
Previous Article in Journal
Basic Psychological Needs Satisfaction Mediates the Link between Strengths Use and Teachers’ Work Engagement
Previous Article in Special Issue
Quantifying the Costs to Different Funders over Five-Years for Women Diagnosed with Breast Cancer in Queensland, Australia: A Data Linkage Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 19
Issue 4
10.3390/ijerph19042331
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Cost-Effectiveness of PET/CT Surveillance Schedules to Detect Distant Recurrence of Resected Stage III Melanoma

by
Mbathio Dieng
1,*,
Robin M. Turner
2,
Sarah J. Lord
1,
Andrew J. Einstein
3,
Alexander M. Menzies
4,5,
Robyn P. M. Saw
4,6,7,
Omgo E. Nieweg
4,6,7,
John F. Thompson
4,6,7 and
Rachael L. Morton
1,4
1
NHMRC Clinical Trials Centre, Faculty of Medicine and Health, The University of Sydney, Camperdown 2050, Australia
2
Biostatistics Centre, Otago University, Dunedin 9016, New Zealand
3
Seymour, Paul, and Gloria Milstein Division of Cardiology, Department of Medicine and Radiology, Columbia University Irving Medical Center and New York-Presbyterian Hospital, New York, NY 10032, USA
4
Melanoma Institute Australia, North Sydney 2060, Australia
5
Department of Medical Oncology, Royal North Shore and Mater Hospitals, North Sydney 2060, Australia
6
Faculty of Medicine and Health, Sydney Medical School, The University of Sydney, Camperdown 2050, Australia
7
Department of Melanoma and Surgical Oncology, Royal Prince Alfred Hospital, Camperdown 2050, Australia
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(4), 2331; https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph19042331
Submission received: 23 November 2021 / Revised: 8 February 2022 / Accepted: 10 February 2022 / Published: 17 February 2022
(This article belongs to the Special Issue The Economic Impact and Cost-Effectiveness of Long-Term Cancer Survivorship (>5 Years))
Browse Figures
Versions Notes

Abstract

:
Objective: To estimate the cost-effectiveness of three surveillance imaging strategies using whole-body positron emission tomography (PET) with computed tomography (CT) (PET/CT) in a follow-up program for adults with resected stage III melanoma. Methods: An analytic decision model was constructed to estimate the costs and benefits of PET/CT surveillance imaging performed 3-monthly, 6-monthly, or 12-monthly compared with no surveillance imaging. Results: At 5 years, 3-monthly PET/CT surveillance imaging incurred a total cost of AUD 88,387 per patient, versus AUD 77,998 for 6-monthly, AUD 52,560 for 12-monthly imaging, and AUD 51,149 for no surveillance imaging. When compared with no surveillance imaging, 12-monthly PET/CT imaging was associated with a 4% increase in correctly diagnosed and treated distant disease; a 0.5% increase with 6-monthly imaging and 1% increase with 3-monthly imaging. The incremental cost-effectiveness ratio (ICER) of 12-monthly PET/CT surveillance imaging was AUD 34,362 for each additional distant recurrence correctly diagnosed and treated, compared with no surveillance imaging. For the outcome of cost per diagnostic error avoided, the no surveillance imaging strategy was the least costly and most effective. Conclusion: With the ICER for this strategy less than AUD 50,000 per unit of health benefit, the 12-monthly surveillance imaging strategy is considered good value for money.

1. Introduction

Melanoma patients with distant metastases in more than two organs and high tumour burden have a poor response to therapy and low survival rates [1,2]. Early treatment appears to improve the chance of survival [3,4]. Therefore, adequate and early detection of distant metastases is paramount. Frequently scheduled imaging seems an attractive tool to this end [5]. Australian and some international consensus-based guidelines recommend regular computed tomography (CT) or fludeoxyglucose positron emission tomography (FDG-PET)/CT in conjunction with clinical consultations for the follow-up of asymptomatic advanced-stage melanoma patients [6,7,8].
International guidelines for follow-up after treatment of stage III melanoma vary by country and discipline, particularly in relation to the type of imaging test recommended, the timing of imaging, and the clinical stage in which they should be performed [9]. The German guidelines recommend CT or PET/CT every 6 months for the first 3 years for resected stage III patients [10]; The United States (US) guidelines recommend imaging every 4 to 12 months for the first 5 years [8], while the United Kingdom (UK) National Institute for Health and Care Excellence (NICE) guidelines recommend surveillance imaging only if there is a clinical trial investigating the value of regular imaging, or a local policy with specific funding for imaging 6-monthly for 3 years [6]. The Australian Melanoma Guidelines (2018) state CT of the chest, abdomen, and pelvis or PET may be performed prior to definitive therapy where the detection of metastatic disease would influence management [11]. Ultrasound assessment can detect loco-regional recurrence but is not applicable for detection of systemic recurrence and is not recommended by Australian guidelines.
Furthermore, few research studies have investigated the cost-effectiveness of a surveillance imaging strategy in resected stage III melanoma patients [12,13,14]. Mostly the published studies have reported costs and benefits of one-off (baseline) imaging prior to surgery or imaging to assess treatment response in the management of distant (Stage IV) disease. As well as exploring whether annual imaging to detect systemic recurrence for resected stage III melanoma patients is more effective than no annual imaging, there is a need to investigate and establish its relative cost-effectiveness for the identification of melanoma recurrence.
The follow-up strategies in current practice were developed before potentially effective systemic therapies were available to treat advanced melanoma and also before patients with stage III melanoma received adjuvant therapy (where different adjuvant treatments have different recurrence risks, particularly in the first 1–2 years). These new therapies have significant benefits for patients and in the metastatic setting the potential to be most efficacious in patients with lower disease burden [3]. Therefore, it might be beneficial to identify recurrence earlier for patients who might benefit more from earlier systemic treatment [15,16,17]. This study aims to investigate the ‘opportunity cost’ of surveillance imaging by exploring the cost-effectiveness of four surveillance strategies for the diagnosis and treatment of distant melanoma recurrence.
Several studies have estimated the potential harms of frequent imaging, including ‘scanxiety’ [18], exposure to radiation [19], incidental findings [20], and increased financial burden [21] to patients, as well as to the healthcare system. With increasing melanoma incidence (60 cases per 100,000 people per year), the cost impact of frequent imaging for Australian healthcare is substantial [22]. According to the Australian Bureau of Statistics, the government expenditure for diagnostic imaging has doubled over the period 2000–2018 (from AUD 1159 M to AUD 3641 M per annum).
Currently, there are no high-level evidence-based follow up guidelines to assist clinicians in weighing up the costs and benefits of surveillance imaging [22]. Robust evidence of the cost-effectiveness of imaging surveillance is required for clinical practice guidelines and health policy decisions relating to post treatment surveillance.
A randomised controlled trial [23] is currently underway; however, until these results are known, a decision-analytic model drawing from the best current sources of evidence is most appropriate to inform clinical decision making [24].
This study aimed to investigate the cost-effectiveness of current surveillance imaging strategies compared with no surveillance imaging, for an accurate diagnosis of distant melanoma recurrence from a health system perspective, by modelling the relative costs and benefits over 5 years.

2. Methods

2.1. Cohort

We examined resected American Joint Cancer Committee (AJCC) stage IIIA-D [25] melanoma patients treated at the Melanoma Institute Australia (MIA) between the years 2000 and 2017 who were disease-free at the first follow-up after initial surgical treatment for melanoma. All patients were followed until the development of distant recurrence or end of follow-up schedule. The identified patients belonged to one of four follow-up categories: (1) 3–4 monthly surveillance imaging; (2) 6-monthly imaging; (3) 12-monthly imaging; or (4) no routine surveillance imaging [26].

2.2. Decision Model Structure

We constructed an analytic decision model to estimate the costs and benefits of follow-up with PET/CT performed 3–4 monthly, 6-monthly, or 12-monthly compared with no surveillance imaging follow up to diagnose distant melanoma recurrence. (Figure 1) The two economic outcomes evaluated were cost per distant recurrence accurately diagnosed and treated, and the cost per diagnostic error avoided. We calculated ‘distant recurrence correctly diagnosed’ as the difference between true positives and false negatives, respectively, for each branch (choice). Similarly, we calculated ‘diagnostic error avoided’ as the difference between true negatives and false positives for each branch. The model was designed to evaluate the ability of the four surveillance strategies to detect distant recurrence for each group of patients. We used an Australian health system perspective with a time horizon of 5-years and applied a 5% discount rate. Discounting is a procedure for converting costs or benefits occurring at different time points to a common measure by use of an appropriate discount rate [27]. The Australian Government recommends a discount rate of 5% is used in health technology assessment submissions [28]. The patient population was disease-free at the first follow-up visit. A detailed protocol of the economic evaluation has been published previously [26].

2.3. Model Assumptions

The key assumptions for the decision model were:
  • Surveillance imaging was primarily used to identify distant recurrence; therefore, loco-regional recurrences were excluded.
  • A first distant recurrence only occurs once. Therefore, the model simulates the costs and benefits up to the first (initial) distant recurrence.
  • Based on Australian guidelines [7], patients with positive imaging test results underwent confirmatory investigations, including fine-needle aspiration biopsy (FNAB) and whole-body PET/CT (if CT only was previously undertaken).
  • All patients with distant melanoma detected underwent treatment.
  • The costs for unresectable stage III or stage IV melanoma treatment included diagnostic imaging, doctors’ visits, genetic testing, and pharmacotherapies where relevant, as well as costs for management of Grade 3 and 4 adverse events from treatment modalities (e.g., colitis) [29].
  • Radiation-attributable malignancies from PET/CT imaging were not included because these generally take >5 years to develop.
  • Costs of palliative and end of life care were excluded because patients were censored after the development of distant disease.

2.4. Model Inputs

The direct unit costs associated with healthcare activities in the four surveillance strategies, including PET/CT imaging, invasive procedures (e.g., FNAB and core biopsy), and clinical follow-up, were estimated from the Medical Benefits Schedule (MBS) for outpatient care and Australian-Refined Diagnosis Related Groups (AR-DRGs) for hospital admissions (Supplementary Material Table S1) [30]. The no surveillance imaging strategy costs over 5 years includes all healthcare costs, such as doctor’s visits, diagnostic tests, and procedures ordered based on symptoms or treatment.
Total costs were adjusted to 2020 Australian dollars (AUD) based on the Consumer Price Index [31]. For a whole-body PET/CT, the unit cost of PET/CT imaging was calculated as the sum of the cost of PET/low dose CT (MBS item: 61553) and adjunctive CT for the purpose of anatomic localisation (MBS item: 61505). For patients who had a positive test result (i.e., true positives and false positives), an additional cost component was added to account for the confirmatory investigations performed to verify the index imaging results. Based on the latest Australian Melanoma guidelines [11], this cost was calculated as a composite of the following investigations: fine-needle aspiration biopsy (FNAB), whole-body PET/CT, and other tests. This cost was added when the test finding was a false negative, as additional tests were often requested by the clinician to investigate the imaging findings. For the first outcome, the cost per distant recurrence correctly diagnosed and treated, patients diagnosed with stage IV disease incurred an additional cost of treatment.

2.5. Base Case Analysis

The base case represented a patient with melanoma stage IIIA to IIID undergoing follow-up for distant disease recurrence with surveillance imaging at the MIA. Probabilities for each branch of the decision tree were taken from the hazard rates for distant recurrence in the MIA cohort and the reported sensitivity and specificity of PET/CT [32,33].
The economic results were presented as the cost per distant recurrence appropriately diagnosed and treated, assuming all patients are treatable. The costs were calculated as an aggregate of follow-up surveillance costs, the cost of confirmatory tests, and the cost of treatment (applied to all patients that developed stage IV disease). For each of the outcomes, the incremental cost-effectiveness ratios (ICER) were calculated using the no surveillance imaging strategy as the comparator over a 5-year follow-up period. The ICER was compared to a willingness-to-pay (WTP) threshold of AUD 50,000 per unit of health benefit, a commonly used threshold for the value of money.

2.6. Sensitivity Analyses

Two major sources of uncertainty were identified for this model, uncertainty in the clinical parameters and uncertainty in the modelled calculations. Deterministic sensitivity analyses were conducted to address uncertainty for all the input clinical parameters over the range shown in Table 1. The results of one-way sensitivity analyses were illustrated using a tornado diagram [34]. To address the uncertainty in the modelled calculations, a probabilistic sensitivity analysis was conducted by assigning distributions to the model parameters to evaluate changes in the cost-effectiveness result based on the parameter distributions and their variance from the mean. Probabilities and test performance characteristics were modelled using a beta distribution, whereas costs were modelled using a gamma distribution. A Monte Carlo simulation with 10,000 replications was used for this purpose [35].
The decision model was built using TreeAge software (TreeAge Software Inc., Williamstown, MA, USA). This software package automatically generates the algorithms required to evaluate the model and choose the optimal strategy by giving weight to each possible outcome based on its probability. The cost-effectiveness analyses and the sensitivity analyses were also carried out using TreeAge, see Figure 1 for a detailed illustration of the model.
Ethics approval was not required for this cost-effectiveness study; however, ethical approval for the original cohort was obtained from the MIA research committee (MIA2016/182), the Australian Institute of Health and Welfare (EO2019-1-454), and the Royal Prince Alfred hospital X18-0144, LNR/18/RPAH/206. The economic evaluation was reported according to the CHEERS (Consolidated Health Economic Evaluation Reporting Standards) checklist [36].

3. Results

Table 1 presents the characteristics of patients in the three imaging regimens, and the volume of resources used. On average, each patient received 6.3 (SD3.7) CTs, PET/CT, or PET scans over 5 years and 4.8 (5.5) extra investigations conducted to confirm equivocal or false-positive findings for the 3-monthly imaging, 3.7 (2.3) scans and 4.7 (4.6) extra investigations for 6-monthly imaging, and 4.3 (2.4) scans and 3.4 (3.2) extra investigations for 12-monthly imaging. Table 2 presents the results for the outcome of cost per case of distant melanoma appropriately diagnosed and treated. For this outcome, the 12-monthly imaging schedule was both more costly and more effective than the no-imaging schedule. When compared with no routine surveillance imaging, 12-monthly imaging incurred greater incremental costs (AUD 2748) per person and resulted in an additional 4% of distant metastasis correctly diagnosed. This represented an ICER of AUD 34,362, which indicates the additional cost required to accurately diagnose and treat one person with distant melanoma. The 12-monthly imaging was less costly at AUD 52,160 per patient and more effective, with a corresponding of 92 of 100 distant metastases correctly diagnosed, than the 6-monthly imaging strategy, with a cost of AUD 77,998 per patient and a corresponding of 88 of 100 distant metastases correctly diagnosed. The 3-monthly imaging schedule was more costly and less effective than the other imaging schedules.
Table 3 presents the additional cost required to avoid a diagnosis error and shows that follow-up without imaging was less costly and more effective, with a total cost of AUD 1513 and a corresponding probability of diagnostic error avoided of 0.8832. The 3-monthly and 6-monthly PET/CT imaging strategies were more expensive options at AUD 16,268 and AUD 25,304 per patient with corresponding probabilities of diagnostic error avoided of 0.8386 and 0.7999.

Sensitivity Analysis

The results of one-way sensitivity analyses are displayed in a tornado diagram in which each bar represents the impact of uncertainty for the individual parameter on the ICER (Figure 2). The model was most sensitive to variations in the prevalence of distant recurrence in the no surveillance imaging and 12-monthly imaging groups and to the sensitivity of PET/CT. For example, adjusting the sensitivity of PET/CT from the base case of 79% up to 86% caused the ICER for the 12-monthly imaging strategy compared with no surveillance imaging to decrease to AUD 20,614 per case of distant melanoma accurately diagnosed and treated. Less pronounced changes in the ICER were seen when varying the costs of treatment. The cost-effectiveness results of the model were robust over a plausible range of parameter estimates.
Overall, our probabilistic sensitivity analysis results were consistent with our base case model and provided evidence that surveillance imaging of resected stage IIIA-D melanoma patients was unlikely to be a cost-effective strategy. The probabilistic sensitivity analyses showed ICERs for imaging-based surveillance were not favourable compared with no imaging. For example, 12-monthly imaging compared with no routine imaging showed less than half (44%) of the simulations were cost-effective using a WTP threshold of AUD 50,000 per distant recurrence appropriately diagnosed and treated (Figure 3).

4. Discussion

Using commonly recommended surveillance imaging strategies and real-world data from a large melanoma treatment centre in Australia, our baseline model suggests 12-monthly PET/CT surveillance imaging for resected stage IIIA-D melanoma patients was slightly more expensive but more effective than a no imaging strategy with an ICER that represents good value for money. Twelve-monthly surveillance imaging was less expensive and more effective than 3 or 6-monthly surveillance imaging for the detection of distant disease. Extensive sensitivity analyses highlighted that these cost-effectiveness results were sensitive to the probability of distant recurrence at 5 years in the 12-monthly and no imaging strategies. When compared with no surveillance imaging, 12-monthly PET/CT surveillance imaging was associated with the largest increase in appropriately diagnosed distant disease versus smaller increases for 6-monthly imaging and 3–4 monthly imaging.
Our model favours less frequent imaging, which we found less expensive primarily due to fewer additional tests related to false positives. From our previous work of a well-documented stage III melanoma cohort, 42% of patients had false-positive findings and incidental findings in 21% of patients [20]. These false positives and incidental findings resulted in subsequent healthcare use, thus the elevated costs of frequent imaging.
Our findings are consistent with other studies that have analysed surveillance imaging modalities for stage II–III melanoma patients, in that frequent follow-up imaging is not cost-effective even for intermediate to high-risk patients [37,38]. However, a recent study found for AJCC stage IIC and III melanoma, 6-monthly CT scans of the chest, abdomen and pelvis were cost-effective for the early detection of metastases in the first 4 years of follow-up, over other follow-up investigations (clinical visit, brain MRI) [39]. This study stratified its population by AJCC sub-stage and compared the cost-effectiveness of CT scanning performed at baseline and first year of follow-up with imaging studies performed each subsequent year of follow-up (years 2–5). Our study population was different in that it included resected AJCC stage III patients without distant disease at their 6 or 12-month follow-up visit.
The results of the cost-effectiveness of surveillance imaging with respect to early diagnosis and treatment need to be taken together with recent research supporting survival benefits of new systemic immunotherapy [40,41]. Observational studies indicate accurate, and early detection of distant recurrence is important so that effective treatments can be commenced without delay.
Our model generated an ICER of AUD 34,362 for each additional case of accurate diagnosis and treatment of distant recurrence with a 12-monthly surveillance imaging strategy. However, an ICER of AUD 34,362 is challenging to interpret, as there is currently no established willingness to pay (WTP) threshold associated with a recurrence appropriately diagnosed and treated [42]. Based on the current arbitrary willingness to pay thresholds of AUD 50,000, this ICER suggests that 12-monthly imaging for the diagnosis of recurrent distant melanoma, in the use of every resected stage III melanoma patient, is likely to be cost-effective.
PET/CT is approved and currently being reimbursed by the Medical Services Advisory Committee (MSAC), and systemic treatment has been approved by the Pharmaceutical Benefits Advisory Committee (PBAC), meaning that the government is willing to pay for these tests and drugs. We could assume that the Australian Government would be willing to pay for the cost of a whole-body PET/CT (AUD 1128.75) for each additional accurate diagnosis, and the cost of therapeutic regimens offered to a patient diagnosed with distant metastatic disease, estimated to be AUD 115,072 [29]. Hence, this is a very wide range for the different WTP thresholds. However, there are no restrictions on the frequency of monitoring that accompanies this care.
Novel research around the standardised estimates for WTP for an appropriate diagnosis of distant recurrence is needed as immunotherapy is now given as adjuvant treatment in stage III disease to prevent distant recurrence, and that surveillance imaging is used to monitor the effectiveness of these drugs, as well as looking for the first signs of disease to treat. Future research is needed to examine the cost-effectiveness of surveillance imaging for patients with stage III melanoma undergoing adjuvant or neo-adjuvant systemic therapies.
The more frequent imaging strategies (3 or 4-monthly and 6-monthly imaging) are more costly and less effective in appropriately diagnosing distant melanoma compared with 12-monthly imaging. Indeed, the more intensive imaging strategies led to a higher number of diagnostic tests, procedures, clinic visits, and referrals to other physicians as a result of frequent false-positive findings.
The main strength of this study is that our model inputs for PET/CT test performance were taken from a recent longitudinal cohort study that assessed the performance of an imaging schedule over several years, rather than at a single (cross-sectional) time point [32]. Our study does have some limitations. First, radiation burden is important when considering the frequency of PET/CT surveillance in clinical practice. However, in this model, we did not account for radiation burden due to the limited duration of 5-years follow-up post-treatment. For lifetime models, the inclusion of lifetime attributable risk estimates for cancer incidence is important given estimates for cancer incidence following exposure to 10 mSv being highly age and gender-dependent, with young females being especially sensitive to radiation [43]. Secondly, our model did not account for survival as an effectiveness measure because the focus for surveillance imaging is on the accurate detection of disease that can lead to early treatment. The effectiveness of that treatment, and therefore, survival gains are less related to the diagnostic test but more related to the efficacy of the treatment. Furthermore, this study was specifically designed to address the impact of imaging frequency on the detection of distant recurrence and was not meant to include the survival benefit of follow-up surveillance. Indeed, the current state of the literature has limited evidence on the impact of routine surveillance imaging on survival and quality of life. In addition, analysis of this cohort showed that the initial surveillance schedule had little impact on distant disease-free survival. Randomised comparison of imaging strategies with long term follow-up is needed as in this MIA cohort, people with more advanced sub-stage were scheduled to more frequent imaging. Thirdly, the effect of PET/CT in this study was solely to assess the detection of distant recurrence in isolation to the detection of loco-regional recurrence. Lastly, concerns remain about imaging-associated patient anxiety during follow-ups for cancer patients [44]. Our model did not account for imaging-associated anxiety and psychological issues faced by patients during follow-ups. Our intent was to address policy issues for the cost-effective surveillance of resected stage III melanoma patients based on the frequency of imaging.
Our modelling suggests that 12-monthly surveillance imaging could be the most cost-effective diagnostic strategy in resected stage III melanoma patients, given current data. This work was based on a large prospective cohort study of routine PET/CT imaging in patients with stage III melanoma and is the first to quantify the costs and benefits of different imaging schedules and contribute to much-needed cost-effectiveness evidence for melanoma guidelines and health policy.
Given the changing context, there still is insufficient evidence to firmly exclude more frequent PET/CT or no routine imaging strategy as cost-effective diagnostic options for resected stage III melanoma patients. These findings must be taken together with recent research supporting that current systemic immunotherapy provides a survival benefit [10]. Indeed, this strengthens the role of early detection of metastases through intensive surveillance programmes, which eventually could lead to a significant rise in the cost associated with early diagnosis and treatment of melanoma metastases. Furthermore, it also needs to be considered that examining the value of follow-up PET/CT for detecting metastases in suspected recurrence in isolation from the same role in primary melanoma and in the adjuvant setting is not necessarily appropriate. This wider context of the use of PET/CT in follow-ups of melanoma patients needs to be considered.
Adopting the approach of less frequent imaging in resected asymptomatic patients following treatment would reduce costs and is unlikely to adversely impact the clinical outcomes of melanoma patients. Although the influence of both clinical practice guidelines and cost-effectiveness analyses on changing physician practice is variable [45,46], this study’s findings will add knowledge to the goal of reducing low-value care.
Large, prospective trials with in-built cost-effectiveness and survival studies are further needed to help guideline bodies provide efficiency-based follow-up recommendations.

5. Conclusions

Healthcare funding decisions are increasingly being assessed on the basis of cost-effectiveness as well as their impact on health outcomes. Among resected stage III melanoma patients, if surveillance imaging is to be performed at all, then 12 months is the most cost-effective strategy. Twelve-monthly surveillance imaging was less expensive and more effective than 3 or 6- monthly compared with no surveillance imaging, and at a WTP that is less than AUD 50,000 per patient, accurately diagnosed and treated.

Supplementary Materials

The following supporting information can be downloaded at: https://0-www-mdpi-com.brum.beds.ac.uk/article/10.3390/ijerph19042331/s1, Table S1: Base Case Model Parameters and Sensitivity Analysis Ranges.

Author Contributions

Conceptualization, R.L.M., M.D., J.F.T., O.E.N., R.P.M.S., A.J.E., R.M.T., S.J.L. and A.M.M.; methodology, M.D., R.L.M.; software, M.D., R.L.M.; validation, R.L.M., R.P.M.S. and M.D.; formal analysis, M.D.; investigation, M.D., R.L.M.; resources, R.L.M. and M.D.; data curation, M.D.; writing—original draft preparation, M.D.; writing—review and editing, M.D.; visualization, M.D.; supervision, M.D.; project administration, M.D.; funding acquisition, R.L.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by [Cancer Australia] grant number [1129568, 2017-2020].

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of the Melanoma Institute Australia (MIA2016/182), the Australian Institute of Health and Welfare (EO2019-1-454) and the Royal Prince Alfred Hospital Ethics Committee X18-0144, LNR/18/RPAH/206.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The others authors declare no conflict of interest. RPMS is supported by Melanoma Institute Australia and has received honoraria for advisory board participation from MSD, Novartis and Qbiotics and speaking honoraria from BMS. JFT has received honoraria for advisory board participation from BMS Australia, MSD Australia, GSK and Provectus Inc. (Knoxville, TN, USA), travel support from GSK and Provectus Inc., and support for conference attendance from Novartis.

References

  1. Nakamura, Y.; Kitano, S.; Takahashi, A.; Tsutsumida, A.; Namikawa, K.; Tanese, K.; Abe, T.; Funakoshi, T.; Yamamoto, N.; Amagai, M.; et al. Nivolumab for advanced melanoma: Pretreatment prognostic factors and early outcome markers during therapy. Oncotarget 2016, 7, 77404–77415. [Google Scholar] [CrossRef] [PubMed]
  2. Long, G.V.; Grob, J.-J.; Nathan, P.; Ribas, A.; Robert, C.; Schadendorf, D.; Lane, S.R.; Mak, C.; Legenne, P.; Flaherty, K.T.; et al. Factors predictive of response, disease progression, and overall survival after dabrafenib and trametinib combination treatment: A pooled analysis of individual patient data from randomised trials. Lancet Oncol. 2016, 17, 1743–1754. [Google Scholar] [CrossRef]
  3. Joseph, R.; Elassaiss-Schaap, J.; Kefford, R.; Hwu, W.-J.; Wolchok, J.D.; Joshua, A.; Ribas, A.; Hodi, F.S.; Hamid, O.; Robert, C.; et al. Baseline Tumor Size Is an Independent Prognostic Factor for Overall Survival in Patients with Melanoma Treated with Pembrolizumab. Clin. Cancer Res. 2018, 24, 4960–4967. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Poklepovic, A.S.; Carvajal, R.D. Prognostic Value of Low Tumor Burden in Patients With Melanoma. Oncology 2018, 32, e90–e96. [Google Scholar]
  5. Deschner, B.; Wayne, J.D. Follow-up of the melanoma patient. J. Surg. Oncol. 2019, 119, 262–268. [Google Scholar] [CrossRef] [PubMed]
  6. National Collaborating Centre for Cancer (UK). Melanoma: Assessment and Management; National Institute for Health and Care Excellence (NICE): London, UK, 2015. [Google Scholar]
  7. Cancer Council Australia. Clinical Practice Guidelines for the Diagnosis and Management of Melanoma; Cancer Council Australia: Sydney, Australia, 2018. [Google Scholar]
  8. National Cancer Council Network. Clinical Practice Guidelines in Oncology. Melanoma; National Comprehensive Cancer Network: Philadelphia, PA, USA, 2014. [Google Scholar]
  9. Marciano, N.J.; Merlin, T.L.; Bessen, T.; Street, J.M. To what extent are current guidelines for cutaneous melanoma follow up based on scientific evidence? Int. J. Clin. Pract. 2014, 68, 761–770. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Pflugfelder, A.; Kochs, C.; Blum, A.; Capellaro, M.; Czeschik, C.; Dettenborn, T.; Dill, D.; Dippel, E.; Eigentler, T.; Feyer, P.; et al. Malignant Melanoma S3-Guideline “Diagnosis, Therapy and Follow-up of Melanoma”. J. Dtsch. Dermatol. Ges. 2013, 11, 1–116. [Google Scholar] [CrossRef]
  11. Coit, D.G.; Thompson, J.A.; Algazi, A.; Andtbacka, R.; Bichakjian, C.K.; Carson, I.W.E.; Daniels, G.A.; DiMaio, D.; Ernstoff, M.; Fields, R.C.; et al. Melanoma, Version 2. 2016, NCCN Clinical Practice Guidelines in Oncology. J. Natl. Compr. Cancer Netw. 2016, 14, 450–473. [Google Scholar]
  12. Bastiaannet, E.; Groot, C.A.U.-D.; Brouwers, A.H.; van der Jagt, E.J.; Hoekstra, O.S.; Oyen, W.; Verzijlbergen, F.; van Ooijen, B.; Thompson, J.F.; Hoekstra, H.J. Cost-effectiveness of Adding FDG-PET or CT to the Diagnostic Work-up of Patients with Stage III Melanoma. Ann. Surg. 2012, 255, 771–776. [Google Scholar] [CrossRef]
  13. Krug, B.; Crott, R.; Roch, I.; Lonneux, M.; Beguin, C.; Baurain, J.-F.; Pirson, A.-S.; Borght, T.V. Cost-effectiveness analysis of FDG PET-CT in the management of pulmonary metastases from malignant melanoma. Acta Oncol. 2010, 49, 192–200. [Google Scholar] [CrossRef]
  14. Hong, N.J.L.; Petrella, T.; Chan, K. Cost-effectiveness analysis of staging strategies in patients with regionally metastatic melanoma. J. Surg. Oncol. 2015, 111, 423–430. [Google Scholar] [CrossRef] [PubMed]
  15. James, L.; Brown, M.P.; Arance, A.M.; Axel, H.; Paola, Q.; Del Vecchio, M.; Ascierto, P.A.; Ivana, K.; Jacob, S.; Bart, N.; et al. An open-label, multicentre safety study of vemurafenib in patients with BRAF(V600)-mutant metastatic melanoma: Final analysis and a validated prognostic scoring system. Eur. J. Cancer 2019, 107, 175–185. [Google Scholar]
  16. Wolchok, J.D.; Hoos, A.; O’Day, S.; Weber, J.S.; Hamid, O.; Lebbé, C.; Maio, M.; Binder, M.; Bohnsack, O.; Nichol, G.; et al. Guidelines for the evaluation of immune therapy activity in solid tumors: Immune-related response criteria. Clin. Cancer Res. 2009, 15, 7412–7420. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. Larkin, J.; Minor, D.; D’Angelo, S.; Neyns, B.; Smylie, M.; Miller, W.H., Jr.; Gutzmer, R.; Linette, G.; Chmielowski, B.; Lao, C.D.; et al. Overall Survival in Patients With Advanced Melanoma Who Received Nivolumab Versus Investigator’s Choice Chemotherapy in CheckMate 037: A Randomized, Controlled, Open-Label Phase III Trial. J. Clin. Oncol. 2018, 36, 383–390. [Google Scholar] [CrossRef] [PubMed]
  18. Li, L.; Severens, J.L.; Mandrik, O. Disutility associated with cancer screening programs: A systematic review. PLoS ONE 2019, 14, e0220148. [Google Scholar] [CrossRef]
  19. Lin, E.C. Radiation Risk From Medical Imaging. Mayo Clin. Proc. 2010, 85, 1142–1146. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  20. Nijhuis, A.A.G.; Dieng, M.; Mecon, N.K.; Lord, S.J.; Dalton, J.; Menzies, A.M.; Turner, R.M.; Allen, J.; Saw, R.P.M.; Nieweg, O.E.; et al. False-Positive Results and Incidental Findings with Annual CT or PET/CT Surveillance in Asymptomatic Patients with Resected Stage III Melanoma. Ann. Surg. Oncol. 2019, 26, 1860–1868. [Google Scholar] [CrossRef]
  21. Ding, A.; Eisenberg, J.D.; Pandharipande, P.V. The Economic Burden of Incidentally Detected Findings. Radiol. Clin. N. Am. 2011, 49, 257–265. [Google Scholar] [CrossRef] [Green Version]
  22. Jones, W.O.; Harman, C.R.; Ng, A.K.; Shaw, J.H. Incidence of Malignant Melanoma in Auckland, New Zealand: Highest Rates in the World. World J. Surg. 1999, 23, 732–735. [Google Scholar] [CrossRef]
  23. Naeser, Y.; Helgadottir, H.; Brandberg, Y.; Hansson, J.; Bagge, R.O.; Elander, N.O.; Ingvar, C.; Isaksson, K.; Flygare, P.; Nilsson, C.; et al. TRIM study protocol-a prospective randomized multicenter Trial to assess the Role of Imaging during follow-up after radical surgery of stage IIB-C and III cutaneous malignant Melanoma. BMC Cancer 2020, 20, 1–10. [Google Scholar] [CrossRef]
  24. Kuntz, K.M. Can mathematical modeling remove the tarnish on old clinical trials? Am. J. Med. 2001, 111, 155–156. [Google Scholar] [CrossRef]
  25. Gershenwald, J.E.; Scolyer, R.A. Melanoma Staging: American Joint Committee on Cancer (AJCC) 8th Edition and Beyond. Ann. Surg. Oncol. 2018, 25, 2105–2110. [Google Scholar] [CrossRef]
  26. Dieng, M.; Khanna, N.; Nguyen, M.T.H.; Turner, R.; Lord, S.J.; Menzies, A.M.; Allen, J.; Saw, R.; Nieweg, E.O.; Thompson, J.; et al. Cost-effectiveness analysis of PET/CT surveillance imaging to detect systemic recurrence in resected stage III melanoma: Study protocol. BMJ Open 2020, 10, e037857. [Google Scholar] [CrossRef]
  27. Culyer, T. The Dictionary of Health Economics, 3rd ed; Edward Elgar Publishing: Cheltenham, UK, 2014. [Google Scholar]
  28. Pharmaceutical Benefits Advisory Committee (PBAC). Guidelines for Preparing a Submission to the Pharmaceutical Benefits Advisory Committee (PBAC Guidelines), Version 5.0; Pharmaceutical Benefits Advisory Committee: Canberra, Australia, 2016. [Google Scholar]
  29. Elliott, T.M.; Whiteman, D.C.; Olsen, C.; Gordon, L.G. Estimated Healthcare Costs of Melanoma in Australia Over 3 Years Post-Diagnosis. Appl. Health. Econ. Health. Policy 2017, 15, 805–816. [Google Scholar] [CrossRef]
  30. Australian Government. Pricing Framework for Australian Public Hospital Services 2019-20, in Authority IHP, 30th ed.; Australian Government: Canberra, Australia, 2018. [Google Scholar]
  31. Australian Bureau of Statistics (ABS). Consumer Price Index of Australia; ABS: Canberra, Australia, 2021. [Google Scholar]
  32. Turner, R.M.; Dieng, M.; Khanna, N.; Nguyen, M.; Zeng, J.; Nijhuis, A.A.G.; Nieweg, O.E.; Einstein, A.J.; Emmett, L.; Lord, S.J.; et al. Performance of Long-Term CT and PET/CT Surveillance for Detection of Distant Recurrence in Patients with Resected Stage IIIA–D Melanoma. Ann. Surg. Oncol. 2021, 28, 4561–4569. [Google Scholar] [CrossRef]
  33. Machet, L.; Nemeth-Normand, F.; Giraudeau, B.; Perrinaud, A.; Tiguemounine, J.; Ayoub, J.; Alison, D.; Vaillant, L.; Lorette, G. Is ultrasound lymph node examination superior to clinical examination in melanoma follow-up? A monocentre cohort study of 373 patients. Br. J. Dermatol. 2005, 152, 66–70. [Google Scholar]
  34. Briggs, A.H.; Weinstein, M.C.; Fenwick, E.A.; Karnon, J.; Sculpher, M.J.; Paltiel, A.D. Model Parameter Estimation and Uncertainty: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Value Health. 2012, 15, 835–842. [Google Scholar] [CrossRef] [Green Version]
  35. Briggs, A.H.; Mooney, C.Z.; Wonderling, D. Constructing confidence intervals for cost-effectiveness ratios: An evaluation of parametric and non-parametric techniques using Monte Carlo simulation. Stat. Med. 1999, 18, 3245–3262. [Google Scholar] [CrossRef]
  36. Husereau, D.; Drummond, M.; Petrou, S.; Carswell, C.; Moher, D.; Greenberg, D.; Augustovski, F.; Briggs, A.H.; Mauskopf, J.; Loder, E.; et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and Elaboration: A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. Value Health 2013, 16, 231–250. [Google Scholar] [CrossRef] [Green Version]
  37. Leiter, U.; Marghoob, A.A.; Lasithiotakis, K.; Eigentler, T.K.; Meier, F.; Meisner, C.; Garbe, C. Costs of the detection of metastases and follow-up examinations in cutaneous melanoma. Melanoma Res. 2009, 19, 50–57. [Google Scholar] [CrossRef]
  38. Hengge, U.R.; Wallerand, A.; Stutzki, A.; Kockel, N. Cost-effectiveness of reduced follow-up in malignant melanoma. JDDG: J. Dtsch. Dermatol. Ges. 2007, 5, 898–907. [Google Scholar] [CrossRef] [PubMed]
  39. Podlipnik, S.; Moreno-Ramírez, D.; Carrera, C.; Barreiro, A.; Manubens, E.; Ferrandiz-Pulido, L.; Sánchez, M.; Vidal-Sicart, S.; Malvehy, J.; Puig, S. Cost-effectiveness analysis of imaging strategy for an intensive follow-up of patients with American Joint Committee on Cancer stage IIB, IIC and III malignant melanoma. Br. J. Dermatol. 2019, 180, 1190–1197. [Google Scholar] [CrossRef] [PubMed]
  40. Hodi, F.S.; Chiarion-Sileni, V.; Gonzalez, R.; Grob, J.-J.; Rutkowski, P.; Cowey, C.L.; Lao, C.D.; Schadendorf, D.; Wagstaff, J.; Dummer, R.; et al. Nivolumab plus ipilimumab or nivolumab alone versus ipilimumab alone in advanced melanoma (CheckMate 067): 4-year outcomes of a multicentre, randomised, phase 3 trial. Lancet Oncol. 2018, 19, 1480–1492. [Google Scholar] [CrossRef]
  41. Hamid, O.; Robert, C.; Daud, A.; Hodi, F.S.; Hwu, W.J.; Kefford, R.; Wolchok, J.D.; Hersey, P.; Joseph, R.; Weber, J.S.; et al. Five-year survival outcomes for patients with advanced melanoma treated with pembrolizumab in KEYNOTE-001. Ann. Oncol. 2019, 30, 582–588. [Google Scholar] [CrossRef] [PubMed]
  42. Sloane, F.A. Valuing Health Care: Costs, Benefits, and Effectiveness of Pharmaceuticals and Other Medical Technologies; Cambridge University Press: New York, NY, USA, 1995. [Google Scholar]
  43. Wall, B.F.; Haylock, R.; Jansen, J.T.M.; Hillier, M.C.; Hart, D.; Shrimpton, P.C. Radiation Risks from Medical X-ray Examinations As a Function of Age and Sex of the Patient; Health Protection Agency: London, UK, 2011. [Google Scholar]
  44. Thompson, C.A.; Charlson, M.E.; Schenkein, E.; Wells, M.T.; Furman, R.R.; Elstrom, R.; Ruan, J.; Martin, P.; Leonard, J.P. Surveillance CT scans are a source of anxiety and fear of recurrence in long-term lymphoma survivors. Ann. Oncol. 2010, 21, 2262–2266. [Google Scholar] [CrossRef]
  45. Grol, R.; Grimshaw, J. From best evidence to best practice: Effective implementation of change in patients’ care. Lancet 2003, 362, 1225–1230. [Google Scholar] [CrossRef]
  46. Ubel, P.A.; Jepson, C.; Baron, J.; Hershey, J.C.; Asch, D.A. The influence of cost-effectiveness information on physicians’ cancer screening recommendations. Soc. Sci. Med. 2003, 56, 1727–1736. [Google Scholar] [CrossRef]
Ijerph 19 02331 g001 550
Figure 1. Decision model structure.
Figure 1. Decision model structure.
Ijerph 19 02331 g001
Ijerph 19 02331 g002 550
Figure 2. One-way sensitivity analysis (tornado) diagrams for the outcome distant disease accurately diagnosed and treated. This diagram shows the degree to which uncertainty in individual variables affects the ICER. Each bar represents the range of the individual parameter and its impact on the ICER across that range. Bars are arranged in order, with the variable with the biggest impact at the top and the variable with the smallest impact at the bottom. Other parameters also evaluated but without influence on the ICER were surveillance imaging sensitivity, specificity, and the probability of distant recurrence with the no imaging strategy. ICER: Incremental Cost-Effectiveness Ratio. (A) is comparing 12-monthly imaging strategy versus Follow-up without routine imaging strategy. (B) is comparing 6-monthly imaging strategy versus Follow-up without routine imaging strategy. (C) is comparing 3-4-monthly imaging strategy versus Follow-up without routine imaging strategy.
Figure 2. One-way sensitivity analysis (tornado) diagrams for the outcome distant disease accurately diagnosed and treated. This diagram shows the degree to which uncertainty in individual variables affects the ICER. Each bar represents the range of the individual parameter and its impact on the ICER across that range. Bars are arranged in order, with the variable with the biggest impact at the top and the variable with the smallest impact at the bottom. Other parameters also evaluated but without influence on the ICER were surveillance imaging sensitivity, specificity, and the probability of distant recurrence with the no imaging strategy. ICER: Incremental Cost-Effectiveness Ratio. (A) is comparing 12-monthly imaging strategy versus Follow-up without routine imaging strategy. (B) is comparing 6-monthly imaging strategy versus Follow-up without routine imaging strategy. (C) is comparing 3-4-monthly imaging strategy versus Follow-up without routine imaging strategy.
Ijerph 19 02331 g002
Ijerph 19 02331 g003 550
Figure 3. Incremental cost and effectiveness of 12-monthly PET-CT over no routine imaging strategy for the outcome of distant disease accurately diagnosed and treated. This scatter plot shows the distribution of 10,000 trials from the Monte Carlo simulation.
Figure 3. Incremental cost and effectiveness of 12-monthly PET-CT over no routine imaging strategy for the outcome of distant disease accurately diagnosed and treated. This scatter plot shows the distribution of 10,000 trials from the Monte Carlo simulation.
Ijerph 19 02331 g003
Table
Table 1. Characteristic and resource use of the people included in the study by imaging schedule.
Table 1. Characteristic and resource use of the people included in the study by imaging schedule.
CharacteristicsImaging Schedule
3–4 Monthly 6-Monthly12-MonthlyNo-ImagingTotal
Total: N (%)146 (18%)47 (6%)284 (35%)346 (42%)823 (100%)
Gender: N (%)
Male 91 (11%)33 (4%)181 (22%)230 (28%)535 (65%)
Age at Diagnosis: Mean (SD)59 (14.19)54 (13.68)52 (14.70)61 (16.71)57 (15.94)
Age at Diagnosis: Median (IQR)60 (49–69)55 (44–64)53 (42–62)62 (50–74)58 (46–69)
Site of Primary: N (%)
Head and Neck 22 (15%)9 (19%)43 (15%)69 (20%)143 (17%)
Trunk38 (26%)12 (26%)102 (36%)91 (26%)243 (30%)
Upper Limbs13 (9%)4 (9%)33 (12%)30 (9%)80 (10%)
Lower Limbs28 (19%)12 (25%)72 (25%)60 (17%)172 (21%)
Occult45 (31%)10 (21%)34 (12%)95 (27%)184 (22%)
Other 1 (0.3%)1 (0.1%)
AJCC Stage: N (%)
III45 (31%)10 (21%)35 (12%)101 (29%)191 (23%)
IIIA26 (18%)12 (26%)117 (41%)42 (12%)197 (24%)
IIIB39 (27%)12 (26%)91 (32%)74 (21%)216 (26%)
IIIC36 (25%)13 (28%)41 (14%)129 (37%)219 (27%)
Resource Use
Diagnostic Imaging (surveillance)
Count902172 1214 2116
Mean (SD)6.3 (3.7)3.7 (2.3)4.3 (2.4)
Range[2–19][1–11][1–13]
Extra investigations *
Count690222960 1872
Mean (SD)4.8 (5.5)4.7 (4.6)3.4 (3.2)
Range[0–28][0–22][0–16]
* No detail on sub staging.
Table
Table 2. Base-case results for the cost per case of distant melanoma appropriately diagnosed and treated (2020 prices) referencing common baseline.
Table 2. Base-case results for the cost per case of distant melanoma appropriately diagnosed and treated (2020 prices) referencing common baseline.
StrategyMean Cost Per Patient
(AUD)		Incremental Cost
(AUD)		Effectiveness *Incremental Proportion of Distant Disease Appropriately Diagnosed and Treated *ICER **
(AUD/Outcome)
No imaging51,149 0.8770--
12-monthly52,16027480.91810.041134,362
6-monthly77,99828,4760.88240.0054Dominated
3 to 4-monthly88,38736,8600.88450.0075Dominated ***
* Proportion of distant recurrence correctly diagnosed and treated; ** ICER: Incremental Cost-Effectiveness Ratio; *** (Over AUD 1 M).
Table
Table 3. Base case results for the cost per diagnostic error avoided (2020 prices).
Table 3. Base case results for the cost per diagnostic error avoided (2020 prices).
StrategyMean Total Cost Per Patient
(AUD)		Incremental Cost (AUD)Effectiveness
(Diagnostic Error Avoided)		Incremental
Effectiveness *
No imaging follow-up1513 0.8832
12-monthly imaging908475710.8503−0.0329
6-monthly imaging16,26814,7550.8386−0.0446
3 to 4-monthly imaging25,30423,7910.7999−0.0833
* The proportion of diagnostic error avoided. The negative sign demonstrated the no imaging strategy incurred less diagnostic error.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Dieng, M.; Turner, R.M.; Lord, S.J.; Einstein, A.J.; Menzies, A.M.; Saw, R.P.M.; Nieweg, O.E.; Thompson, J.F.; Morton, R.L. Cost-Effectiveness of PET/CT Surveillance Schedules to Detect Distant Recurrence of Resected Stage III Melanoma. Int. J. Environ. Res. Public Health 2022, 19, 2331. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph19042331

AMA Style

Dieng M, Turner RM, Lord SJ, Einstein AJ, Menzies AM, Saw RPM, Nieweg OE, Thompson JF, Morton RL. Cost-Effectiveness of PET/CT Surveillance Schedules to Detect Distant Recurrence of Resected Stage III Melanoma. International Journal of Environmental Research and Public Health. 2022; 19(4):2331. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph19042331

Chicago/Turabian Style

Dieng, Mbathio, Robin M. Turner, Sarah J. Lord, Andrew J. Einstein, Alexander M. Menzies, Robyn P. M. Saw, Omgo E. Nieweg, John F. Thompson, and Rachael L. Morton. 2022. "Cost-Effectiveness of PET/CT Surveillance Schedules to Detect Distant Recurrence of Resected Stage III Melanoma" International Journal of Environmental Research and Public Health 19, no. 4: 2331. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph19042331

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop