Introduction

Mortality data provide essential evidence on the health status of populations in crisis-affected and resource-poor settings and can help to guide and assess relief operations [1], [2]. Relief agencies most commonly measure mortality through retrospective surveys, in which a representative sample of households is interviewed using a standardised questionnaire about demographic changes (births, deaths, in- and out-migrations) in the household over a specified period in the past (the recall period). However, these surveys have serious feasibility limitations. First, they require significant time and resources to carry out because of the large sample sizes required (usually >900 households): this includes transport, hiring, training and supervising a team of interviewers, and entering data from a large number of questionnaires. Second, most data collection time during retrospective surveys is spent collecting data on living people (death is a comparatively rare event), which leaves less time for in-depth and reliable investigation of causes and circumstances of death. Third, retrospective surveys are poorly suited to investigating very recent periods (e.g. the last 1–2 months), as this entails interviewing a very large number of households (for example, estimating a crude mortality rate of 1 death per 10 000 per day with precision ±0.3 deaths per 10 000 per day over 1 month through a conventional two-stage cluster sample survey with a design effect of 2 and a mean household size of 5 would require a sample of 5691 households) [3], [4], [5], [6]. As a result, mortality surveys conducted in the humanitarian sector are not done frequently enough and often do not provide information useful for assessing programme performance. They also usually generate limited or unreliable data on causes of death, thereby reducing their usefulness for operational purposes [4], [7].

In a separate paper [8], we reported the validity of an alternative method for measuring mortality (the informant method) designed to provide estimates of mortality over a short recall period (i.e. almost on a real-time basis), and enable ascertainment of causes of death. We showed that the method's sensitivity, measured in four separate field settings against a best estimate of mortality derived through capture-recapture analysis, was moderate (55–73%), but encouraging enough to warrant further development. Elsewhere, we also reported findings of verbal autopsy questionnaires performed as part of the informant method in Malawi [9]. Here we report on the informant method's feasibility. The study objective was to assess the economic feasibility of the informant method.

Methods

Feasibility evaluation

During the field testing, we systematically recorded time inputs in hours and minutes for paid staff, study participants, and community informants on standardised forms. We also systematically recorded the cost in local currency of activities by staff type. These included salary payments and associated costs such as for vehicle hire and fuel which were included under the driver staff type. Costs for key informants, FGD participants, and respondents were not included as they did not receive any payment. The activities and staff types included in the analysis are shown in the first column of Table 1. We computed the total person-hours and financial cost for implementation of the informant method, as well as the time and cost required to detect each death. The local currency costs were converted to US dollars at local exchange rates on the first data collection day in each study site (see Table 2 for exchange rates used).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Activities, staff types, and assumptions for time inputs for a retrospective survey, by site.

https://doi.org/10.1371/journal.pone.0025175.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Costings for staff used for the informant method and a retrospective mortality survey, by site.

https://doi.org/10.1371/journal.pone.0025175.t002

We then compared the observed time and cost inputs for implementing the informant method in each study site with those estimated for a conventional retrospective mortality survey. To do this, we assumed that a retrospective survey would be conducted in each of the four sites broadly in line with guidelines described in the Standardised Monitoring and Assessment in Relief and Transition (SMART) initiative (www.smartindicators.org) [10]. Taking into account the layout and organisation of the four sites, we assumed that cluster sampling with probability proportional to size (PPS) would be used in Kabul and Chiradzulu, and linear systematic random sampling in the Mae La and Tanzania camps, where household lists were available. We assumed that a survey with 6 months recall would feature typical sample sizes of 30 clusters of 32 households in Kabul and Chiradzulu (i.e. 960 households in total, based on a design effect of 2.0), and 480 households (i.e. the same effective sample size, but with no design effect) in Mae La and Tanzania. The above sample sizes are adequate to detect a crude mortality rate over 6 months of 2, 1 or 0.5 deaths per 10 000 person-days with an absolute precision of ±0.4, 0.3 and 0.2 deaths per 10 000 person-days respectively (i.e. a relative precision no worse than ±40%), assuming a mean household size of five persons. We also compared the informant method to a survey with a 30 day recall (i.e. the same as the informant method), for which we assumed a sample size of 200 clusters of 32 households (6402 households; cluster sampling with design effect = 2.0) or 3201 households (no design effect), needed to obtain relative precision no worse than ±40% if the crude mortality rate is ≥0.5 per 10 000 person-days. Other assumptions for time inputs for a retrospective survey are reported in Table 1.

Estimated economic costs were multiplied by time inputs for both the informant method and retrospective mortality surveys to enable an economic cost-based comparison between the two methods. The costs for a hypothetical retrospective survey were calculated for each site using the unit costs recorded for implementing the informant method (Table 2).

We also evaluated the feasibility of adding a verbal autopsy (VA) component to the informant method in the Chiradzulu District site. This was evaluated only in one site due to budget constraints of the research project. We also felt that one site would provide sufficient evidence for the use of verbal autopsy with the informant method. We selected Chiradzulu district because the partner agency (Médecins Sans Frontières-France) expressed a desire to measure cause-specific mortality and had the capacity to use such data for its operational interventions. We used the most recent, standardised World Health Organization VA questionnaires (http://www.who.int/whosis/mort/verbalautopsystandards/en/index.html) [11], [12]. These consist of three different questionnaires according to the age of the decedent (less than four weeks; four weeks to 14 years; older than 14 years), and containing several sub-modules (e.g. on neonatal conditions, injuries, and maternal mortality) depending on the signs and symptoms reported by the respondent. The questionnaires were translated into Chichewa by a clinical officer, and translations were reviewed by another clinician as well as other members of the study team. If during the exhaustive search, a death was established as taking place within a 30 day recall period, the respondent was asked to also participate in a separate VA interview, conducted by a clinical officer. We timed each verbal autopsy interview, as well as time required for its analysis according to World Health Organization guidelines (parallel review by two independent clinicians, and an additional review by a third clinician in case of discrepant classification of cause of death by the first two). Details on causes of death as ascertained through VA in Chiradzulu are reported elsewhere [9].

Results

Descriptive information on the population size, data collection period, response rate, mortality data collected by the informant method and its sensitivity is given in Table 3. All key community informants who were found agreed to provide information. In Chiradzulu District, two households delayed their consent after consulting with family members or the headman. In Tanzania, one household refused to give consent.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Timeframe, population covered, response rate, mortality data and sensitivity recorded by the informant method, by site (from Roberts et al. [8]).

https://doi.org/10.1371/journal.pone.0025175.t003

The summary time and cost inputs for conducting the informant method in each of the four sites are given in Table 4 (please see Table S1 for detailed activity time and cost inputs by staff type). There was significant variance in the time-inputs among the study sites. For example, the study in Mae La camp required only 168 person-hours (21 person-working days) to be completed compared to 2069 (259 person-working days) in Chiradzulu District.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Summary time and cost inputs of the informant method and retrospective surveys with 6 month and 30 day recall periods, by site and activity.

https://doi.org/10.1371/journal.pone.0025175.t004

When compared to a survey with a 6 month recall period, the informant method was estimated to require 658 (31%) fewer person-hours in Kabul, 432 (72%) in Mae La camp, and 230 (34%) in the Tanzania camps than the survey. However, in Chiradzulu District, the informant method was estimated to require 335 (19%) more person-hours than the survey (Table 4). If only study staff time is considered (i.e. excluding respondents, participants and key-informants), the informant method was estimated to require 509 (27%) fewer person-hours in Kabul, and 390 (83%) fewer person-hours in Mae La camp, and 238 (44%) fewer person-hours in the Tanzania camps, but 438 (30%) more person-hours in Chiradzulu District. Population estimation accounted for only 107/1482 (7%) person-hours of the informant method in Kabul, but considerably more (1003/2069 or 48%) in Chiradzulu (Table S1 and Table 5).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Time and cost inputs of the informant method and retrospective surveys with 6 month and 30 day recall periods, by site and staff type.

https://doi.org/10.1371/journal.pone.0025175.t005

The total cost of the informant method ranged from $1065 in Mae La camp to $15158 in Chiradzulu. The informant method was estimated to cost $1127 (9%) less than a survey with 6 months recall period in Kabul, $3330 (76%) less in Mae La camp, $646 (4%) less in Chiradzulu, and $1830 (42%) less in Tanzania (Table 4).

When the informant method is compared to a survey with a 30 day recall period, the informant method was estimated to require 11148 (88%) fewer person-hours in Kabul, 3379 (95%) fewer in Mae La camp, 8029 (80%) fewer in Chiradzulu, and 3251 (88%) fewer in the Tanzania camps. The informant method also cost $51290 (81%) less than a survey in Kabul, $18664 (95%) less in Mae La camp, $65660 (81%) less in Chiradzulu, and $14796 (85%) less in Tanzania.

Table 5 presents time and cost inputs by staff type. The informant method required substantially less time inputs for data collectors and particularly respondents in all four study sites when compared with retrospective surveys. Less time was spent by study investigators on the informant method compared with retrospective surveys, with the exception of Kabul (when compared with a 6 month recall survey): here two investigators were present as this was the first site in which the informant method was tested. The ‘other staff’ category recorded more time input for the informant method than a survey. This was particularly the case in Chiradzulu District due to the time input by village residents hired to perform structure counts for the population estimation. The informant method incurred a lower burden to respondents than a retrospective survey in terms of respondent time inputs: 216 (90%) fewer respondent hours in Kabul; 112 (93%) fewer in Mae La camp, 215 (90%) fewer in Chiradzulu and 104 (87%) fewer in the Tanzania camps. These differences are even greater when compared with 30 day recall survey, with over 98% fewer respondent hours in all the study sites.

The time (and US$ cost) required per death detected using the informant method was 5 hours ($28) in the Tanzania camps, 6 hours ($39) in Mae La camp, 22 hours ($178) in District One of Kabul, and 23 hours ($163) in Chiradzulu District.

For the feasibility of adding verbal autopsy questionnaires to the informant method, each interview for the verbal autopsy questionnaire took an average of 38 minutes, and 56 verbal autopsy questionnaires were completed in Chiradzulu District. Each questionnaire took an average of 44 minutes to analyse (including time input by all questionnaire reviewers). The total time to conduct the verbal autopsy component was therefore 76.5 person-hours (4% of total person-time for the informant method's implementation in Chiradzulu District).

Discussion

This study presents findings on the economic feasibility of a new method to estimate mortality in crisis-affected and resource-poor settings, based on field evaluations in four different sites.

There was significant variance in the time-inputs among the study sites, reflecting their different characteristics. The informant method in Mae La camp in Thailand and the Tanzania camps required only 168 person-hours and 444 person-hours respectively. This was because little travel and data collection time were required due to the low number of deaths recorded and high population density of these camp settings. They also did not require population estimation to be conducted. The study in Tanzania required greater time inputs than Mae La camp principally because two camps were included. By contrast, the study in Chiradzulu District recorded 93 deaths in 96 villages covering an area of approximately 875 km², and required population estimation, which accounted for almost half of total person-hours (most of which was attributable to villagers hired to count structures). The Kabul site also entailed high person-time inputs, partly due to the requirement to hire additional data collection staff to ensure same-sex interviews and FGDs; population estimation in Kabul, on the other hand, accounted for a small proportion of total person-time (7% of total person hours).

Comparisons between the informant method and conventional retrospective surveys suggest that the informant method could offer considerable economic benefits over retrospective surveys. The informant method required less time than retrospective surveys with a 6 month recall period in three of the four study sites. It also showed considerable monetary savings, with staff cost estimates in all four study sites lower for the informant method than for a survey. In future studies, where accurate population data (for all ages and for under 5 years) are not available, population estimation would be required, stratifying the estimate by age group, as we did in this study for the Kabul and Chiradzulu District study sites. However, even where population estimation is required, the results from our study suggest that the informant method would require less time and monetary inputs than for a survey. Importantly, the recall period for the informant method was 1 month which would be of much greater operational use for relief agencies than a survey with a 6 month recall period, particularly in acute emergencies in which humanitarian agencies need recent mortality data. When the results of the informant method were compared against a retrospective survey with the recall period of 1 month, the time and costs benefits of the informant method increased substantially. However, a drawback of the informant method is that it only measures mortality, whereas retrospective surveys can include other outcomes, such as child nutrition and micronutrient data and vaccination status at little extra cost [10].

Data entry is substantially less for the informant method than retrospective surveys. If the entire community is surveyed exhaustively, data analysis is also simpler than for surveys by removing the need for weighting and design effect adjustment inherent in sample surveys, or individual person-time calculation when the recall period is long and the cohort very dynamic. The method could therefore be used by programme staff with limited research skills.

Generally, we believe that the informant method would be most feasible in camps and concentrated populations. In post-emergency camps or other communities where population size is being monitored, feasibility results from Mae La and the Tanzania camps represent what might be routinely expected. In chaotic situations where no population estimates are available, time and costs of the population size estimation required for the informant method would be expected to increase.

A major benefit of the informant method was that respondent burden was considerably less than for retrospective mortality surveys, with approximately 90% less time spent by informants on the informant method compared with surveys with a 6 month recall period (rising to over 98% for surveys with a 30 day recall period). Surveys can be burdensome for communities, and reducing respondent time has ethical implications [13]. In addition, all key community informants who were found agreed to provide information, and household response was almost 100%.

However, a number of potential ethical issues regarding willingness to participate potentially exist. The referral by key informants who are also community leaders could infringe on the principle of voluntary participation. The informant method exploits hierarchical social structures: deference towards and/or fear of authorities may mean that households might be unable to refuse participation in the study, or decide which information about the decedent they wish to disclose (while introducing bias into estimation, deliberately false answers may be a justifiable way for the household to protect itself against the consequences of sharing certain information with strangers). We did not observe or hear of incidents suggesting any of these dynamics, although we recognise that such delicate issues are difficult to gauge in the short amount of time we spent in each study location. Moreover, the method's reliance on key informants or neighbours to identify households with deaths essentially deprives those households of the choice to disclose the death to investigators, and to some extent to disclose details on the decedent. We also found it difficult to persuade key informants not to be present during the interview (no key-informants were actually present during the interviews but they were present during the initial introductions for around 20% of the households visited), but they should be actively discouraged from being present during the interview to ensure full confidentiality. Follow-up work could be conducted to explore any untoward effects of the referral system that underpins the informant method, and find ways to strengthen the consent and confidentiality arrangements. However, it should also be recognised that similar ethical issues around voluntary participation exist in retrospective surveys which often seek permission for elders and village leaders and such permission can also place pressure on respondents to participate.

The addition of verbal autopsy questionnaires in the Chiradzulu District site took a total additional time of 76.5 hours, 4% of the total person-time input in that site. We believe this suggests that it is feasible to routinely add verbal autopsy questionnaires to the informant method given the reduced time required for conducting the informant method when compared to a retrospective survey. The addition of verbal autopsy questionnaires when measuring mortality rates provides an extremely important way of increasing the accuracy of recording the causes of mortality and so helping to inform appropriate health interventions and responses.

There was a very high willingness of households to respond. If response rates were lower then it would reduce the sensitivity and economic feasibility of the method. However, we do not feel that these high response rates were due to the particular research situations in our study, and the varied settings and populations used for the study provide a good indication that such response rates could be expected elsewhere. We also do not believe there were any aspects of our own research approach that meant response rates were higher in our study than if the informant method was used by other researchers applying sufficient levels of care and the same basic principles and standards required for research in such settings [10].

The informant method also had only moderate sensitivity and the next steps in achieving better sensitivity would include the use of ethnographic research on how information of death is shared in a community, in a variety of settings, to help refine the informant method procedures and questionnaires for future use. Another round of validating the method in a few more sites, focussing on increasing sensitivity through more effective use of key informants, would also be beneficial. Such sites should include unstable displaced camps with fluctuating populations.

Supporting Information

Acknowledgments

We thank the field staff and residents in the four sites. We are grateful to Megan Deitchler at FANTA; the Ministry of Public Health in Afghanistan; Annemarie terVeen; Aide Médicale Internationale in Thailand; Médecins Sans Frontières-France in Malawi; the United Nations High Commissioner for Refugees, in particular the late Herman Tirwosha; and the Tanzania Red Cross Society.