Cluster randomized trial assessing the effects of rapid ethical assessment on informed consent comprehension in a low-resource setting

Abstract

Background

Maximizing comprehension is a major challenge for informed consent processes in low-literacy and resource-limited settings. Application of rapid qualitative assessments to improve the informed consent process is increasingly considered useful. This study assessed the effects of Rapid Ethical Assessment (REA) on comprehension, retention and quality of the informed consent process.

Methods

A cluster randomized trial was conducted among participants of HPV sero-prevalence study in two districts of Northern Ethiopia, in 2013. A total of 300 study participants, 150 in the intervention and 150 in the control group, were included in the study. For the intervention group, the informed consent process was designed with further revisions based on REA findings. Informed consent comprehension levels and quality of the consent process were measured using the Modular Informed Consent Comprehension Assessment (MICCA) and Quality of Informed Consent (QuIC) process assessment tools, respectively.

Result

Study recruitment rates were 88.7 % and 80.7 % (p = 0.05), while study retention rates were 85.7 % and 70.3 % (p < 0.005) for the intervention and control groups respectively. Overall, the mean informed consent comprehension scores for the intervention and control groups were 73.1 % and 45.2 %, respectively, with a mean difference in comprehension score of 27.9 % (95 % CI 24.0 % - 33.4 %; p < 0.001,). Mean scores for quality of informed consent for the intervention and control groups were 89.1 % and 78.5 %, respectively, with a mean difference of 10.5 % (95 % CI 6.8 -14.2 %; p < 0.001).

Conclusion

Levels of informed consent comprehension, quality of the consent process, study recruitment and retention rates were significantly improved in the intervention group. We recommend REA as a potential modality to improve informed consent comprehension and quality of informed consent process in low resource settings.

Background

Informed consent comprehension is an important determinant of study compliance and retention [1]. Quality in the informed consent process include adequacy of the information provided; understand-ability of the purpose, benefits and risks of the research; distinction between research and clinical care; voluntariness of participation; recall of signing a consent document; and satisfaction with the consent process as well as the levels of recall and understanding by study participants [2]. Important parameters in monitoring the quality of research implementation in addition to levels of study recruitment include levels recall and understanding of the informed consent information by study participants [3–7]. To this effect, informed consent documents may require to be adapted to the local culture and the educational level of the population, which have been shown to strengthen research efforts, through improved recruitment and retention of participants who better understand their roles and responsibilities [8–10].

Rapid assessment techniques such as Rapid Ethical Assessment (REA) have been documented to play an important role in improving informed consent process for medical research in developing countries by providing better understanding of research contexts [11–14]. REA is a brief (an average of 6 weeks long) qualitative intervention designed to map the ethical terrain of the research setting prior to a research team starts recruiting participants with the purpose of connecting ethical principles to contexts and realities on the ground. REA attempts to discover, describe and respond to the ethical issues specific to a particular research setting, and help researchers to address the issues that genuinely matter to proposed study participants and their community. REA methodology involves interviews, focused discussion and observation conducted among key stakeholders to inform the design of the particular research project. Its findings are utilised to inform and guide the research consent process; ranging from the conception and development of the consent form, to the way consent is obtained. It has been documented that REA is considered relevant and of utmost use for researchers in low income settings [15]. However, the effects of REA on study participants’ level of comprehension, recruitment and retention and quality of informed consent have not been documented.

The current study investigated effects of REA on informed consent comprehension, study compliance and quality of informed consent in a low-income setting. We measured consent information comprehension level, recruitment rate, retention rate and quality-of-informed consent among the participants enrolled into another research: “HPV-Subtype Prevalence Study” [16] in the Tigray Regional State of Northern Ethiopia. The HPV study was identified for the REA intervention due to potential ethical issues anticipated in the study.

Methods

Study design

A cluster randomised controlled trial was conducted among participants of an HPV Sero-prevalence study in Northern Ethiopia, from July 8 to August 23, 2013. The intervention for the trial was conducting REA and subsequent revision of the informed consent's content and processes based on the REA findings. The intervention and control groups were compared for levels of comprehension, recruitment, retention (compliance) and perceived qualities of the consent process two weeks after initial consent. More description on the measurements is given in the data collection procedures section below.

Source and study population

Pregnant women between 18-45 years attending ANC follow-up in four selected health facilities and who were targeted for the HPV sero-prevalence study were both the source and study population for this comprehension assessment. Those with communication challenge, severe illness, gestational-age greater than 34 weeks, or those who were not willing to return after two weeks for the follow-up interview were excluded.

Sample size determination

Sample size was determined using the sample size formula for double-population proportion -

$$ \frac{\mathrm{n} = {\mathrm{P}}_1\Big(1 - {\mathrm{P}}_{1\Big) + }{\mathrm{P}}_2\left(1-{\mathrm{P}}_2\right)\ \mathrm{X}\ \mathrm{f}\ \left(\upalpha,\ \upbeta \right)}{{\left({\mathrm{P}}_2 - {\mathrm{P}}_1\right)}^2} $$

where ;

n : number of participants per group

p₁ : expected levels of comprehension in the control group

p₂ : expected levels of comprehension in the intervention group

α : degree of error

β : power of the study

Level of comprehension (P₁) for the control group was estimated based on an earlier study (hence P₁ = 0.75) [3]. We assumed an improvement in the level of comprehension of 13 % (P₂ = 0.88) for the intervention group with 80 % power (β = 0.80), and 95 % confidence interval (α = 0.05) (hence A(α, β) = 7.85). After adjusting for 10 % non-response rate, the total sample size required was 300 participants with a 1:1 ratio between intervention and control participants (i.e. 150 in each group).

Sampling procedure

Two woredas (districts) targeted for the HPV sero-survey were selected purposively based on comparability. The selected woredas were randomly assigned using a lottery method to either intervention or control. Health centres within each of the woredas were shortlisted based on: availability of Ante Natal Care (ANC) service provision, laboratory services, and appropriate professionals to collect and handle specimen and flow of adequate number of ANC clients. Two health centres from each woredas (i.e. a total of four health centres), were randomly selected from the shortlisted health centres (Fig. 1). Sample size was proportionally allocated to the health facilities based on reported flow of ANC clients in the previous one year reported by the respective health bureau at the district level.

Fig. 1

Flow-Chart showing the sampling procedures for the study. Illustrates the sampling procedures of the study and the steps followed in the recruitment of study subject for the study. Recruitment of the study participants followed the REA (Rapid Ethical Assessment) which was conducted at the very beginning. The study participant were in total 300, with 150 in the intervention and 150 in the control arms. They were selected from two health centres in both instances and the number selected were proportional to the turn-out of patients in each health institution

Data collection procedures, instruments and variables

REA conducted at the beginning of recruitment identified a number of ethical considerations. Based on these findings revisions were suggested on (a) the consent form and information sheet and (b) the consent process and procedures (Table 1). Accordingly the IRB-approved consent form (Additional file 1) was revised for the intervention group (Additional file 2). The revisions made were: use of additional local terminologies, concrete explanations of study information with examples, and contextual clues based on the REA findings. Consent information was provided by reading the information sheet to participants in both groups. In the intervention group, this was accompanied by additional narratives which were attached to the consent document based on the issues identified by REA. Accordingly participants were asked whether they had any questions after reading each section of the form, and once more after the end of the entire reading. Consent in the control group, was obtained by signing on the consent form, but verbally in the intervention group with data collectors signing as witnesses. Sentences and paragraphs in the revised version of informed consent form were relatively short, and the words chosen were more familiar and local. The major characteristics of the two versions are summarized in Table 1.

Table 1 Comparison of 'standard' and 'revised' versions of information sheet, consent forms and procedures

Level of comprehension of informed consent, and the quality of the consent process were assessed two weeks after consent had been taken. We used Modular Informed Consent Comprehension Assessment (MICCA) and Brief Investigator Questionnaire (BIQ) [17] to measure informed consent comprehension of participants; and Quality of Informed Consent (QuIC) [18] to assess quality of the informed consent process. The MICCA and BIQ instruments for our assessment were developed by adopting the existing generic tools (Additional file 3). Both tools have the flexibility to test comprehension specific to a particular trial, yet can be utilized across a variety of trials. MICCA measures comprehension by incorporating both generic and trial-specified approaches. Psychometric evidence suggests that MICCA can be utilized in various trial settings and can produce reliable and valid scores [17].

Comprehension was assessed as a function of understanding and recall. Understanding refers to the concepts explained in the consent process such as purpose, benefits and risk. Recall refers to issues that needed to be memorized such as date of appointment, and specific procedures such as signature, copy of consent given or not [19] (Table 2). A total of 25 questions were used to assess informed consent comprehension of participants; 13 questions were used to assess levels of understanding and 12 questions to asses levels of recall. Test items consisted of both generic and trial specific questions (Additional file 3). Of the 25 test items, 14 were generic test items that appeared on each version of MICCA, 6 were trial-specific test items which were generated based on responses to BIQ and the remaining 5 were trial-specific test items in which the response options for each of the test items are generated based on response to BIQ. QuIC consisted of 13 subjective questions on the perceived quality of the informed consent process (Additional file 3). Each correct response was scored as 1, while each incorrect response was scored 0. For multiple choices with more than one correct response a score value of 1 was given for each possible answer. All the correct answers for understanding and recall were separately summed and calculated out of hundred.

Table 2 Operational Definitions

MICCA, BIQ and QuIC-based questionnaires for the study were initially prepared in English (see Additional file 3) and then translated to Tigrigna, the local language. The Tigrigna version was translated back to English to check for consistency of meaning. Data collectors who recruited participants and obtained initial consent were nurses from the respective health centres, who had been trained for the study. Comprehension assessment was done by data collectors who had been trained on the use of the assessment tools. Data collectors were supervised by public health experts with postgraduate qualifications.

Data analysis

Data were pre-coded and entered into Epi-Info version 3.5.1. Data were cleaned and edited using simple frequencies and cross tabulation and exported to SPSS Version 16 for further analysis. Descriptive statistics were generated for describing the study population in relation to relevant socio-demographic variables. Regression analysis were done for comparative analysis and assessing association between the major outcome variables and associated factors such as educational status, occupation and previous participation in research. Risk ratios with 95 % confidence interval were used to determine levels of significance. Multivariate logistic regression model was used to control for confounders and adjust the measures of association.

Results

REA findings

Rapid Ethical Assessment conducted at the beginning of recruitment identified a number of ethical considerations, based on which the consent form and process for the intervention group were revised. Major REA findings included low awareness of the concept of research (therapeutic misconception), presence of indigenous descriptions and terminologies for technical disease-related terms; concerns about biological samples; major discomfort around signing a consent form; discomfort with male enumerators; suspicions around the study selection criteria; concerns with outsiders coming to the community with incentives; and concerns over test results and their implications. These findings dictated the need for further elaboration on some of the components of the research, for extended discussion on the issues identified, and for changes in the consenting mechanism.

Socio-demographic and economic characteristics of study participants

A total of 300 pregnant women were approached for the study, each half from control and intervention locations. Of the 254 who consented (133 from intervention and 121 from control sites), 199 (114 from intervention and 85 from control site) also came to the follow-up visit after two weeks and participated in the comprehension assessment.

The mean age (±2 SD) of the participants who complied with the follow-up appointment was 27.3 (+/-6.3) years. Majority (83.4 %) were married; from the Tigre ethnic group (99.5 %); Orthodox by religion (86.4 %); able to read and write (70.8 %); and had a monthly family income of >1000 Ethiopian Birr ETB) (61.8 %) which is equivalent to 50 USD. For most participants (72.9 %), it was the first time they had participated in any medical research (Table 3).

Table 3 Demographic and economic characteristics of respondents of WukiroKilte-Awulaelo and Hintal-wajiratworedas (districs), Tigray region, Northern Ethiopia, July 2013

Recruitment and retention rates

The overall study recruitment rate was 84.7 % with 88.7 % in the intervention and 80.7 % in the control groups (p = 0.05). The overall study retention rate was 78.3 % with 85.7 % for the intervention and 70.3 % for the control groups (p < 0.005). The loss-to-follow-up rates after initial consent were 14.3 % and 29.8 % in the intervention and control groups respectively.

Informed-consent comprehension scores

In the control group, majority (77.6 %) had low comprehension score (50 % or less). Only 4.7 % had high comprehension (75 % or greater) score. The remaining 17.6 % had medium comprehension scores (between 50 % and 75 %). In the intervention group, 43 % of participants had high comprehension scores (75 % or greater), only 8.8 % had a low comprehension score (50 % or less) and the remaining 48.2 % had a medium score (between 50 % and 75 %). Participants in the intervention group were 4.6 times more likely to have medium comprehension scores and were 14.5 times more likely to have high comprehension scores than participants in the control group [Table 4]. The mean comprehension scores of the two groups (intervention and control) for all the 25 questions used in the comprehension test are presented in Table 5.

Table 4 Level of informed consent comprehension in the intervention and control groups, WukiroKilte-Awulaelo and Hintal-wajiratdistrics, Tigray region, Northern Ethiopia, July 2013

Table 5 Mean informed consent comprehension score (CS) of participants, Tigray region, Northern Ethiopia, July 2013

Participants in the intervention group obtained an average score of 73.1 % in overall comprehension; 73.2 % in the recall and 73.0 % in the understanding sections. Participants in the control group obtained an average score of 45.2 % in the overall comprehension; 44.8 % in the recall and 45.6 % in the understanding categories. There were statistically significant mean differences in all the three categories between the intervention and control groups (p < 0.001). There was a significant net difference in the mean score comprehension in almost all of the components in the intervention group compared to the standard. The highest mean difference (42.7 %) was observed in the understanding of participant rights. There was no statistically significant mean score difference in disease-related information between the intervention and control groups [Table 6].

Table 6 Comparison of percentage of participants giving correct responses to the main consent components in the intervention and standard group, WukiroKilte-Awulaelo and Hintal-wajiratworedas(districts), Tigray region, Northern Ethiopia, July 2013

Further analysis was done for potential socio-demographic factors associated with comprehension irrespective of REA. Participants with higher educational level (secondary and above) were found to be about 6.7 times more likely (AOR = 6.68, 95 % CI 1.01 to 44.03) to obtain high scores (> = 75 %) in the comprehension assessment than those with no formal education. Participants who had ever participated in any medical research were 2.8 times more likely to score high (AOR = 2.77, 95 % CI 1.01 to 7.58) than those who had never. Occupation and income was also found significantly associated with high score of comprehension [Table 7].

Table 7 Variables associated with comprehension status among study participants in WukiroKilte-awulaelo and Hintalo-wajirat districts,Tigray region, Northern Ethiopia, July 2013

Quality of informed consent

Overall, the mean informed consent quality assessment score for the control group was 78.5 %, and that for the intervention group was 89.1 % [Table 8]. More than eighty percent (81.2 %) in the control and 99.1 % in the intervention group were satisfied with the overall consent process, while a statistically significant mean difference in the overall mean quality of informed consent scores between the intervention and control groups. Statistically significant difference was detected in all but three components of quality assessment [Table 8].

Table 8 Comparison of mean quality of informed consent process (QuIC) score from the participant’s perspective by intervention group in WukiroKilte-Awulaelo and Hintal-wajiratdistricts, Tigray region, Northern Ethiopia, July 2013

Discussion

The study demonstrated the possible impacts of REA on study retention rate, levels of comprehension and quality of the informed consent process among study participants in a low-income setting. REA conducted prior to recruitment of participants revealed a number of relevant and context-specific ethical issues concerning the HPV study and its targeted study population. These findings helped in adapting the study information sheet, consent form and consent procedures to the local culture and educational level of the population. The HPV study was selected for the REA intervention due to expressed interest from the investigators and expressed concerns in relation to the sensitivity of the issues as the study deals with cancer of the reproductive organ and sexually transmitted infection. Such parameters which suggested that REA will be relevant as the study was expected to generate ethical issues related to the sensitivity of the subject, ambiguity of terms, and decision-making dynamics of the community. The study was conducted in a well-defined population group with relative homogeneity in language, ethno-cultural and geographic parameters. The two study sites (intervention and control) were similar to each other on major societal parameters. They were geographically adjacent to each other and belonged to the same ethno-cultural cosmos including the health system. The socio-demographic characteristics of study participants in both locations revealed overall similarity except few irregularities. We employed MICCA and BIQ for measuring comprehension and QuIC for assessing quality of informed consent. The tools were able to capture both standard informed consent items as well as study-specific test items [17]. Certain allowable practical adjustments were done on the tools and some of the components, otherwise both tools have been validated for use by researchers in different settings [17].

Recruitment rate and the recruitment index are considered important parameters in monitoring the quality of research implementation [6, 7]. In this study higher rates of both recruitment and retention were documented for the intervention group. While we documented strong statistical association between study retention rate and use of REA-based revision on informed consent process, no statistically significant difference was documented between the recruitment rates of the two groups. Previous studies have tried to identify potential contributors to recruitment rates of study participant such as 'opt-out' approaches [20], early consideration of participants' perspectives [21], addressing ethical issues such as the therapeutic misconception [6], and establishing trust between researchers and potential participants [22]. The recruitment rate, however, may have been affected by other factors beyond the consent information as the complexities of the challenges of recruitment are also determined by the complexities of the respective studies [7, 23, 24] and decisions being made prior to recruitment based on information circulating in the community [25].

Recruitment rate has been documented as an important determinant of subsequent retention [6, 24]. Other factors affecting retention include mental state of the person [26], availability of tracking and follow-up mechanisms [27], availability of mechanisms addressing community and context-specific issues and needs in special communities [28]. However every study has its own peculiar features and it may be difficult to generalise these factors. The registered improvement in the study retention rate in our study may have resulted from improved understanding of the consent information.

The current study documented significant differences in the comprehension levels (understanding and recall) between the intervention and control groups. REA-based adjustments to the content and delivery of consent information in the intervention group were linked to the improved levels of understanding and recall in the intervention group. These findings concur with studies which have reported improved consent information comprehension with appropriate interventions such as provision of more explanation and socio-culturally tailored approach [29]. Provision of tailored terminologies and additional narrative explanations on the pre-identified ethical issues and allowing more time for discussion and questions from the participants in the intervention group might have been potential determinants. There was no significant difference in comprehension levels related to disease information between the intervention and control group. The reasons for non-significance could be the sensitivity and novelty of the issues of cervical cancer in the area.

In the study there was a significant difference in the overall levels of perceived quality of the consent process in the intervention compared to the control group. There was a statistically significant difference in all but three of the components of quality assessment; whether consent form was read and explained carefully; whether consent from was important source of information; and whether consent form was important for consent decision. Possible reasons for the improved perception of the quality of the consent process in this study include the improvements made in the consent information and the revised consent procedures both of which were geared to the needs and concerns of the potential participants, based on the REA. There are several possible reasons for lack of statistically significant improvements in three areas. Regarding readability, the participants in the intervention groups mainly had the form read to them rather than reading it themselves. Another reason is that all three measures already scored highly in the control group with little room for improvements. Though not significant, there were improvements observed in the three categories and if a larger sample had been used this might have been significant. Quality of informed consent depends on factors such as type of consent information provided, amount of information, adequacy of comprehension, and the voluntariness of decision making by the participants [5]. In the clinical set up, participant-related factors such as age, IQ, level of cognitive function and external locus of control were associated with poor quality and information recall. Written information provided immediately before admission was associated with better outcomes [30].

Various researchers have utilized different interventions to improve the consent process and understanding, such as use of local narratives [31], extended one-to-one discussion [32–34], enhanced consent [33, 35], a revised consent process [36], and a booklet of participants’ rights [37]. Cultural and linguistic modifications to the informed consent process were considered to significantly enhance understanding by study participant in low income settings [29]. On the other hand it has been shown that lexicosyntactic readability improvement of the consent form [38], multimedia enhanced consent [34], and use of a concise version of the consent form [39] do not have effects on comprehension. There are still ongoing evidence-based debates as to which interventions are effective in enhancing comprehension [40]. The intervention in this study involved improvements changes in the consent procedures based on REA findings.

One of the ethical dilemmas faced by the researchers while revising the consent form is what level of change on an IRB-approved form would be appropriate and allowable once in the field, and what principles should govern this. According to existing guidelines, any amendments of an approved research project require further IRB approval before implementation [41, 42]. The only exceptions to this are if the changes are immediately and urgently required for the safety of the subjects. In this case, the IRB must be informed immediately after establishing the need for the changes -in most cases within five business days [43]. The other possible exception to this rule is if the study is a determined to be non-research by the IRB and does not need further IRB follow-up [41]. We made revisions to the consent form in the field on the following basis; a) that the revision added to the approved consent and did not reduce any major component (the consent form content and structure approved by IRB were maintained with improvements); b) that the comprehension assessment project and the possibility of revising the consent form were already approved by IRB; c) that the changes were important for the welfare of the subjects in the study. Since getting approval from IRB for the modification might take time, some provision for making appropriate adjustment without tampering with the basic components of consent may be necessary in future.

The Ottawa Statement provides a framework for addressing ethical issues around cluster randomised trials. According to the statement on the Ethical Design and Conduct of Cluster Randomized Trials, "When participants’ informed consent is required, but recruitment of participants is not possible before randomization of clusters, researchers must seek participants’ consent for trial enrollment as soon as possible after cluster randomization—that is, as soon as the potential participant has been identified, but before the participant has undergone any study interventions or data collection procedures" [44, 45]. In the current study consent was obtained soon after recruitment in to the planned study without further additional approval of the REA based revisions on the originally approved consent form, based on the premises presented above.

The study has some potential limitations. Efforts were made to ensure the uniformity of the intervention and control sites, however there may have been undocumented differences in the settings which may have confounded the study outcomes. Both levels of comprehension and quality of the informed consent process are based on participant perceptions using questionnaire based interviews which were not accompanied by independent observation methods. This might reduce the objectivity and validity of the responses. Reasons for loss to follow-up of those who did not attend were not documented, and neither did we document the basic socio-demographic characteristics of non-attendees. The sample size determination was done based on assumptions from other countries as there was no local estimate available. In addition we assumed a 10 % non-response rate while the non-response rate for the study was about 15 %. The single cluster intervention also might have reduced the study power.

Conclusions

Based on our findings, REA is associated with improved understanding of study concepts, purposes and procedures and participants’ perceptions of the informed consent process. Investing in an intervention which enhances researchers' understanding of the local context and tailoring the consent information and process is associated with improvement s in overall consent comprehension, study recruitment rates and retention rates in low-income settings. REA is also an appropriate intervention for further use in similar settings and situations.

We would like to encourage researchers and research ethics committees and policy makers to employ REA to improve research consent comprehension, recruitment and retention levels in certain low income settings. As this is the first study exploring the effects of REA on comprehension, additional studies in other settings with more objective methods of measurement would provide greater insight into the overall effects of REA on comprehension of consent. Such studies might even expand into consent processes for medical interventions. Ethicists and research policy makers are encouraged to explore the consequences of utilizing REA in the light of IRB requirements and operating procedures. In future studies, if REA is planned, it should be included in the research proposal, so it and any subsequent consent form modification are approved by the IRB.

Abbreviations

ANC, Ante natal care; BIQ, Brief investigator questionnaire; HPV, Human papilloma virus; IRB, Institutional review board; MICCA, Modular informed consent comprehension assessment; QuIC, Quality of informed consent; REA, Rapid ethical assessment; SPSS, Statistical package for social sciences