Keywords
Observational studies, health equity, research reporting, research methodology, study design
This article is included in the Sociology of Health gateway.
This article is included in the Health Services gateway.
This article is included in the Research on Research, Policy & Culture gateway.
This article is included in the Equitable Cancer Care collection.
Observational studies, health equity, research reporting, research methodology, study design
Less than a decade remains until the 2030 deadline set by the United Nations to meet the 17 Sustainable Development Goals1 aimed at global economic, social, and environmental transformation. One of the 17 goals, and a cross-cutting theme across several others,2 is health equity, defined as the absence of differences in health that are avoidable and unjust.3 The achievement of health equity as a priority has been widely endorsed by several international organizations4,5 and resonates with the moral imperative of social justice and recognition of human rights principles.
A major challenge is the lack of evidence that includes thoughtful, rigorous collection of health equity data to inform program development and policymaking.6 In fact, global organizations such as the World Health Organization (WHO) and the Pan American Health Organization (PAHO) have called for the improvement of equity considerations in the reporting of research.7–10 Thus, the integration of health equity in research is needed in both primary and secondary research.11–14 In primary research, observational studies are particularly well-suited for investigating health equity as they often include populations at risk of experiencing inequities.15,16 Observational studies are analytical or descriptive studies evaluating a research question without changing exposure to an intervention.17 In the absence of a control group, they can be descriptive in nature but do not permit the investigation of causal associations.18 Observational studies predominate in health-related research,19,20 and are used to investigate numerous types of research questions, such as detecting rare or late adverse effects of treatments and to explore complex systems.21 Furthermore, observational studies have highlighted the vast inequities in society during the COVID-19 pandemic,22 and simultaneously, have had an important role in informing public health responses.23,24
The well-known STROBE (Strengthening Reporting of Observational studies in Epidemiology) reporting guideline, which is intended to be used for the reporting of observational studies, is used by over 60% of authors of observational studies.25 However, despite widespread use of the STROBE reporting guidelines, there is a persistent lack of integration and reporting of sex and gender (among other dimensions) in observational studies.26–28 Comprehensive reporting in research is essential to assess its reliability, reproducibility and methodological rigour, and ensure that the research meets the needs of the potential users.29 Using reporting guidelines contributes to meeting these goals by increasing the completeness and transparency of research papers.30–33 The inadequate reporting may be, in part, due to lack of guidance on reporting equity items in such studies.
Complex health systems questions and addressing social determinants of health may require multi-stakeholder and intersectoral collaboration and engagement, for which equity perspectives can serve as a linchpin.34 Observational studies also allow for addressing questions on health policy, health systems, and health services organization, delivery, prioritization, and implementation.35–37 Different frameworks are available to describe factors associated with health equity.38,39 The Campbell and Cochrane Equity Methods Group, for example, proposes using the PROGRESS-Plus framework. This acronym is widely used and stands for Place of residence, Race or ethnicity, Occupation, Gender or sex, Religion, Education, Socioeconomic status, and Social capital.39 The “plus” accounts for additional characteristics that could result in inequitable health outcomes. Although the reporting on equity in observational research would increase the value of research outcomes through the recognition of how these dimensions impact interventions, the application and use of equity frameworks in reporting health research are not yet standard practice. Therefore, our objective is to systematically evaluate how equity-relevant observational studies describe characteristics of their samples, design features and analysis, and interpretation of their findings across PROGRESS-Plus; we also aim to use the findings of this study to inform the development of an equity reporting guideline.
Figure 1 outlines the overarching process to be followed in the conduct of this study; from identifying relevant studies at title and abstract stage to data extraction and analysis.
We will include health equity-relevant observational studies from all settings, published between 1 January 2020, and 27 April 2022. We define health equity, health equity-relevant studies and observational studies in Table 1. We chose to focus on equity-relevant studies, since these are most likely to have methods of describing populations experiencing inequities, and are most likely to include methods to assess similarities or differences in effects for populations experiencing inequities.
Term | Definition and explanation |
---|---|
Health inequity | Health inequity is defined as avoidable and unfair differences in health.3 |
Health equity-relevant studies | We define “health equity-relevant” as studies that focus on individuals or populations experiencing inequities or studies of mixed populations that analyzed at least one health outcome across one or more PROGRESS factor.12,40 We define health outcomes according to the WHO definition, as “changes as in health status that result from the provision of health services”.41 This includes studies focused on populations at risk of experiencing inequities such as people who are homeless and asylum seekers, as well as studies which present disaggregated results or evaluate outcomes across one or more PROGRESS factors. However, studies which control for PROGRESS in analyses without analyzing differences across the factors are not considered equity-relevant. For example, studies of breast cancer in women would not be considered health equity-relevant since women are not disadvantaged in opportunities for health as it relates to breast cancer care. However, if stratified analysis was done to compare mortality of women living in rural vs urban area (or other PROGRESS factors), the study would be considered equity-relevant. Similarly, a study of low-income or racialized women accessing breast cancer care would be considered health equity-relevant because they experience different opportunities in obtaining care. |
Observational studies | We define observational studies as analytical or descriptive studies evaluating a research question without changing exposure to an intervention.17,18 Observational studies are classified in the STROBE reporting guideline into three types: 1) cohort: following an exposed population over time; 2) case-control: comparing exposures between people with a particular disease outcome (cases) and people without that outcome (controls); and 3) cross sectional: assessing all individuals in a sample at the same point in time.21 Studies conducted using routinely collected data stored in administrative datasets can be classified into these three types of studies.42 |
We will not include studies focused on Indigenous populations in this evaluation, as they will be assessed in a separate but complementary study led by Indigenous researchers as it will require a different sampling approach and methodological assessment. We decided not to use “Plus” criteria for eligibility since they require a greater deal of judgement regarding the contextual factors that may result in inequities. We will exclude case series and case reports since they do not include a reference population or comparison group enabling assessment of differences across PROGRESS factors.
Based on consultation with our interdisciplinary team of researchers and decision-makers, and patients and public, we will use a three-factor, randomized sampling strategy to balance: 1) studies in high-income countries (HIC) and low- and middle-income countries (LMIC) according to the World Bank classification, 2) studies focused on COVID-19 and those which are not, 3) studies focused on populations experiencing inequities and those that stratify their analyses. We chose to stratify by country because of the importance of the context and resources in LMIC settings which affect health inequities both among LMIC and between LMIC and HIC,43–45 Given the evidence on exacerbation of inequities during the COVID-19 pandemic, we decided to ensure balance across studies related to COVID-19 and those that are not.46
Although we chose only three factors for the sampling strategy, we will assess all aspects of PROGRESS-Plus in the included studies; thus other populations experiencing inequities will be included, such as people who are living with low income, homeless, migrants, asylum-seekers, and racialized individuals. Table 2 lists examples of studies that meet our eligibility criteria.
LMIC | HIC | |||
---|---|---|---|---|
COVID+ | COVID- | COVID+ | COVID- | |
General population-i.e. mixed | Epidemiological characteristics of patients with severe COVID-19 infection in Wuhan, China: evidence from a retrospective observational study47 | Effectiveness of seasonal malaria chemoprevention at scale in west and central Africa: an observational study48 | Patterns of use of secondary mental health services before and during COVID-19 lockdown: observational study49 | School Entry, Educational Attainment, and Quarter of Birth: A Cautionary Tale of a Local Average Treatment Effect50 |
Focused on populations experiencing inequities | Ethnic and regional variations in hospital mortality from COVID-19 in Brazil: a cross-sectional observational study51 | Demystifying the factors associated with rural–urban gaps in severe acute malnutrition among under-five children in low- and middle-income countries: a decomposition analysis52 | Hesitant or Not? The Association of Age, Gender, and Education with Potential Acceptance of a COVID-19 Vaccine: A Country-level Analysis53 A case-control and cohort study to determine the relationship between ethnic background and severe COVID-1954 | Health Care Utilization Before and After the “Muslim Ban” Executive Order Among People Born in Muslim-Majority Countries and Living in the US55 |
We will randomly select a sample of 320 equity-relevant observational studies, since assessing all potential eligible studies would be unmanageable for our review team. To ensure we assess a sample that provides comparable results, we conducted sample size calculation for binary outcomes (categorical questions for consideration of equity in the studies) with 95% confidence intervals of ± 6% for observed proportions of 50% (i.e. assuming that half of the studies would report at least one PROGRESS characteristic). This sample size has also been used in similar studies.56–59 We acknowledge that the study does not have sufficient power to allow a comparison of reporting across the three prioritized groups, so we will only report data descriptively. Eligible studies will be exported into Microsoft Excel and sorted using a random sequence using the built-in random-number generator. We will sample the studies as specified in Table 3.
We have designed a search strategy in collaboration with a librarian (TR) experienced in systematic reviews, using a method designed to optimize term selection.60 A validated search filter to identify equity-focused studies was adapted for use in this study,61 and additional terms to increase retrieval of papers about indigenous populations were added (lines 213-232).62,63 These were developed in collaboration with our Indigenous Steering Committee. Studies related to Indigenous populations will be identified from these search terms and will be utilized to conduct an independent but highly complementary study focused on Indigenous populations.
We used a validated search filter for observational studies with two extra terms to retrieve cross-sectional studies.64 We further added the terms “Instrumental variable”, “discontinuity design”, “interrupted time series”, “discontinuity design”, “matching”, “synthetic control” and “difference-in-difference” to our search strategy to ensure that we capture observational econometric studies that meet our eligibility criteria. Review articles and randomized controlled trials will be excluded from these results using validated study design filters for Ovid MEDLINE.65 The draft search strategy is stored in the Open Science Framework (OSF) repository (Extended data67). We will test the search strategy with a set of 10 studies that we identify to be eligible.
We will restrict our search to the last two years because we are interested in describing current analysis and reporting of equity, not trend over time. This period also captures the girth of the literature published throughout the pandemic. We conducted a preliminary search of Ovid MEDLINE for papers published between 1 January 2020, to 17 April 2022 and obtained 16,263 citations. Due to the high volume of results retrieved, we decided that searching only one database would be feasible. In the conduct of this study, we expect an adequate number of observational studies in this period to meet our eligibility criteria.
Studies will be screened at title and abstract level by a single investigator using DistillerSR software.66 One investigator will screen the full texts of studies, and OD will validate. Conflicts will be resolved through discussion with a third reviewer at weekly meetings. We will use the DistillerSR randomization function to randomly sample citations at a set interval of 20% of unreviewed citations until we reach the planned sample size.
At full-text screening, we will tag eligible studies according to our prioritized topics (Type of equity relevance, country income setting, COVID-19 research).
Data will be extracted independently by two extractors. We will develop and pretest a data extraction form using DistillerSR. We will conduct training on the data collection with data extractors using sets of three to five studies at a time, until there is 80% agreement across all items between extractors. Conflicts will be resolved by discussion between extractors and reaching consensus. We will develop a data dictionary to ensure consistent extraction across team members and refine during pre-testing.
We will capture study characteristics including the purpose of the study (focused on populations experiencing inequities or whether the study included general populations in which there may be populations experiencing inequities), title/abstract, background/rationale, study design, population characteristics, results, analyses, interpretation of applicability and discussion. We will collect details on the motivation behind studies for a separate evaluation of the way equity is defined in these studies and the motivation for studying equity-seeking populations. The complete list of items is provided in our OSF repository (Extended data67).
We will assemble the author contact details for a separate survey of authors (of the included studies) to ask about engagement with populations experiencing inequities, since we believe that engagement with populations experiencing inequities informs better reporting on health equity. The reason for conducting the survey is because we expect that some of the results related to equity would not be well reported in the articles.
We will descriptively present the reporting of each item for the observational studies in our sample by tabulating the data for each item to assess the frequency of reporting. We will conduct subgroup analysis on studies that considered health equity as a reason for conducting the study.
The primary objective of this study will be to describe the reporting and analysis of equity in equity-relevant observational studies. By focusing on studies that meet this criteria, we will document studies that have missed opportunities for providing information regarding populations at risk of experiencing inequities. Moreover, this exercise may help us identify examples of equity considerations in observational studies, according to our candidate items for the equity extension for STROBE reporting guideline, to illustrate how and to what extent they are presented in the literature.
Our study is not without limitations. First, our search of only MEDLINE would be expected to yield studies mostly from HIC and focused on clinical topics. However, we plan to mitigate this selection bias by purposefully stratifying sampling across country income setting and randomly selecting a sample from the eligible studies. Second, although a third of early COVID-19 literature has been shared as preprints, we will not include studies from pre-print servers as studies in peer-review journal articles tend to show substantial improvements in the quality of reporting compared to pre-prints. Therefore, it may not be appropriate to pool preprints with peer-reviewed published articles. Future work could aim to investigate the difference in reporting of health equity in articles published in pre-print servers compared to peer-reviewed articles.
The proposed list of extraction items is extensive and provides several gateways for future projects to fill gaps in understanding the integration and reporting of equity in research. Using the results of this methodological assessment, we plan to conduct a sociological discourse analysis of motivation for assessing health equity in equity-relevant observational studies. In accordance with the plan for this extension, we will conduct an independent study on Indigenous research, led by Indigenous researchers, to identify and describe well-reported studies using the CONSIDER checklist. Furthermore, the selective sampling across prioritized groups provides an opportunity to conduct more focused investigations, tailored to each respective topic (i.e. LMIC, COVID-19 research).
Open Science Framework: STROBE-equity reporting guidelines, https://osf.io/cp3z2.67
This project includes the following extended data:
Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).
We would like to acknowledge our partners and collaborators on the STROBE-Equity team.
Indigenous Research Steering Committee: Catherine Chamberlain (Australia), Sarah Funnell (Canada), Billie-Jo Hardy (Canada), Matire Harwood (Aotearoa New Zealand), Michelle Kennedy (Australia).
Patients and Public Steering Committee: Holly Ellingwood (Canada), Regina Greer-Smith (USA), Clara Juando-Prats (Canada), Janet Jull (Canada), Bev Shea (Canada), Janice Tufte (USA).
Knowledge Users Steering Committee: Marc Avey (Canada), Luis Gabriel Cuervo (USA), Tanya Horsley (Canada), Tamara Kredo (South Africa), Elizabeth Loder (USA), Melissa Sharp (Ireland), Laura Weeks (Canada), Howard White (UK).
Technical Oversight Committee: Elie A. Akl (Lebanon), Alba Antequera (Barcelona (Spain)), Zulfiqar A. Bhutta (Canada), Peter Craig (Scotland), Omar Dewidar (Canada), Alassane Dicko (Canada), Cindy Feng (Canada), Isabel Fortier (Canada), Damian Francis (USA), Elizabeth Ghogomu (Canada), Janet Hatcher-Roberts (Canada), Alison Krentel (Canada), Elizabeth Kristjansson (Canada), Laurenz Langer (South Africa), Daeria O. Lawson (Canada), Julian Little (Canada), Olivia Magwood (Canada), Michael Johnson Mahande (Tanzania), Lawrence Mbuagbaw (Canada), G.J. Melendez- Torres (UK), David Moher (Canada), Ainsley Moore (Canada), Loveline Lum Niba (Cameroon), Stuart G. Nicholls (Canada), Miriam Nkangu (Canada), Stephen G. Noorduyn (Canada), Ekwaro Obuku (Uganda), Patrick M. Okwen (Cameroon), Tomas Pantoja (Chile), Jennifer Petkovic (Canada), Mark Petticrew (UK), Kevin Pottie (Canada), Tamara Rader (Canada), Jacqueline Ramke (Aotearoa New Zealand), Alison Riddle (Canada), Larissa Shamseer (Canada), Hugh Sharma Waddington (UK), Peter Tanuseputro (Canada), Peter Tugwell (Canada), Erik Von Elm (Switzerland), Harry Wang (Canada), Xiaoqin Wang (Canada), George Wells (Canada), Charles S. Wiysonge (South Africa), Luke Wolfenden (Australia), Taryn Young (South Africa).
All the team members have consented to be listed in the manuscript.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Is the rationale for, and objectives of, the study clearly described?
Yes
Is the study design appropriate for the research question?
Yes
Are sufficient details of the methods provided to allow replication by others?
Partly
Are the datasets clearly presented in a useable and accessible format?
Not applicable
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Health services research; Digital health equity; sexual health
Is the rationale for, and objectives of, the study clearly described?
Yes
Is the study design appropriate for the research question?
Partly
Are sufficient details of the methods provided to allow replication by others?
Partly
Are the datasets clearly presented in a useable and accessible format?
Not applicable
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Health services and systems research, evaluation and implementation science, mixed-methods research
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||
---|---|---|
1 | 2 | |
Version 1 06 Jun 22 |
read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)