Core competencies for scientific editors of biomedical journals: consensus statement

Abstract

Background

Scientific editors are responsible for deciding which articles to publish in their journals. However, we have not found documentation of their required knowledge, skills, and characteristics, or the existence of any formal core competencies for this role.

Methods

We describe the development of a minimum set of core competencies for scientific editors of biomedical journals.

Results

The 14 key core competencies are divided into three major areas, and each competency has a list of associated elements or descriptions of more specific knowledge, skills, and characteristics that contribute to its fulfillment.

Conclusions

We believe that these core competencies are a baseline of the knowledge, skills, and characteristics needed to perform competently the duties of a scientific editor at a biomedical journal.

Introduction

Scientific editors (editors are responsible for the content and policies of journals, and scientific editors are members of the team who contribute to that process by virtue of their scientific knowledge and experience) are responsible for deciding which articles to publish in biomedical journals [1]. A scoping review of the skills and requirements for scientific editors at biomedical journals carried out by some of the authors of this paper showed that most of the literature that contained recommendations on this issue was not research-based [2]. Rather, recommendations were documented in position papers and in guidance for members of editor organizations [3,4,5,6,7,8]. While many of these publications have offered perspectives on the knowledge, skills, and characteristics needed to be an effective scientific editor, there appears to be no consensus on which of these are fundamental to the scientific editor role. To our knowledge, no formal set of core competencies for this group has been established locally or globally. Our aim was to develop a minimum set of core competencies for scientific editors of biomedical journals.

Developing the core competencies

We used an integrated knowledge translation approach [9, 10] to engage stakeholders in a consensus-based process to develop a minimum set of core competencies for scientific editors of biomedical journals that was informed by a scoping review and editors’ perspectives. At the program outset, the team from the Centre for Journalology at the Ottawa Hospital Research Institute (JG, DM, KDC, and LS) assembled a core group of experts to represent scientific editing and publisher stakeholder groups. The experts included scientific editors from different parts of the world and various types and sizes of journals, editors-in-chief, and representatives from editorial organizations, biomedical journals, and publishers (Table 1). Our goal was to include diverse perspectives representing the spectrum of work involved in scientific editing.

Table 1 List of participating stakeholder groups

We followed a three-step process to develop the core competencies, which is followed by a fourth step to be implemented post-publication:

• 1. Pre-meeting activities (conduct scoping review and environmental scan; survey of editors’ perceptions/training needs; modified Delphi exercise)

• 2. Face-to-face consensus meeting (present results of pre-meeting research; hold consensus-based discussions)

• 3. Post-meeting activities (finalize competencies; solicit feedback from managing editors; survey editors for usefulness of competencies)

• 4. Post-publication activities (seek endorsement; plan for dissemination and implementation activities)

Pre-meeting activities

Scoping review and environmental scan

A subset of authors from the current publication (VB, PB, SB-S, KDC, JD, JG, PG, HM, DM, LS, SS, PT, EW, and MW) conducted a scoping review and environmental scan of the literature related to core competencies for scientific editors [2]. This included a review of the published and unpublished scientific and non-scientific literature that contained competency-related statements pertaining to scientific editors. They found a total of 225 full-text documents, 25 of which were research articles. From the 225 documents, they extracted a total of 1566 statements possibly related to core competencies for scientific editors of biomedical journals, which ultimately produced a list of 202 unique competency-related statements after de-duplication [2] (Fig. 1).

Fig. 1

Flow diagram for core competency development

Survey of editors’ perceptions and training needs

Another subset of authors from the current publication (VB, PB, SB-S, KDC, JD, JG, PG, DM, LS, SS, PT, and MW) engaged stakeholder organizations by inviting their scientific editor members to participate in an online survey of editors’ perceptions and their training needs [11]. The participants were respondents to advertisements seeking current or former scientific editors of journals. Advertisements for the research were sent to organizations having a large scientific editor membership (e.g., World Association of Medical Editors [WAME], Council of Science Editors [CSE], European Association of Science Editors [EASE], Cochrane), who forwarded an announcement about the survey to their membership. They collected demographic data and invited respondents to share their perceptions of the relevance of competency-related statements in their role as editors. They also asked respondents to share their perceptions of their own competence related to these statements. There were 38 statements, developed based on data collected in our scoping review [2] and from input from the publication’s authors. These statements were chosen to broadly cover major areas associated with the scientific editor role, including editors’ knowledge, expertise, skills, and experience. Finally, they asked respondents to create a ranked list of their training needs. A total of 148 participants from around the world contributed to the needs assessment survey. The ranked list of needs provided an additional 12 unique competency-related statements that were not previously included in the scoping review and environmental scan (Fig. 1). This provided valuable insight into the views and needs of scientific editors from different demographics and circumstances in the journal publishing landscape.

Modified Delphi process

A final subset of authors from the current publication (VB, PB, SB-S, KDC, JD, JG, PG, DM, LS, SS, PT, and MW) invited the respondents from the editor survey to participate in a three-round modified Delphi process to rate the importance of the 214 competency-related statements arising from the scoping review, environmental scan, and editor survey [2]. During the first round of the Delphi, they also invited participants to suggest any missing items, from which a further 16 unique items were found, bringing the total number of competency-related statements to 230. A total of 105 participants participated in the Delphi, with 27 of them completing one round, 20 completing two rounds, and 58 participants completing all three rounds. Their responses produced a list of 23 “highly rated” and 86 other “included” competency-related statements to help inform the decision-making process during the consensus meeting (Fig. 1). (The manuscript describing this process and the survey of editors’ perceptions and training needs [11].)

Face-to-face consensus meeting

In early June 2016, the Centre for Journalology group, in consultation with the other authors of the pre-meeting activities publications, assembled a group of 23 stakeholders in Strasbourg, France for a one-and-a-half-day meeting to work towards a minimum set of core competencies for scientific editors of biomedical journals. This group included nine stakeholders previously involved in the program (PB, SB-S, JG, PG, HM, DM, PT, EW, and MW) and 13 new stakeholders (SA, KB, JC, AG, KG, FH, SJ, DK, JL, AM, JM, JS, and GZ). The group was purposively sampled using snowballing principles; we invited our core group of experts to attend the consensus meeting and also asked them to contribute the names of other relevant editors (and others) who could potentially represent a range of perspectives, for example, due to their geographical location, size and type of journal where they work, experience with the publishing process, etc.). Participants were invited via a formal letter of invitation emailed by the lead author. We did not specifically solicit representatives of author and peer reviewer groups, as most of the consensus meeting participants were, or had been at one time, authors and/or peer reviewers and therefore could provide insight concerning these perspectives. The results of the scoping review and environmental scan, survey of editors’ perceptions and training needs, and modified Delphi were presented to the group. The presentation was followed by focused discussions on the 23 highly rated competency-related statements resulting from the Delphi, which were divided into four broad categories. Within these discussions, the group identified the competency-related statements that represented core competencies and suggested how to improve each statement. Other competency-related statements from the list of 86 included statements were also considered. Following these discussions, the selected core competencies were reviewed to determine whether there were any missing competencies. At the conclusion of the consensus meeting, the group emerged with a draft list of 24 core competencies (Fig. 1).

Post-consensus meeting activities

Finalizing the competencies

Following the consensus meeting, numerous email rounds of editing and feedback took place among consensus meeting participants (led by JG), stakeholders who did not attend the consensus meeting (KDC, JD, LS, and SS), and other stakeholders who were invited to the consensus meeting but were unable to attend (VB, LC, and TG). After removing redundancies and overlap between items, combining similar items, refining wording, and removing items after further discussion, the group finally arrived at a final set of 14 core competencies for scientific editors of biomedical journals (Table 2).

Table 2 Minimum set of core competencies for scientific editors of biomedical journals

External validation

We also asked two managing editors (one not involved in this initiative) to review the proposed competencies, and we incorporated their feedback into the refining process. The managing editor of The Journal of the American Medical Association (JAMA) and Jason Roberts of Headache: The Journal of Head and Face Pain are responsible for facilitating the peer review operations of their respective journals, the implementation of editorial policies and procedures, and ensuring that accepted manuscripts are formatted to fit the needs of the publisher.

Survey of editors on the usefulness of the core competencies

After reaching agreement on the final version of the competencies, we solicited the feedback of scientific editors from a small (Headache) and a medium-sized (Canadian Medical Association Journal [CMAJ]) journal. These editors were asked to take 2–3 weeks to consider and reflect on the relevance of the competencies in the context of their role as a scientific editor. Eight editors answered a short survey (hosted on SurveyMonkey.com) asking about the usefulness, aspirational qualities, and relevance of the competencies and whether any important competencies were missing. Their answers were generally supportive of the competencies as useful and relevant and somewhat mixed on their aspirational qualities. Two new items were suggested, which were later determined to already be included in the list of core competencies.

The core competencies for scientific editors of biomedical journals

Table 2 displays the final minimum set of core competencies for scientific editors of biomedical journals. It contains 14 key core competencies divided into three major areas. Each competency has a list of associated elements or descriptions of more specific knowledge, skills, and characteristics that contribute to the fulfillment of the associated core competency. These elements are meant to be illustrative examples of the key competencies rather than a comprehensive breakdown of the competencies.

Scope of the core competencies

We have made extensive efforts to produce consensus-based, end-user informed core competencies for scientific editors of biomedical journals that are driven by a scoping review and informed by end users. However, we acknowledge the limits of their scope as well. Specifically, we attempted to identify only the competencies that would be applicable across the entire spectrum of scientific editors of biomedical journals, regardless of journal size, type, geographic location, publishing model, or any other defining characteristic.

In some instances, there were important elements that we believed should be included, with the recognition that they may not apply to all scientific editors of biomedical journals in all situations. Therefore, we have inserted conditional language (e.g., “if applicable”, “as appropriate”) into some of the competency statements. It is possible that some other statements without the conditional language may also not be applicable or advisable given a scientific editor’s specific circumstances. In these cases, editors should act in the spirit of the competency instead of the literal description. Conversely, we also expect that each individual scientific editor position will potentially involve more competencies and/or elements than are identified in this list to address the particularities of the role and the characteristics of the journal. We encourage editors to identify these additional competencies and elements in order to complete the core competency profile for their particular role.

Since these core competencies are directed to editors of biomedical journals, they may not apply as well to editors in other scientific disciplines or domains outside of the scientific realm. It would be important to test these competencies with scientific editors in other fields to better understand their applicability in other disciplines. Although these core competencies can and should be applicable to those who hold the role of editor-in-chief, it is important to note that any competencies related exclusively to the editor-in-chief position were purposely removed from this list, as they do not necessarily apply to all scientific editors. We encourage other editorial groups and members of the editor-in-chief community to collaboratively create extensions to this list that address their more specialized role.

We took considerable care in crafting the specific language used to describe each competency, including trying to preserve the original language used in the scoping review, environmental scan, needs assessment, and modified Delphi, whenever possible. However, some of this language may be open to varying interpretations; thus, we hope to clarify any language issues in an upcoming explanation and elaboration document on each of the key competencies and their associated elements.

Post-publication activities

Endorsement

With the core competencies now established, we have begun the process of seeking a formal statement of endorsement from our stakeholders which will be used when promoting the competencies. At the time of submission, the competencies have been formally endorsed by Cochrane, EASE, and the Asia Pacific Association of Medical Journal Editors (APAME). The remaining editorial organizations on our stakeholder list (Table 1) are in the process of considering the core competencies for endorsement.

Dissemination

A subset of our authors will form a small committee that is tasked with developing a strategy to effectively disseminate the core competencies worldwide. At the time of publication, the core competencies have already been presented at the 2016 APAME conference and the 2017 CSE annual meeting and during an invited talk at the 2017 International Congress on Peer Review and Scientific Publication.

Implementation

Another subset of our authors will form a small committee to address how to best implement the core competencies. Editors, their publishers, and editorial groups who endorse these core competencies may be wondering how best to implement them. We believe it will be important to tailor training against each core competency described above. Some high-quality training might already exist for some competencies, while training for others will likely require development. Most editors are geographically dispersed, and it might be most effective to consider online training to maximize reach.

Evaluation

The process of developing the core competencies is similar to that of developing any intervention. As with any intervention, it will be important to address whether implementation of these core competencies is associated with improvements in the roles and functions of scientific editors, such as increased mentorship within a journal and applying best practices in evaluating adherence to research methods of submitted manuscripts [12]. One strong evaluation option is to consider an experimental design whereby some journals expose their scientific editors to formal core competency training while other journals act as a ”standard practice” control. The details of any evaluation require further deliberation and engagement. The recently established Best Practice Journal Research Network is one possible group to conduct such a study [13].

Discussion

The need for consistent, core competencies in scientific editing is clear. Proponents of the reducing waste in research campaign, for example, say that the system of assessing quality of scientific research needs improvement [14]. Scientific editors are clearly central to that system, which the Declaration of Helsinki recognizes by noting the responsibilities of editors in ensuring the highest possible standards in what is published in their journals [15]. Specifically developed for scientific editors of biomedical journals, these core competencies establish a baseline for the knowledge, skills, and characteristics needed in order to competently perform the duties of a scientific editor. In essence, they describe the agreed-upon minimum criteria for effectively performing the duties of a scientific editor at a biomedical journal. Our focus was on developing an intervention (i.e., core competencies) to help scientific editors. While scientific editors are central to helping improve the publication record, there are other constraints in the system on which they have limited influence. Most researchers find themselves in a ”publish or perish” environment. Academic institutions typically assess their faculty for promotion and tenure based on bibliometrics (i.e., some form of counting publications), which are often misaligned with societal needs. Quantity may be given undue priority over quality. Some researchers may “short-circuit” the quality of their research to meet publication needs, which can be difficult or impossible for editors to detect [16]. The push for quantity has also resulted in some authors circumventing peer review and editorial oversight to achieve publication [17].

The competencies themselves are not novel or new, per se, to the published literature. They were derived from our previous scoping review of existing published (and unpublished) competency statements, and despite having the opportunity throughout the process to add other competencies that were not derived from our comprehensive evidence-gathering process (e.g., scoping review, Delphi exercise), no completely novel competencies emerged. Likewise, the core competencies, in general, do not appear to be novel to most of the editors we surveyed. In comparing the findings from our survey of scientific editors of biomedical journals' training needs, perceptions of competence, and ratings of importance of competency-related statements, we found a high degree of congruency between the core competencies presented in this manuscript and editors’ needs and ratings of importance. In fact, both the top five editor training needs and the six competency-related statements rated most important are all found within the 14 core competencies and/or their elements. These areas include: statistics, research methods, publication ethics, the peer review process, integrity/professionalism, good decision-making, language skills, and journal indexing. However, presented here the competencies represent a new level of collaboration and rigor in their development for scientific editors of biomedical journals. To our knowledge, these core competencies did not exist in the peer-reviewed literature previously.

Although we used a rigorous, consensus-based approach in developing these core competencies, our methods nevertheless have limitations. Time and resource constraints, limited participation, and differences in participants’ perceptions, experiences, and interpretations may also have influenced the process of developing these competencies. In addition, the restriction of the consensus meeting and post-consensus meeting participation mainly to individuals representing editors and publishers may have limited inclusion of perspectives of other relevant groups (e.g., authors, readers, peer reviewers) in the selection and wording of the core competencies. However, the editors involved in the process were also authors and peer reviewers previously or currently. All participants are also readers. As such, these perspectives were not completely lost.

These core competencies might also be useful to other types of editors at biomedical journals, such as technical editors (i.e., those responsible for substantial editing of manuscripts, including re-writing for clarity and language), and to editors in other disciplines. Some editors and publishers might find these competencies simply aspirational, while others may want to recommend their implementation. We encourage stakeholders in the biomedical (and other) domain(s) to collaborate with each other to develop extensions (or modifications) to these core competencies to address the specific needs of particular groups of editors (such as scientific editors at small or large journals, editors-in-chief, or scientific editors in other disciplines), much in the same way that extensions to the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline have been created to address the reporting of specific types of trials [18].

The aim of our program to develop core competencies was to provide guidance to scientific publishers and editors of biomedical journals worldwide on the minimum knowledge, skills, and characteristics that are needed to be effective in their role. We emphasize that this list of core competencies is meant to represent the minimum standards for the role of scientific editor, regardless of the particularities of each journal. We acknowledge there may be other essential competencies that relate to scientific editors, depending on their specific circumstances.

The immediate short-term goal of this program was to develop an essential set of core competencies and examples and to encourage endorsement across a broad spectrum of journals and editorial groups. A subsequent short-term goal is to develop training modules based on these core competencies. Another short-term goal of this program is to develop a core competency-based curriculum with which to train scientific editors of biomedical journals. Once the curriculum is completed, evaluating the competencies will be essential. We hope these short-term goals will help scientific editors improve their journals and the publication record. A longer term goal is to consider a certification process whereby journal editors can obtain official recognition for demonstrating that they possess all of the core competencies. This process would also allow journals and publishers a way to distinguish themselves as having ensured a minimum standard of competency among all of their scientific editors, possibly through a system of digital badges [19]. The downstream consequences of these efforts might include an increase in the research value of science and a higher quality of scientific publications.