http://orcid.org/0000-0002-4702-2472
Figures
Abstract
Publications determine to a large extent the possibility to stay in academia (“publish or perish”). While some pressure to publish may incentivise high quality research, too much publication pressure is likely to have detrimental effects on both the scientific enterprise and on individual researchers. Our research question was: What is the level of perceived publication pressure in the four academic institutions in Amsterdam and does the pressure to publish differ between academic ranks and disciplinary fields? Investigating researchers in Amsterdam with the revised Publication Pressure Questionnaire, we find that a negative attitude towards the current publication climate is present across academic ranks and disciplinary fields. Postdocs and assistant professors (M = 3.42) perceive the greatest publication stress and PhD-students (M = 2.44) perceive a significant lack of resources to relieve publication stress. Results indicate the need for a healthier publication climate where the quality and integrity of research is rewarded.
Citation: Haven TL, Bouter LM, Smulders YM, Tijdink JK (2019) Perceived publication pressure in Amsterdam: Survey of all disciplinary fields and academic ranks. PLoS ONE 14(6): e0217931. https://doi.org/10.1371/journal.pone.0217931
Editor: Zubin Master, Mayo Clinic, UNITED STATES
Received: January 15, 2019; Accepted: May 21, 2019; Published: June 19, 2019
Copyright: © 2019 Haven et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because of participants’ privacy. The pseudonymized personal data is available for research purposes only under a data transfer agreement to ensure that persons authorized to process the personal data have committed themselves to confidentiality and the receiving party agrees that the pseudonymized personal data will only be used for scientific research purposes. To ease the possible exchange of data with fellow researchers, a concept data sharing agreement can be found here: https://osf.io/8jye2. Data inquires may be addressed to the steering committee of the Netherlands Research Integrity Network (NRIN) via info@nrin.nl.
Funding: LB, JT and TH were supported by the Templeton World Charity Foundation (https://www.templetonworldcharity.org/) under the grant #TWCF0163/AB106. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. TH was also partly supported by contributions from the Vrije Universiteit Amsterdam, the University of Amsterdam and the Amsterdam University Medical Centers. These institutions had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The current state of academia is sometimes referred to as a system affected by hyper-competition [1–3]. This goes hand in hand with strong emphasis on quantitative assessment of scientific output through journal impact factors, citation analyses and the H-index [4–6]. The number of publications, citations and grants determine to a large extent the status and recognition of academic researchers [7–10]. Consequently these indicators influence the recruitment, promotion and tenured appointments of researchers [11,12]. This may in turn induce a high level of perceived publication pressure.
In line with Woolf [13], we define perceived publication pressure as the subjective pressure resulting from the feeling that one has to publish. In line with work stress literature, strong perceptions of pressure could provoke stress, but need not to when one has many resources available to manage the pressure [14]. Applied to publication pressure: Publication demands and attitude towards the current publication climate determine the perceived pressure, yet pressure can be alleviated by resources like helpful co-authors, involved colleagues or supervisors, and a sense of academic competence [15].
Some degree of publication pressure can be an incentive to produce high quality scientific work [13,16]. Yet, too much publication pressure may have detrimental effects on the scientific enterprise in general and on individual researchers in particular [17]. Excessive publication pressure is associated with poor quality research (and teaching), a decreased willingness to share raw data, less involvement from researchers in public and policy issues, and less academic creativity [16,18–20]. The perceived hypercompetition is thought to lead to less rigorous (“rushing into print”) and less reliable science [1,21,22]. Publication pressure is associated with a greater likelihood to engage in research misbehaviours [23–25]. Lastly, publication pressure is associated with a disproportionate focus on positive and specular findings [21,22,26,27].
Publication pressure may also have detrimental effects on individual researchers. It is linked to a poor research climate and may render academic researchers emotionally exhausted [3,28]. Previous research on publication pressure found junior researchers to experience more publication pressure compared to their senior counterparts [20,23]. Studies investigating publication pressure thus far have mainly included academic researchers from particular disciplines like biomedicine, management and population studies, and included only a subset of academic ranks [16,20,23]. This limits the generalizability of the degree to which researchers perceive publication pressure.
The current study aims to assess whether researchers from all academic ranks (including PhD students) and all disciplinary fields perceive publication pressure. This is important, as differences between academic ranks could signal the need for tailored interventions. Besides, comparing different disciplinary fields may enable us to determine fields that perceive less publication pressure. This may generate new insights in the nature of publication pressure and possible protective factors. Our research question was: What is the level of perceived publication pressure in the four academic institutions in Amsterdam and does the pressure to publish differ between academic ranks and disciplinary fields?
Materials and methods
Ethical statement
Our study was ethically reviewed and approved by the Scientific and Ethical Review board of the Faculty of Behavioural and Movement Sciences (Vrije Universiteit Amsterdam).
Participants
All academic researchers in Amsterdam employed in research for at least one day per week at one of the four academic institutions (Vrije Universiteit Amsterdam, University of Amsterdam and the two Amsterdam University Medical Centers) were eligible to participate. This included PhD students, as in The Netherlands PhD students are employees.
Procedure
First, we set up a data sharing agreement with all participating institutions to safely obtain the e-mail addresses of their researchers. Second, we sent an information letter inviting all academic researchers in Amsterdam (n = 7465) to take part in our study. The information letter contained links to the study protocol (S1 Protocol) and the study’s privacy policy (S1 Appendix). In addition, we included a link to a short non-response questionnaire where we asked researchers to report their academic rank, gender and enquired whether the reason for declining participation resulted from a sense that their data were not protected. For the full non-response questionnaire, see S2 Appendix.
A week later, researchers were invited to complete an online survey. The survey started with an informed consent statement followed by the inclusion check (“Are you currently employed in research for at least one day per week?”) and ended with the demographic items about participants’ academic rank (PhD student, postdoc, assistant professor, associate professor and full professor) and major disciplinary field: biomedicine (consisting of life and medical sciences), natural sciences, social sciences (included both social and behavioural sciences) and humanities (consisting of humanities, language, communication, law and arts). We used Qualtrics (Qualtrics, Provo, UT, USA) to create and distribute the survey, that took approximately 15 minutes to complete. We sent three reminders, each 10 days apart.
Instruments
We used the revised Publication Pressure Questionnaire (PPQr) to measure publication pressure [15]. The PPQr is a valid and reliable instrument to measure publication pressure and consists of 3 subscales scored on a 5-points Likert scale (‘Totally agree’ = 5, ‘Totally disagree’ = 1). The Publication Stress subscale (6 items—Cronbach’s α = .804) regards the stress a researcher experiences due to the feeling she/he has to publish and includes items such as “I feel forced to spend time on my publications outside office hours”. The Publication Attitude (6 items—Cronbach’s α = .777) subscale reflects researchers’ attitudes towards publication pressure, for example: “Publication pressure harms science”. Finally, the Publication Resources subscale (6 items—Cronbach’s α = .754) consist of factors that can help prevent publication pressure (e.g. feeling of competence, freedom to choose topics of scientific investigation; involved colleagues). A typical item would be: “When working on a publication, I feel supported by my co-authors.”. The full PPQr questionnaire can be found in S3 Appendix.
PPQr subscale scores are computed by taking the average of all items in the subscale. A higher score on all subscales means the researcher perceives publication stress, has a negative attitude towards the publication climate and perceives little publication resources to alleviate publication stress.
The survey contained two other instruments (Survey of Organizational Research Climate [29] and 60 major and minor misbehaviours [30]), but those analyses will be part of another report see [31] and [32]. The interrelations between these concepts will be reported in a separate future paper.
Statistical analyses
We preregistered our analyses on the Open Science Framework, see osf.io/w4t7u. To summarise: First we calculated overall mean scores for all three subscales and stratified these for academic ranks and disciplinary fields. Second, we assessed whether there were differences between particular academic ranks or disciplinary fields using Bonferroni corrected F-tests and Mean Differences (MD) with 95% Confidence Intervals (CI). Third, we build multivariable regression models to test whether academic rank and disciplinary field were associated with PPQr subscale mean scores. In these regression models, we also looked for evidence of confounding and interaction. Estimates corrected for confounding are provided and instances of interaction were reported. All analyses were conducted using SPSS Statistics.
Results
Response rate and inclusion
From the 7548 researchers that were invited to participate, 30% (n = 2274) followed the link to the online survey. 1073 of invitees filled in the PPQr (response rate = 14%), demographic information is listed in Table 1. About 2% of the invitees filled in the non-response questionnaire. See Fig 1.
Overall, we find academic researchers in our sample to score highest on Attitude (M = 3.59). This indicates that the negative attitude towards the publication climate is substantial. There is on average a somewhat lesser degree of Publication Stress (M = 3.22) and a relatively small lack of Publication Resources (M = 2.21). Stratified and total sample mean scores can be found in Table 1.
Publication pressure by academic rank
Pairwise Bonferroni and confounding-corrected (disciplinary field and gender) mean differences between academic ranks indicate that postdocs and assistant professors perceive significantly more publication stress than both PhD students and associate and full professors. Besides, both PhD students as well as postdocs and assistant professors have a more negative attitude towards the publication culture compared to full professors. Furthermore, PhD students perceive a significantly greater lack of resources than both postdocs and assistant professors as well as associate and full professors. Finally, postdocs and assistant professors perceive less resources than associate and full professors. See Fig 2. Crude and Bonferroni corrected mean differences between pairs of groups can be found in S1 Table. For crude and corrected association models between academic rank and the PPQr subscales, see S2 Table.
Letters denote significant (α = .05) Bonferroni corrected mean differences (MD) between pairs of (denoted by the brackets) academic ranks and error bars express 95% Confidence Intervals (CI). MDs are corrected for confounders (disciplinary field and gender) if applicable. N = 1073. a: MD = .237, CI = (.103, .371) b: MD = .384, CI = (.219, .549) c: MD = .181, CI = (.049, .314) d: MD = .282, CI = (.139, .426) e: MD = .322, CI = (.223, .421) f: MD = .645, CI = (.532, .757) g: MD = .322 CI = (.201, .444).
Publication pressure by disciplinary field
Pairwise Bonferroni and confounding-corrected (academic rank and gender) mean differences indicate that researchers in the humanities perceive more publication stress than both biomedicine and the natural sciences. Yet the researchers from the social sciences perceive more publication stress than their biomedical colleagues. There were no statistically significant differences between disciplinary fields on attitude scores. Finally, researchers in biomedicine as well as social sciences perceive a significantly greater lack of publication resources than researchers in the natural sciences. See Fig 3. Crude and Bonferroni corrected mean differences between pairs of groups can be found in S1 Table. For crude and corrected disciplinary field association models, see S3 Table.
Letters denote significant (α = .05) Bonferroni corrected mean differences (MD) between pairs of (denoted by the brackets) disciplinary fields and error bars express 95% Confidence Intervals (CI). MDs are corrected for confounders (academic rank and gender) if applicable. N = 1073. a: MD = .297, CI = (.080, .515) b: MD = .202, CI = (.042, .363) c: MD = .318, CI = (.040, .596) d: MD = .204, CI = (.048, .359) e: MD = .210, CI = (.036, .384).
Effect modification
We only found effect modification by disciplinary field of the differences between academic ranks’ Publication Resources scores. Differences between PhD students and senior academic researchers in perceived Publication Resourced are greater in natural sciences compared to other disciplinary fields. Stratified results are displayed in Table 2.
Effect sizes
We found 12 significant differences between pairs of groups and since we performed many statistical tests, it is likely that some of the significant differences are in fact due to chance. To provide the reader with some guidance on which effects are relevant, we calculated effect sizes of each difference. This analysis was not preregistered and thus should be considered exploratory. The effect sizes range from small to very large using Cohen’s effect size criteria [33], see Table 3. To prevent overinterpreting small differences, we will focus further discussion on differences with an effect size of medium or above.
Discussion
We assessed the level of perceived publication pressure in the four academic institutions in Amsterdam and whether the pressure to publish differed between academic ranks and disciplinary fields. Overall, there is a negative attitude towards the publication climate. Hence the ‘publish or perish’ mantra from the late 20th century may turn into ‘publish and perish’, since even when a researcher publishes reasonably, chances for tenure in academia may still be low [20,34,35]. Below we elaborate on the differences of effect sizes that were medium or above or on those where we found interaction effects [33].
Academic rank differences
Postdocs and assistant professors perceive most publication stress and have the most negative attitude towards the current publication climate, which is in line with previous studies assessing perceived publication pressure in biomedicine and organisation science [20,23]. This finding seems intuitive as this particular group aims for a (tenured) position and promotion criteria are to a large extent based on quantitative publication indicators. Associate and full professors have already an established position, and consequently may perceive less publication pressure. PhD candidates’ likelihood of successfully defending their thesis is usually not dependent on the number of publications. This may explain why their publication pressure level is somewhat lower. Besides, some PhD students may not aspire an academic career and will therefore presumably perceive less publication pressure then their colleagues who wish to pursue an academic career.
However, PhD candidates perceive the greatest lack of resources. This is both alarming and understandable. Arguably, PhD students are inexperienced in handling difficulties that may arise when working on a publication. The same holds for starting postdocs. Consequently, junior researchers could benefit most from supportive colleagues and supervisors. Unfortunately mentoring may be suboptimal [30,36,37].
Disciplinary field differences
Differences between disciplinary fields were significant but small. Hence, we focus here on the interaction between disciplinary field and academic rank in perceived Resources. Researchers from the natural sciences perceived most publication resources, which may be due to their typical organisation (large) research teams where collaboration is vital for discovery. However, PhD students in the natural sciences perceive a lack of resources that is similar to PhD students from the other disciplinary fields. It may be that insufficient mentoring in the publication process makes them feel incompetent and insecure.
Strengths
This is the first study that comprehensively measured publication pressure with a validated measurement instrument. The three dimensions, stress, attitude and resources, respectively, are meaningful components when conceptualising publication pressure. Also, these three dimensions are sufficiently distinctive in the data reported here [15].
Second, this is the first study to investigate publication pressure across academic ranks and disciplinary fields. It can serve as a benchmark for future studies. We managed to include a substantial number of participants in our study that increases the reliability of the differences found.
Limitations
Our study also has some limitations we would like to address. First, we have a relatively low completion rate (14%) which may be an indication of response bias, although our completion rate is similar to other web-based surveys [38]. Only 2% of our invitees filled in the non-response questionnaire, which we consider to be too little to assess whether non-responders differed from responders. Perhaps invitees chose not to respond because they were to focused on their publications, leading to an underestimation. Related, simply mentioning that our study investigated the publication culture could have prompted negative connotations with the publication culture, as it has not gone unnoticed in the public debate in The Netherlands.
To assess the representativeness of our sample, we first looked into the population characteristics. In our sample, 56% of completers indicated working in the biomedical field, whereas 53% of our invitees was employed at one of the Amsterdam University Medical Centers, indicating a small overrepresentation from biomedicine.
Statistics on PhD students employed at both universities in Amsterdam indicated that PhD students make up 30% of the academic workforce, whereas PhD students formed 41% of our sample. Likewise, 44% of academic researchers in Amsterdam is female, yet women made up 57% of our sample, indicating overrepresentation of both PhD students and women.
However, we corrected for the potential gender bias by adjusting our estimates for confounding variables. Besides, we found no effect modification from gender. To conclude, it is unlikely that the selectivity of our sample biased our results.
To assess possible response bias, we conducted a wave analysis. We used late responders–those who responded after the last reminder–as a proxy for nonresponders and compared these to early responders–those who responded after the initial invitation–as described by Phillips’ and colleagues [39]. Differences were .13, .07 and .02 for Stress, Attitude and Resources, respectively. These differences were then multiplied by the proportion of non-responders, in our case 86%. Consequently, the non-response bias was .11, .07 and .02 for Stress, Attitude and Resources, respectively. This was found to be small compared to the difference that we observed between the subgroups which ranged from .18 to .65. It is therefore unlikely that non-response affected our conclusions.
Besides, the PPQr focuses exclusively on publication pressure. However, research is not conducted in a vacuum and if teaching or other professional duties put excessive demands on a researcher, then naturally there is less time left for publishing, which could lead to elevated levels of publication pressure. This can be labelled as role-conflict: you are expected to meet different obligations, i.e. teaching, research, and professional duties, in a naturally limited amount of time [40]. How much stress is due to just publication pressure is unclear (see also [15]).
Relatedly, universities have been subject to neoliberal and Taylorist reforms that were—in a nutshell—intended to make universities more competitive and were accompanied with an excessive focus on researchers’ performance management, perhaps at the expense of traditional hallmarks of the academia such as teaching and collegiality [41,42]. A full review of Neoliberal and Taylorist reforms in academia is beyond the scope of this paper (the reader is referred to Lorenz’ excellent paper [43] that includes specific examples of reforms in Dutch academia) but it seems feasible to reason that publication pressure is one of its consequences, although the exact relation has, to our knowledge, not been studied systematically.
Finally, since this is the first study conducted with the PPQr, it’s rather difficult to interpret the absolute levels and differences in publication pressure we found.
Future research
Future work should aim to explore if the differences we found generalise internationally. Publication climates in the USA and Asian countries may be different as their funding systems greatly differ [2,44,45]. However, the same could apply to closer examples such as Germany and Belgium, as those funding systems are also somewhat different from those in the Netherlands. Interestingly, a study with an previous version of the PPQ found Flemish biomedical researchers to experience more publication pressure than their Dutch colleagues [23]. Besides, it will be informative to study publication pressure longitudinally to see if it is associated with burn-out and research misbehaviour. Finally, it would be intriguing to investigate qualitatively what it means for researchers to experience high publication pressure and how it impacts their academic work.
Conclusions
Taken together, publication pressure concerns researchers from all disciplinary fields and seems to be a particularly detrimental stressor for postdocs and assistant professors. In addition, PhD students perceive a significant lack of resources that may hamper their development into responsible researchers. The amount of resources is perceived to be better among researchers from the natural sciences, but PhD students in this disciplinary field nevertheless would also benefit from more support from their senior colleagues. Our findings emphasize the need to move the debate forward towards a healthy publication climate, where researchers are incentivised to focus on the quality and the integrity of their publications and feel supported to conduct responsible research.
Supporting information
S1 Protocol. Academic Research Climate Amsterdam study protocol.
https://doi.org/10.1371/journal.pone.0217931.s001
(DOCX)
S3 Appendix. Full PPQr questionnaire with corresponding Cronbach’s alphas.
Items are scored on a 5-points Likert scale where 1 = “Totally disagree” and 5 = “Totally agree”.
https://doi.org/10.1371/journal.pone.0217931.s004
(DOCX)
S1 Table. Crude and Bonferroni corrected mean differences (MD) with 95% Confidence Intervals (CI) between academic ranks and disciplinary fields.
https://doi.org/10.1371/journal.pone.0217931.s005
(DOCX)
S2 Table. Crude (in italics) and corrected academic rank association models.
Interpretation: For gender, male is the reference category. For the disciplinary field dummies, humanities is coded as the reference category. For academic rank dummies, this is associate and full professors. PhDs = PhD students, assis prof = assistant professors, asso prof = associate professors, full prof = full professors.
https://doi.org/10.1371/journal.pone.0217931.s006
(DOCX)
S3 Table. Crude (in italics) and corrected disciplinary field association models.
Interpretation: For gender, male is the reference category. For the disciplinary field dummies, humanities is coded as the reference category. For academic rank dummies, this is associate and full professors. PhDs = PhD students, assis prof = assistant professors, asso prof = associate professors, full prof = full professors.
https://doi.org/10.1371/journal.pone.0217931.s007
(DOCX)
Acknowledgments
We would like to acknowledge all members of the steering committee of the Academic Research Climate Amsterdam project (Frans Oort, Gerben ter Riet, Hanneke de Haes, Guy Widdershoven and René van Woudenberg) for their continuous critical input. Frans Oort has been crucial in the creation of the PPQr. Peter van de Ven has been indispensable in discussing statistical tests and models. We also wish to acknowledge Gowri Gopalakrishna and Venetia Qendri for the independent re-analysis of the survey data.
References
- 1. Anderson MS, Ronning EA, De Vries R, Martinson BC. The perverse effects of competition on scientists’ work and relationships. Sci Eng Ethics. 2007;13(4):437–61. pmid:18030595
- 2. Alberts B, Kirschner MW, Tilghman S, Varmus H. Rescuing US biomedical research from its systemic flaws. Proc Natl Acad Sci. 2014;111(16):5773–7. pmid:24733905
- 3. Tijdink JK, Schipper K, Bouter LM, Pont PM, De Jonge J, Smulders YM. How do scientists perceive the current publication culture? A qualitative focus group interview study among Dutch biomedical researchers. BMJ Open. 2016;6(2).
- 4. Gannon F. The Impact of the Impact Factor. EMBO Rep. 2000;1(4):292–3.
- 5. Hirsch JE. An index to quantify an individual’s scientific research output. PNAS. 2005;102(46):16569–16572. pmid:16275915
- 6. Dinis-Oliveira RJ, Magalhães T. The Inherent Drawbacks of the Pressure to Publish in Health Sciences: Good or Bad Science. F1000Research. 2016;4(1):419.
- 7. Bird S. Research Ethics, Research Integrity and the Responsible Conduct of Research. Sci Eng Ethics. 2006;12(3):56.
- 8. Walker RL, Sykes L, Hemmelgarn BR, Quan H. Authors’ opinions on publication in relation to annual performance assessment. BMC Med Educ. 2010;10(1):2–7.
- 9. Anderson PA, Boden SD. Ethical Considerations of Authorship. SAS J. 2008;2(3):155–8.
- 10. Buela-casal G. Pathological publishing: A new psychological disorder with legal consequences? Eur J Psychol Appl to Leg Contex. 2014;6(2):91–7.
- 11. Court S, Kinman G. Tackling Stress in higher education. 2008.
- 12. Glick W, Miller C, Cardinal L. Making a life in the field of organization science. J Organ Behav. 2007;28:817–35.
- 13. Woolf PK. Pressure to Publish and Fraud in Science. Ann Intern Med. 1986;194:254–6.
- 14. Demerouti E, Nachreiner F, Bakker AB, Schaufeli WB. The job demands-resources model of burnout. J Appl Psychol. 2001;86(3):499–512. pmid:11419809
- 15. Haven TL, Tijdink JK, De Goede MEE, Oort F. Personally perceived publication pressure—Revising the Publication Pressure Questionnaire (PPQ) by using work stress models. Prepr ahead Publ. 2018;August, 12.
- 16. Dalen Van, Hendrik ; Henkens K. Intended and unintended consequences of a publish-or-perish culture: A worldwide survey. J Am Soc Inf Sci Technol. 2012;63(7):1282–1293.
- 17. Tijdink JK, Smulders YM, Vergouwen ACM, de Vet HCW, Knol DL. The assessment of publication pressure in medical science; validity and reliability of a Publication Pressure Questionnaire (PPQ). Qual Life Res. 2014;23(7):2055–62. pmid:24522963
- 18. DeAngelis CD. Professors not professing. J Am Med Assoc. 2004;292(9):1060–1.
- 19. Blumenthal D, Campbell EG, Gokhale M, Yucel R, Clarridge B, Hilgartner S, et al. Data Witholding in Genetics and the Other Life Sciences: Prevalences and Predictors. Acad Med. 2006;81(2):137–45. pmid:16436574
- 20.
Miller AN, Taylor SG, Bedeian AG. Publish or perish: academic life as management faculty live it. Vol. 16, Career Development International. 2011. p. 422–45.
- 21. Fanelli D. Do pressures to publish increase scientists’ bias? An empirical support from US states data. PLoS One. 2010;5(4).
- 22. Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):0696–701.
- 23. Tijdink JK, Verbeke R, Smulders YM. Publication pressure and scientific misconduct in medical scientists. J Empir Res Hum Res Ethics. 2014;9(5):64–71. pmid:25747691
- 24. Bedeian A, Taylor S, Miller A. Management science on the credibility bubble: Cardinal sins and various misdemeanors. Acad Manag Learn Educ. 2010;9(4):715–25.
- 25. Bouter LM. Commentary: Perverse incentives or rotten apples? Account Res. 2015;22(3):148–61. pmid:25635847
- 26. Munafò MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, Percie Du Sert N, et al. A manifesto for reproducible science. Nat Hum Behav. 2017;1(1):1–9.
- 27. Harzing A-W. Publish or Perish: A proposal. Ann Intern Med. 1986;104:261–2. pmid:3946958
- 28. Tijdink J, Vergouwen A, Smulders Y. Publication pressure and burn out among Dutch medical professors: A nationwide survey. PLoS One. 2013;8(9).
- 29. Martinson BC, Thrush CR, Lauren Crain A. Development and Validation of the Survey of Organizational Research Climate (SORC). Sci Eng Ethics. 2013;19(3):813–34. pmid:23096775
- 30. Bouter LM, Tijdink J, Axelsen N, Martinson BC, ter Riet G. Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity. Res Integr Peer Rev. 2016;1(17):1–8.
- 31. Haven TL, Tijdink JK, Martinson BC, Bouter LM. Perceptions of research integrity climate differ between academic ranks and disciplinary fields: Results from a survey among academic researchers in Amsterdam. PLoS One. 2019;14(1):e0210599. pmid:30657778
- 32.
Haven T, Tijdink J, Pasman R, Widdershoven G, Bouter LM. Do research misbehaviours differ between disciplinary fields? Prepr April 3rd, osf.io/7d4qz.
- 33.
Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, N.J.: Erlbaum; 1995.
- 34. Csiszar A. The catalogue that made metrics, and changed science. Nature. 2017;551(7679):163–5. pmid:29120444
- 35. Glausiusz J. Tenure derailed. Nature. 2019;565:525–7. pmid:30670860
- 36. Levecque K, Anseel F, De Beuckelaer A, Van der Heyden J, Gisle L. Work organization and mental health problems in PhD students. Res Policy. 2017;46(4):868–79.
- 37. Panger G, Tryon J, Smith A. Graduate Student Happiness & Well-Being Report. 2014.
- 38. Cook C, Heath F, Thompson R. A meta-analysis of response rates in web-or internet-based surveys. Educ Psychol Meas. 2000;60(6):821–36.
- 39. Phillips AW, Reddy S, Durning SJ. Improving response rates and evaluating nonresponse bias in surveys: AMEE Guide No. 102. Med Teach. 2016;38(3):217–28. pmid:26648511
- 40.
Kahn RL, Wolfe DM, Quinn RP, Rosenthal RA. Organizational stress: Studies in role conflict and ambiguity. New York, SE.: Wiley; 1964.
- 41. Shore C. Beyond the multiversity: Neoliberalism and the rise of the schizophrenic university. Soc Anthropol. 2010;18(1):15–29.
- 42. Bal E, Grassiani E, Kirk K. Neoliberal individualism in Dutch universities: Teaching and learning anthropology in an insecure environment. Learn Teach. 2014;7(3):46–72.
- 43. Lorenz C. If You’re So Smart, Why Are You under Surveillance? Universities, Neoliberalism, and New Public Management. Crit Inq. 2012;38(3):599–629.
- 44. Zare RN, Winnacker EL. China’s science funding. Science (80-). 2011;334(6055):433. pmid:22034401
- 45. Vaesen K, Katzav J. How much would each researcher receive if competitive government research funding were distributed equally among researchers? PLoS One. 2017;12(9):4–6.