Rapid Communications
Print
Rapid Communications
Gender bias when assessing recommended ecology articles
expand article infoCorey J. A. Bradshaw, Franck Courchamp§
‡ Flinders University, Adelaide, Australia
§ Univ. Paris-Sud, Orsay, France
Open Access

Abstract

Gender bias is still unfortunately rife in the sciences, and men co-author most articles (> 70%) in ecology. Whether ecologists subconsciously rate the quality of their peers’ work more favourably when they are the same gender (homophily) is still unclear. To test this hypothesis, we examined how ecologist editors ranked important ecology articles based on a previously compiled list where they had first each proposed some articles and then voted on all proposed articles. The proportion of female co-authors on the articles proposed by men were lower (0.06 to 0.09) than those proposed by women (0.13 to 0.27), although the data were highly skewed and most proposed articles (77%) had no female co-authors. For the 100 top-ranked articles voted by women or men only, the gender difference remained: female voters ranked articles in the top 100 that had more female co-authors (0.029 to 0.093 proportion women) than did those voted by men (0.001 to 0.029). Female voters tended to rank articles more highly as the number of male co-authors increased, and the relationship between article rank and proportion of male co-authors was even stronger when only men voted. This effect disappeared after testing only articles that editors declared they had actually read. This could indicate a persistent, subconscious tendency toward homophily when assessing the perceived quality of articles that ecologists have not actually read.

Keywords

gender, ecology, evolution, scientific publishing, homophily, disparity

Introduction

Despite a general reduction in gender disparities in academia (Ceci and Williams 2011, Zakaib 2011, West et al. 2013), there remains ample gender-bias across scientific disciplines. Experimental evidence shows that scholars tend to rate men-authored writings higher (Knobloch-Westerwick et al. 2013), and that academic scientists tend to favour men for lab-manager positions (Moss-Racusin et al. 2012). In the United Kingdom, there is also evidence that female academics in science, engineering, and mathematics have more administrative duties on average than men, and hence, less time to do research (Aldercotte et al. 2017). Female scientists also have fewer opportunities for career development and training, and tend to earn lower salaries, hold fewer senior roles, and are less likely to be offered permanent positions (Buckley et al. 2000, Aldercotte et al. 2017).

In ecology, despite approximate gender parity among undergraduates and young researchers (Damschen et al. 2005) – as is now the case in most science disciplines (Damschen et al. 2005, Ceci et al. 2014) – senior academic positions are still dominated by men (Tregenza 2002, Larivière et al. 2013, Howe-Walsh and Turnbull 2016). This means that most ecology papers are written by men (Cameron et al. 2013, West et al. 2013); for example, in a study examining the proportion of female authorships in papers published from 1990–2011 across 21 science and humanities disciplines (West et al. 2013), the field of ecology and evolution (JSTOR ‘field’) had the seventh lowest proportion of female authors (22.8% of 279012 total authorships). Female scientists are also consistently under-represented in ecology textbooks compared to baseline assumptions of no bias (Gurevitch 1988, Damschen et al. 2005).

Due to the availability of more men in senior positions, scientific journals also tend to appoint more men than women on their editorial boards, and editors tend to select reviewers of the same gender as themselves (a preference known as homophily) (Helmer et al. 2017). Evidence for homophily among ecologists has been found previously; for example, male editors selected < 25% female reviewers, but female editors consistently selected between 30 to 35% female reviewers for all papers submitted to the journal “Functional Ecology” from January 2004 to June 2014 (Fox et al. 2016). Yet, this is not due to the actual performance of female reviewers, because reviewer scores for that journal did not differ between male and female reviewers, and the proportion of papers rejected did not differ between female and male editors (Fox et al. 2016). However, there are gender differences in how papers are reviewed. For example, from a much broader sample of journals in ecology and evolution, a survey of 1334 ecologists and evolutionary biologists identified that women spent more time reviewing papers than men, and women reviewed fewer manuscripts on average (a logical outcome of being asked less frequently than men to review) (Grod et al. 2008). In seeming contradiction to the lack of a gender difference in reviewer scores for Functional Ecology (Fox et al. 2016), men from the broader sample recommended rejection more frequently than did women (Grod et al. 2008).

Despite the evidence that female authors tend to receive fewer citations than men in some academic disciplines (Aksnes et al. 2011, Maliniak et al. 2013, Beaudry and Larivière 2016, Atchison 2017), there is little evidence for gender bias in acceptance or citation rates of ecology papers. In one regional ecology journal (New Zealand Journal of Ecology), publication success between 2003 and 2012 was not related to the gender of the authors or that of the editor, but like Functional Ecology, editors selected more male reviewers (Buckley et al. 2014), likely because there are simply more male ecologists from which to choose reviewers. Likewise, there was no author-gender bias for the citation rates for 5883 ecology articles published between 1997 and 2004 (from the journals Animal Behaviour, Behavioral Ecology, Behavioral Ecology and Sociobiology, Biological Conservation, Journal of Biogeography, and Landscape Ecology) (Borsuk et al. 2009). A similar conclusion was reached for 507 ecology and evolution articles from five ‘leading’ (but unidentified) journals (Tregenza 2002). Nor was there an overall difference in the acceptance rates of papers according to gender for 2550 ecology and evolution articles (even for single-authored papers), although this differed among journals (Tregenza 2002). However, in one journal (Behavioral Ecology), the number of female first-authored papers (published) increased following the implementation of double-blind reviews (Budden et al. 2008), suggesting that either women were being given harsher treatment during review, or were less likely to submit when their gender could be identified at the time of submission.

Examining the publication output of 187 individual editorial board members of seven ecology and evolution journals, women had a lower mean h-index than did men (after controlling for scientific ‘age’) (Kelly and Jennions 2006). Given the lack of evidence for gender bias in citation rates in ecology (Tregenza 2002, Leimu and Koricheva 2005, Borsuk et al. 2009; Buckley et al. 2014), it is thought that this was mainly a result of the lower average publication output of female ecologists (Kelly and Jennions 2006, Cameron et al. 2013). Indeed, in a sample of 39 women and 129 men in evolutionary biology and ecology from the same approximate cohort (who held research and faculty positions in the life sciences departments of British and Australian universities), men produced almost 40% more papers than did women, and this difference appeared as early as two years from initial publication (Symonds et al. 2006). Likewise, a sample of 182 academic biologists (69 women and 113 men) with at least ten years of experience in academia indicated that women produced between 19 and 29% fewer papers after ten years of employment than did men (Laurance et al. 2013).

Gender differences in publication frequency can occur for many reasons, including possibly having less time to do research (Aldercotte et al. 2017), higher demands of motherhood (Monosson 2008, McGuire et al. 2012, O’Brien and Hapgood 2012), a lower relative tendency compared to men to seek self-promotion (Rudman 1998, Wade 2001, Moss-Racusin and Rudman 2010), fewer academic grants and accolades (Wenneras and Wold 1997, Bornmann et al. 2007, Lincoln et al. 2012), among other reasons (Damschen et al. 2005, Holt and Webb 2007, Laurance et al. 2011, Nature 2011, Cameron et al. 2013). Despite no strong evidence yet for gender biases in citation rates in ecology, it is still unclear whether established ecologists – both female and male –subconsciously rate the quality of their peers’ work more favourably when the co-authors are predominantly of the same gender. Using our previously published study (Courchamp and Bradshaw 2017) that asked ecologist editors to rank important ecological publications, here we did a post hoc analysis to assess: (i) whether the gender of the respondent was correlated with the overall co-author gender ratios of the articles the respondent ranked highest, and (ii) whether there was a correlation between an article’s mean rank and the co-author gender ratio.

Methods

The full details of how we generated the list of recommended ecology articles and how they were ranked are given in our previous study (Courchamp and Bradshaw 2017); however, we briefly describe the approach and main characteristics of the list here. We contacted the editorial members of some of the most renowned journals in general ecology: Ecology Letters, Trends in Ecology and Evolution, Ecology, Oikos, The American Naturalist, Ecology and Evolution and Ecography, as well as all the members of the Faculty of 1000 Ecology Section (https://f1000.com/prime/thefaculty/browse/ecol). We asked them by e-mail to send us three to five peer-reviewed papers (or more if they wished) that they deemed each postgraduate student in ecology – regardless of their particular topic – should read by the time they finish their dissertation, and that any ecologist should also probably read.

We successfully elicited propositions from 113 respondents of the 665 we contacted, who in total nominated 544 different articles to include in the primary list. We then asked these same 665 editors to vote on each of the papers to obtain a ranking, assigning each article to one of four categories: Top 10, Between 11–25, Between 26–100 or Not in the top “100”. We gave one (1) point for each selection of the Top 10 category, two points for the Between 11–25, three points for the Between 26–100, and four points for the Not in the top “100”. As we described in Courchamp and Bradshaw (2017), we averaged all article scores across all randomly sampled sets of votes for each article, and then applied a simple rank to these (ties averaged), thus avoiding any contrived magnitude of the differences between arbitrary score values (i.e., 1 to 4 base scores). The lowest scores therefore indicate the highest ranks.

Here, to test for evidence of homophily, we manually classified the gender of all proposers, voters, and article co-authors by searching the internet, requesting confirmation from colleagues, or from personal knowledge. We searched meticulously and are confident that we have a correct gender assignment for all people included in the analysis. The Ethics Committee of the Centre National de Recherche Scientifique (CNRS, employer of FC) deemed that no ethics approval was necessary for the voluntary and anonymous survey that generated the ranked list of articles.

Analyses

We took those articles proposed by either women only, or men only, to examine trends between the proposer genders (74% of all proposed papers were proposed only once) from our previous study (Courchamp and Bradshaw 2017). For determining trends between genders of the voters, we subset the entire dataset for female- and male-only voters, tabulating the proportion of female co-authors and the gender of lead authors for the different top-100 ranks resulting from each gender-specific voter subset.

To test for correlations between rankings and gender, we used the proportion of female co-authors for each article as the response variable in all analyses. We also used the same resampling approach from our previous study (Courchamp and Bradshaw 2017) to determine correlations by taking the raw, average scores for each article (independent variable) and compared them to randomised orders of the corresponding correlate (dependent variable) for each test. This approach requires no assumption of particular error distributions because it resamples from the data themselves (Manly 2006). For each randomised order over 10,000 iterations, we calculated a root mean-squared error (RMSErandom) and compared this to the observed RMSE between the two variables. When the probability that randomisations produced RMSE ≤ observed RMSE was small (i.e., number of times [RMSErandom ≤ RMSEobserved] ÷ 10,000 iterations << 0.05), we concluded that there was evidence of a correlation.

Data and code availability

All code and data for the analysis are available online at https://github.com/cjabradshaw/HIPE/gender/.

Results

More men responded and proposed more articles – among the 665 editors we contacted to propose articles, 22.1% (141) were women; 12 women (8.5% of the women contacted) and 101 men (19.3% of the men contacted) proposed articles. The proportion of female co-authors on the articles proposed by men were lower (0.06 to 0.09) than those proposed by women (0.13 to 0.27), although the data were highly skewed and most proposed articles (77%) had no female co-authors (Fig. 1a). For the 100 top-ranked articles voted by women or men only, the gender difference remained: female voters ranked articles in the top 100 that had more female co-authors (0.029 to 0.093 proportion women) than did those voted by men (0.001 to 0.029) (Fig. 1b). Female voters tended to rank articles more highly as the number of male co-authors increased (β = 0.03, pran = 0.011; Fig. 1c). For male voters only, the relationship between article rank and proportion of male co-authors was stronger (β = 0.11, pran < 0.0001; Fig. 1d). The inverse-score-weighted mean proportion of female co-authors was 0.0277 (standard error of the mean [s.e.m] = 0.0027) for female voters, and 0.0251 (s.e.m. = 0.0024) for male voters (Fig. 1c, d).

Figure 1.

a Mean (dashed horizontal lines) and 95% confidence interval (error bars) of the proportion of female co-authors for the proposed articles relative to the gender of the proposer (articles proposed by 68 women only, and 418 proposed by men only). The values (proportion female co-authors) are ‘scattered’ to show their distribution within each proposer gender; note that 55.9% and 80.1% of the articles proposed by women only and men only, respectively, had no female co-authors (i.e., zero values) b Mean (dashed horizontal lines) and 95% confidence interval (error bars) of the proportion of female co-authors of the 100 top-ranked articles relative to the gender of the voter (62 women and 292 men voted in total). The values (proportion female co-authors) are ‘scattered’ to show their distribution within each voter gender; note that 83% and 94% of the articles proposed by women and men, respectively had no female co-authors (i.e., zero values). c) Proportion of female co-authors on articles relative to their mean rank (score; where lower scores indicate a higher ranking) when voters were restricted to women. There was a weak (β = 0.03), but non-random (pran = 0.011) correlation between article gender ratio and score, such that the lower the proportion of female co-authors, the higher they were ranked by women d Proportion of female co-authors on articles relative to their mean rank when voters were restricted to men. There was a stronger (β = 0.11) and non-random (pran < 0.0001) correlation between article gender ratio and score, such that the lower the proportion of female co-authors, the higher they were ranked by men. Also shown in both panels is the inverse-score-weighted mean proportion of female co-authors (Σwi/si = 0.0277 for i female voters, or 0.0251 for i male voters, where s = score from 1 to 4, and w = proportion of female co-authors) e As in c, but when the scored articles were only those actually read by the voters (Courchamp and Bradshaw 2017) f As in d, but when the scored articles were only those actually read by the voters. The inverse-score-weighted mean proportion of female co-authors for these read-only articles was higher for female-only (0.0458) versus male-only voters (0.0303).

These relationships could arise because older articles were more highly ranked, and gender biases in authorship are generally stronger in older articles according to our previous research (Courchamp and Bradshaw 2017). The entire list of proposed articles indicated a trend of increasing proportion of female co-authors, from < 5% female co-authors before the 1990s, to > 25% female co-authorship in the most recent decade (Fig. 2). From our previous study (Courchamp and Bradshaw 2017), we had also distinguished between papers that editors knew and papers they had actually read. Using the ‘read-only’ article scores, the relationships between article rank and proportion of female co-authors disappeared for both female (Fig. 1e) and male voters (Fig. 1f), although the inverse-score-weighted mean proportion of female authors was still higher for female (0.0458 ± 0.0065 s.e.m.) than male voters (0.0303 ± 0.0031 s.e.m.).

Figure 2.

Time series of mean (± 2 standard errors of the mean; grey dashed lines) decadal gender ratio (proportion female co-authors) for all 544 proposed articles. Numbers above the graph indicate sample size (number of articles) used to calculate decadal means (‘←1’ indicates one article from 1858) (Darwin and Wallace 1858). For comparison, the proportion of female authorships in articles published from 1990–2011 in ecology and evolution (22.76% of 279012 total authorships; lower black horizontal dashed line) (West et al. 2013) are shown. Also shown are the 95% confidence limits of the proportion of female co-authors of the 100 top-ranked articles assessed by female (green shaded area: 0.029 to 0.093; Fig. 1b) and male voters (orange-shaded area: 0.001 to 0.029; Fig. 1b).

In the analysis of the genders of the lead authors only, a man was the lead author in 510 of the 544 proposed papers (93.8%). For the 100 top-ranked papers (read or not), 98 were led by a man; when men alone voted, 99 of the 100 top-ranked papers were led by a man, and when women alone voted, 96 were. The difference between female and male voters disappeared in the read-only list of the 100 top-ranked papers (93 were led by a man when women voted; 92 were when men voted).

Discussion

At least for the ecologist editors we contacted, both men and women tend to propose and rank articles more highly when they were co-authored by the same gender, indicating a degree of homophily for both men and women when assessing article importance. However, this effect disappeared after testing read-only articles. These results endure despite little evidence that male biologists view themselves as having higher self-perceived expertise than women (Laurance et al. 2011). Older papers with which ecologists are at least familiar are generally ranked higher (Courchamp and Bradshaw 2017), but because older articles had fewer female co-authors, all voting ecologists had little choice but to score the ‘classics’ more highly. When we restricted the ranked articles to those that voting ecologists claimed they had actually read, the relationships disappeared.

While these results could potentially be interpreted as a lingering, subconscious gender bias among ecologists, the dominance of male co-authors in the overall pool of proposed papers (especially for older articles) restricts inference. Once voters claimed to have read an article, the relationship between article rank and gender ratio disappeared. This read-only group of younger articles (by 14 years, on average) (Courchamp and Bradshaw 2017), combined with the observation that there is an increasing proportion of female co-authors on highly ranked articles, are nonetheless encouraging signs – our reported gender ratios for the most recent ‘recommended’ articles (> 25%, Fig. 2) agree with the approximate overall pool of female co-authors in the discipline (22.76% in ecology and evolution) (West et al. 2013) (Fig. 2), and the 24% of female corresponding authors on manuscripts submitted to Nature Ecology and Evolution in 2017 (The Editors 2018). It is also in line with the 22.1% of female editors in our pool of eight sources, suggesting the still – skewed sex ratio among editors does in fact mirror the availability of established ecologists of both genders. Although all of this shows a clear improvement over time, female ecologists are still in the minority in terms of high-ranking article authorships (< one third) and editorships (< one fifth). We hypothesize that female editors are less likely to contribute important articles, in part because they are disproportionately requested to take part in time-consuming tasks such as surveys, consortia, juries, and committees in an attempt to seek gender parity (Laurance et al. 2013), thus leaving less time to produce articles, but mostly because they are still in the minority at the highest levels of academia (and even less so several decades ago) (Aldercotte et al. 2017; Buckley et al. 2000).

We concede that our study was not specifically designed to test for the phenomenon of homophily, for it is plausible that the gender of some authors of papers both proposed and voted were unknown to some of the editors (especially those co-authors listed between the lead and terminal authors). Ideally, a follow-up questionnaire could be designed to ask the contributing editors to identify the gender of random subsets of authors to estimate the error probability in the attribution of author gender, but this was beyond the scope of our current assessment. Restricting the temporal window allowed for proposing articles toward more recent decades could also potentially provide a better evaluation of the modern tendency toward homophily, although this approach would not have allowed older articles to be proposed and assessed as potential classics in ecology.

Published evidence shows that gender bias in ecology is not strong, and in most cases is explained by a higher availability of established male ecologists due to the historical imbalance of gender in our discipline. However, even if that imbalance is slowly waning, there remains unconscious, gender-related biases. Our results suggest that (i) ecologists of both genders subconsciously commit homophily regarding their opinions of article importance, and (ii) the degree of homophily is higher in men than women. Potential solutions to reducing the incidence of homophily include increasing gender discussions in university teaching (Damschen et al. 2005), improving workplace flexibility (Barres 2006; Ceci and Williams 2011; Holt and Webb 2007), ensuring mixed-gender interview panels (Holt and Webb 2007), embracing positive discrimination (Barres 2006), encouraging double-blind reviews (Budden et al., 2008), and advocating alternative metrics of publication and citation performance (Cameron et al. 2013; Symonds et al. 2006). We add that all ecologists would benefit from serious, personal introspection about their own biases, because denial of one’s own contribution to the problem only serves to perpetuate it (Barres 2006). Consciously increasing the number of female ecologists among our students, in our labs, on our editorial boards, requested to review papers, and as co-authors will also help.

Author contribution

CJAB and FC collected the data, developed the concept, and designed the manuscript; CJAB did the analysis.

Authors Contribution ACI
CJAB 0.60 1.50
FC 0.40 0.67

Acknowledgements

We thank the many participating editorial members, as well as C Albert and GM Luque for help with the survey and article management. We are particularly grateful to the two reviewers for their objective, constructive, and high-quality reviews. FC was supported by BNP-Paribas and ANR grants, and CJAB was supported by BNP-Paribas and Australian Research Council grants.

References

  • Aksnes DW, Rorstad K, Piro F, Sivertsen G (2011) Are female researchers less cited? A large-scale study of Norwegian scientists. Journal of the American Society for Information Science and Technology 62: 628–636. https://doi.org/10.1002/asi.21486
  • Aldercotte A, Guyan K, Lawson J, Neave S, Altorjai S (2017) Equality Challenge Unit. Asset 2016: Experiences of gender equality in STEMM academia and their intersections with ethnicity, sexual orientation, disability and age. London, United Kingdom.
  • Borsuk RM, Budden AE, Leimu R, Aarssen LW, Lortie CJ (2009) The influence of author gender, national language and number of authors on citation rate in ecology. The Open Ecology Journal 2: 25–28. https://doi.org/10.2174/1874213000902010025
  • Buckley HL, Sciligo AR, Adair KL, Case BS, Monks JM (2014) Is there gender bias in reviewer selection and publication success rates for the New Zealand Journal of Ecology? New Zealand Journal of Ecology 38: 335–339.
  • Buckley LM, Sanders K, Shih M, Kallar S, Hampton C (2000) Obstacles to promotion? Values of women faculty about career success and recognition. Academic Medicine 75: 283–288.
  • Budden AE, Tregenza T, Aarssen LW, Koricheva J, Leimu R, Lortie CJ (2008) Double-blind review favours increased representation of female authors. Trends in Ecology and Evolution 23: 4–6. https://doi.org/10.1016/j.tree.2007.07.008
  • Ceci SJ, Williams WM (2011) Understanding current causes of women’s underrepresentation in science. Proceedings of the National Academy of Sciences of the USA 108: 3157–3162. https://doi.org/10.1073/pnas.1014871108
  • Darwin CR, Wallace AR (1858) On the tendency of species to form varieties; and on the perpetuation of varieties and species by natural means of selection. Journal of the Proceedings of the Linnean Society of London Zoology 3: 45–62. https://doi.org/10.1111/j.1096-3642.1858.tb02500.x
  • Fox CW, Burns CS, Meyer JA (2016) Editor and reviewer gender influence the peer review process but not peer review outcomes at an ecology journal. Functional Ecology 30: 140–153. https://doi.org/10.1111/1365-2435.12529
  • Grod ON, Budden AE, Tregenza T, Koricheva J, Leimu R, Aarssen LW, Lortie CJ (2008) Systematic variation in reviewer practice according to country and gender in the field of ecology and evolution. PLoS One 3: e3202. https://doi.org/10.1371/journal.pone.0003202
  • Gurevitch J (1988) Differences in the proportion of women to men invited to give seminars: is the old boy still kicking? Bulletin of the Ecological Society of America 69: 155–160.
  • Holt A, Webb T (2007) Gender in ecology: where are the female professors? Bulletin of the British Ecological Society 38: 51–62.
  • Laurance WF, Laurance SG, Useche DC (2011) Gender differences in science: no support for the ‘Homer Simpson effect’ among tropical researchers. Trends in Ecology and Evolution 26: 262–263. https://doi.org/10.1016/j.tree.2011.03.010
  • Lincoln AE, Pincus S, Koster JB, Leboy PS (2012) The Matilda effect in science: awards and prizes in the US, 1990s and 2000s. Social Studies of Science 42: 307–320. https://doi.org/10.1177/0306312711435830
  • Manly BFJ (2006) Randomization, Bootstrap and Monte Carlo Methods in Biology (3rd edn). Chapman and Hall/CRC Press, London.
  • Monosson E (2008) Motherhood, the elephant in the laboratory: women scientists speak out. Cornell University Press, Cornell, New York.
  • Moss-Racusin CA, Dovidio JF, Brescoll VL, Graham MJ, Handelsman J (2012) Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences of the USA 109: 16474–16479. https://doi.org/10.1073/pnas.1211286109
  • Rudman LA (1998) Self-promotion as a risk factor for women: the costs and benefits of counterstereotypical impression management. Journal of Personality and Social Psychology 74: 629–645. https://doi.org/10.1037/0022-3514.74.3.629
  • Symonds MRE, Gemmell NJ, Braisher TL, Gorringe KL, Elgar MA (2006) Gender differences in publication output: towards an unbiased metric of research performance. PLoS One 1: e127. https://doi.org/10.1371/journal.pone.0000127