Women's Participation in Open Source Software: A Survey of the Literature

Introduction

The effects of genderIn this study, we use the term “gender” as a socially constructed concept (Butler, 1999), where gender identification, display, and performance may or may not align with sex assigned at birth. To reflect this social concept of gender, in this paper we use the term “women” and “men” as a shorthand for people who self-identify as such. diversity in Free and Open Source Software (OSSin this study we use OSS, but other possible acronyms include FLOSS, F/OSS, or FOSS) projects has gained increasing attention from practitioners and researchers (Wurzelová et al., 2019; Robson, 2018; Bosu and Sultana, 2019; Vasilescu et al., 2015b; Catolino et al., 2019; Fossatti, 2020). Gender diversity has a positive effect on productivity as it brings different perspectives together, improving outcomes (Vasilescu et al., 2015b), problem-solving capacity, and leading to a healthier work environment (Earley and Mosakowski, 2000). A gender-diverse team has been shown to increase innovation and productivity in software engineering (Østergaard et al., 2011; Vasilescu et al., 2015b; Tourani et al., 2017).

Although OSS organizations have been taking actions to increase gender diversity and placing more women in leadership positions, the numbers are still low. Women represent only 5.2% of the contributors in Apache Software Foundation (Sharan, 2016) and 9.9% in Linux kernel (Bitergia, 2016), two of the largest and best-known OSS communities. Existing research suggests that gender bias and sexist behavior pervade OSS (Nafus, 2012; Gallego et al., 2015). While OSS projects idealize a strict meritocracy wherein quality speaks for itself (Feller and Fitzgerald, 2000), several biases undermine women, who feel their quality is not able to speak for itself and report experiencing “impostor syndrome” (Vasilescu et al., 2015b). Gender biases can represent a “glass floor” and a persistent barrier to entry (Mendez et al., 2018; Padala et al., 2020). To avoid negative gender-based biases, some women hide their gender by using neutral aliases in their profile (Lee and Carver, 2019; Vasilescu et al., 2015a).

Although many published scientific studies investigate women’s participation in OSS, the findings are scattered across multiple fields and venues, and the results are disconnected from one another. Summarizing and aggregating existing research can help communities fully comprehend women’s participation in OSS to define proper mechanisms to increase the number of women contributors and produce new knowledge by connecting previously dissociated research. Therefore, the purpose of this study is as follows: To review, summarize, and synthesize the current state of research on women’s participation in Open Source Software.

Through a systematic mapping of the literature, including database search, backward and forward snowballing, and input from prolific authors, we selected and retrieved information from 51 primary studies published between 2000 and 2021. Our contributions include characterizing who the women contributors to OSS are, what are their motivations to contribute, what types of contributions they make, what challenges they face while contributing, and what strategies can mitigate these challenges and support women’s participation in OSS.

Research Method

We performed a systematic mapping of the literature (SML) because we expected the existing research to be fragmented and not follow common terminology or use established theoretical concepts. The systematic mapping of the literature is suitable in such circumstances because it maps the available evidence (Petersen et al., 2008; Petersen et al., 2015), categorizing and aggregating knowledge dispersed across disconnected studies. This section details our method, which is based on established guidelines (Petersen et al., 2015; Kitchenham et al., 2010).

Fig 1 depicts the steps and phases of our research method. Three researchers held four meetings to formulate the protocol. We piloted the search in Google Scholar and refined the search string, validating it with four control papers (Robles et al., 2016; Vasilescu et al., 2015a; Terrell et al., 2017; Canedo et al., 2020). We then executed the search string in the selected databases, conducted backward and forward snowballing, contacted the most prolific authors, applied the inclusion and exclusion criteria to filter the studies, and extracted data to answer the research question.

To understand women’s participation in OSS projects, we formulated the following research question: “How has the participation of women in OSS been characterized in the literature?” While reading the primary studies, we categorized the participation of women in OSS according to five perspectives that emerged from our analysis, as discussed in the results section.

We selected papers for our study based on the following Inclusion (IC) and Exclusion (EC) criteria. See supplemental materialhttps://figshare.com/s/6c54861e6829cb65f3cf for additional details, including how the criteria were applied to each paper.

(+) IC1. The paper should characterize or be related to women’s participation in Free/Open Source Software projects

(-) EC1. The publication is just an abstract

(-) EC2. The publication is not written in English

(-) EC3. The publication is a copy or an older version of another article already considered

(-) EC4. The publication is not a peer reviewed paper

(-) EC5. The publication is a doctoral/master’s dissertation, research plan, short paper, or report

(-) EC6. It was not possible to access the complete work.

1.2. Databases and Search String

We used a hybrid strategy (Mourão et al., 2020) to collect primary studies, including database search (IEEE Xplore, Scopus, and ACM Digital Library) (Petersen et al., 2015), snowballing, and contact with the most prolific authors. These databases are suggested by Kitchenham and Charters (Kitchenham and Charters, 2007) to conduct secondary studies in computer science and software engineering topics, and in our experience provide good coverage. Similar to related work (Cosentino et al., 2017; Munir et al., 2016; Marques et al., 2014; Elberzhager et al., 2012), we limited the search to title, abstract, and keywords. Table 1 provides the search string used for each database.

The first part of the search string (before the AND) aims to limit the results to studies about women or the diversity and heterogeneity they can represent. The second part of the search string (after the AND) seeks to limit results to Free/Open Source Software. All searches were completed on November 15, 2020.

As we depicted in Table 1, to avoid missing studies that do not directly mention the terms “women” or “female” in the title, abstract, or keywords, we included in the search string more generic terms such as “diversity” and “gender”, understanding that diversity can include other aspects (e.g. culture, age, tenure) and that gender can refer to genders beyond our focus on women.

1.3. Control Papers and Piloting

Before searching the selected databases, we conducted exploratory searches using Google Scholar to refine the inclusion and exclusion criteria, research question, synonyms for the search string, and information extraction template. From that exploratory search and based on our experience, we selected four well-known, relevant studies to act as control papers (Robles et al., 2016; Vasilescu et al., 2015a; Terrell et al., 2017; Canedo et al., 2020). In our pilot studies to test the protocol, all four control papers were found.

2. Conducting the Systematic Mapping of the Literature

We executed the research protocol, summarized in Fig.1, which comprised the following six steps: execute the search string in selected databases, remove duplicates between databases using manual de-duplication, read and filter the papers applying IC and EC, perform the snowballing, and contact the most prolific authors to find additional studies. Fig. 1 shows the number of papers selected in each step of the selection process. Throughout the paper, we refer to the selected primary studies with ‘PS’ (e.g., (PS1)) to distinguish them from citations to other references (e.g., ).

Step 1: Search. In November 2020, we performed the search on the selected search databases and found 2,396 papers.

Step 2: Remove Duplicates. We used a spreadsheet to store and organize the retrieved papers, ordered the papers by title, and manually removed 482 duplicates.

Steps 3 and 4: Select Primary Studies. Selection of papers followed the guidelines proposed by Kitchenham and Charters (Kitchenham and Charters, 2007) and Petersen et al. (Petersen et al., 2015) using the steps shown in Fig.1. In the first step, we analyzed only the papers’ title and abstract, consulting the entire text only when necessary to reach a confident judgment. In the second step, we filtered the papers considering the full-text reading. In both filters, papers were included to match the inclusion criteria or excluded if they fit any exclusion criteria.

Step 5: Backward and Forward Snowballing: For Backward Snowballing, we conducted a single-step snowballing sampling, checking the primary studies references and looking for additional relevant papers. This step resulted in four additional primary studies. For Forward Snowballing, we also conducted a single-step sampling, checking what other papers cited the selected papers. This step resulted in nine additional primary studies.

Step 6: Contact prolific authors. We reviewed the publication records and contacted the top 3 most prolific authors according to our primary studies, who are arguably specialists in the field. We searched for the Google Scholar profile and personal website of these authors and sent emails to them showing the list of primary papers and asking if they suggest others that we missed and should be included. This step resulted in three additional primary studies.

3. Extracting and Analyzing Data

We extracted quantitative data from the papers to analyze what the literature had presented about women’s representation in OSS projects and the rate of acceptance of women’s contributions. We also qualitatively analyzed the papers and inductively applied open coding to create categories of data and organize what was reported by the literature about the demographics of women who contribute to OSS (Fig. 5), their motivations (Fig. 6), the types of contributions they make (Fig. 7), the challenges (Fig. 10), and strategies (Fig. 11). For types of contributions and motivations, the coding approach was informed by previous work (Von Krogh et al., 2012; Trinkenreich et al., 2020). For challenges and strategies, we built post-formed codes and associated them with respective parts of the studies’ text. Three of the authors conducted three card sorting sessions (Spencer, 2009) and discussed the codes and categorization until reaching consensus about the meaning of and relationships among the codes.

4. Characterizing the Studies

In this section, we present some characteristics of the primary studies (e.g., when they were published and who published them).

Fig. 3 shows the number of primary studies published per year. The earliest paper was published in 2000. There were up to six papers per year until 2019, after which we saw a noticeable peak. We included one paper from 2021 through forward snowballing, but decided to not include it in Fig. 3 since it was outside the time period of our analysis.

We ranked all 118 authors by the number of publications in the sample to investigate the most prolific authors. Among them, 95 authors had one paper; 17 authors had two papers; five authors had three papers; two authors had four papers; and one had 6 papers. The top authors were Alexander Serebrenik (10) and Bodgan Vasilescu (7).

Results

In this section, we present the results based on the perspectives that emerged from our analysis, namely the characterization of women’s participation (Section 3.1), motivation (Section 3.2), types of contributions (Section 3.3), challenges (Section 3.4), and the identification of strategies proposed in the literature to promote women’s participation (Section 3.5).

The literature presents different statistics for women’s representation in OSS, which were measured in different ways, at different moments, and considering different sets of projects. We collected these results to provide a comprehensive view of the state of women’s participation. Organizing data obtained from different sources may provide researchers with an expanded view of the phenomenon under study, prompting new insights and allowing for further discoveries (Robles et al., 2014). Additionally, analyzing women’s representation at different time periods can help us understand the evolution of gender imbalance. Understanding the characteristics (e.g., education level, family status, diversification of projects, time to volunteer, tenure) can help communities and researchers refine their strategies to attract profiles of women who still do not participate and retain those who currently contribute to OSS.

Several primary studies quantified women’s participation using various methods, including mining software repositories and mailing lists, surveys, and participation in mentorship programs. Table 3 summarizes the frequency of women’s participation reported by the primary studies.

Participation of women in OSS found via mining software repositories: From a dataset of 23,493 GitHub projects, Vasilescu et al. (PS17) used the genderComputer tool (PS49) (with 93% of precision) to identify gender, based on personal names, and, if available, countries, of 873,392 GitHub contributors. They found 91% men and 9% women. From 5,250 OpenStack contributors, Izquierdo et al. (PS12) inferred gender using the genderize.io tool and found that 10% are women. From a dataset of 8,338 GitHub projects, Prana et al. (PS16) found that the percentage of new GitHub accounts created by women remained around 10% from 2014 to 2018. Bosu and Sultana (PS9) analyzed a dataset of 683,865 code review requests from 10 popular OSS projects. Authors inferred gender using the Gerrit-Miner tool (Bosu and Carver, 2013) and found that women represent 6.70% (out of 4,543) of non-casual developers (those who submitted at least five code changes) and only 4.27% (out of 936) of core developers (those who are the top 10% developers in terms of the number of code commits in a project). From a random sample of 300,000 GitHub users from a dataset with 16M users, Qiu et al. (PS46) inferred gender using genderComputer and NamSor tools and identified 9.7% as women. Terrel et al. (PS4) analyzed a GitHub dataset with 4,037,953 profiles and identified the gender of 1,426,127 (35.3%) users through their public Google+ profiles. Based on those profiles, the authors analyzed pull-request submission and acceptance by women and men. They found that 8,216 of the pull-requests were submitted by women (5.2%) and 150,248 (94.8%) by men. Imtiaz et al. (PS3) used the same GitHub dataset of (PS4) and identified 529,253 men (93.7%) and 35,676 women (6.3%). Kofink (PS10) also analyzed a dataset of 1,811,631 pull requests and found that 4.5% were submitted by women and 95.5% by men. Zacchiroli (PS44) also analyzed the authors of contributions. With 1.6 billion commits from the combined projects of GitHub, GitLab, and other development forges (using the Software Heritage projecthttps://www.softwareheritage.org/), corresponding to the development history of 120 million projects. Zacchiroli found that 33 million distinct people authored contributions over 50 years, and that there is a significant growth of active women as authors from around 4% in 2005 to 10% in 2019.

Distribution of women who participate in mentorship programs: By analyzing the gender of Google Summer of Code participants from 2016 to 2018, Canedo et al. (PS43) found that while there is a minor variation across the years, the volume of women stayed close to 11.98% of the total number of participants in the program. The Google Summer of Code (GSoC) is a 3-month OSS engagement program that offers stipends and mentorship to students willing to contribute to OSS projects (Silva et al., 2017; Trainer et al., 2014; Silva et al., 2020b).

Distribution of women in OSS through surveys: Mani and Mukherjee (PS2) and Robles et al. (PS5) analyzed the same OSS 2013 survey data (Arjona-Reina et al., 2014). 2,183 OSS contributors answered the survey, 226 of whom identified as women (10.35%). Lee and Carver (PS21) received 119 answers to their questionnaire, of which 10.92% of respondents identified as women, while Gerosa et al.’s (PS51) questionnaire received 224 answers with 7.6% who identified as women.

In summary, the primary studies reported women’s participation ratios ranging between 4% to 14% across different measurements and OSS communities. When we analyze the distribution over time based on when the primary studies were published (Fig 4b), barring some fluctuation women’s participation ratio stays stable at around 10%. However, when taking a broader timeline view, Zacchiroli (PS44), through his analysis of public code contributions over the last 50 years, found that women’s contributions appear to be on the rise and are rising faster than those of male authors. This shows that while much still needs to be done, OSS projects are becoming more gender diverse, albeit slowly.

To gain a deeper understanding of the demographics of women who contribute to OSS projects, we analyzed available data reported in the primary studies along the following criteria: education level, time dedicated to contributions, diversification of projects, family status, and tenure (Fig 5).

Education Level. Based on Stack Overflow’s 2018 developer survey that included 43,000+ OSS contributors, Wurzelova et al.’s (PS7) study reported that 82.8% of women who contribute to OSS are at least undergraduate students, compared to 76% of the overall rate of contributors (Overflow, 2018). Mani and Mukherjee (PS2) found a similar rate (81.4%) using data from the 2013 FLOSS Survey (226 women), while the corresponding figure for the whole dataset is 72%. All of the 36 developers who identified as women on the survey from Canedo et al. (PS6) were at least undergraduate students. A substantial number of contributors from the three studies were post-graduates, who either achieved a Master’s degree—11.6% (PS6), 27.6% (PS7)— or Ph.D.—4.2% (PS7), 10% (PS2), 22.9% (PS6).

Volunteering. Approximately half of the 226 women (53.59%) who answered the 2013 FLOSS questionnaire devote less than five hours per week to OSS projects. Only 14.77% of the women who answered this questionnaire dedicate more than 40 hours per week (PS5), which can represent OSS as a full-time job. Although not mentioning the number of hours per week, according to the Powell et al.’s (PS28) results, 89% of women said they contribute to OSS both at home and at work, which includes bringing their work home and contributing to OSS during their leisure time.

Diversification of projects. By mining software repositories, three studies (PS3; PS4; PS46) concluded that women concentrate their efforts on fewer projects than men. From a dataset of 152,534 pull-requests created by 20,926 women and 3,135,384 pull-requests created by 308,062 men, Imtiaz et al. (PS3) concluded that women’s pull-requests are concentrated in fewer projects and fewer organizations than men’s. Indeed, Qiu et al. (PS46) analyzed a balanced sample of the dataset, including 28,995 women and 29,096 men and also concluded that women tend to concentrate their contributions in fewer projects than men. From a dataset of 1,426,127 users whose gender could be identified, Terrell et al. (PS4) analyzed the acceptance of submitted pull-requests and concluded that women contribute to fewer projects than men. From another perspective, Vasilescu et al. (PS1) ran a survey answered by 199 women and 611 men, in line with the previous studies, and concluded that women own fewer public repositories than men.

Family Status. Almost half of the 226 women who took part in the FLOSS 2013 survey are not married or did not live with their partners (PS2). This rate was composed of 35% single women, over 11% not living with their partners, 3% living with their partners, 3% married, 0.1% separated from their partners (PS2) and 20% living with children (PS2; PS5).

Tenure. Most (77.2%) of the 199 women who answered Vasilescu et al.’s (PS1) questionnaire contributed to OSS projects for fewer than five years. Authors found that women have on average six years of experience in IT/programming, a significantly lower tenure than men with nine years of experience.

Women are underrepresented in central OSS roles, although they are better represented earlier in the joining process (e.g. in mentoring programs). Women’s participation has increased in recent times, but those who contribute reported having a few hours per week to devote to OSS. Several studies found women’s participation ranges from 4.3% to 14.2%.

2. What motivates women to contribute to OSS projects?

Research has shown that women are generally more motivated to use technology to accomplish a goal rather than for fun (Burnett et al., 2010). For the past 20 years, much academic work has theorized about and empirically examined OSS contributors’ motivations. Retrieving and consolidating women’s motivations from the existing studies is relevant to communities seeking to recruit and retain women. Proper management of motivation and satisfaction helps software organizations achieve higher productivity levels and avoid turnover, budget overflows, and delivery delays (Beecham et al., 2008; França et al., 2011; da Silva and França, 2012).

We consolidated the studies reporting women’s motivation to participate in OSS projects, aggregating the results according to Von Krogh et al.’s categories (Von Krogh et al., 2012). Von Krogh et al. (Von Krogh et al., 2012) surveyed the literature and identified ten categories of motivation, grouping them as intrinsic, internalized-extrinsic, and extrinsic. Intrinsic motivation (Enjoyment and Fun, Kinship, Ideology, and Altruism) moves a person to act for the fun or challenge entailed rather than in response to external pressures or rewards (Ryan and Deci, 2000). In contrast, extrinsic motivations (Career and Pay) are based on outside incentives when people change their actions due to an external intervention (Frey, 1997). Contributors can also internalize extrinsic motivators (Learning, Own-Use, Reciprocity, and Reputation) as self-regulating behavior, rather than as an external imposition (Deci and Ryan, 1987; Roberts et al., 2006). We summarize our findings organized in these higher level categories in Table 4 and Figure 6.

Among all 10 categories, the literature showed that women see Enjoyment and Fun, Reciprocity, Kinship, and Pay differently from the motivation reported in the literature (Hertel et al., 2003; Hars and Ou, 2004; Lakhani and Wolf, 2003; Ghosh et al., 2002; PS51)—based on surveys answered predominantly by men. Enjoyment and Fun consistently has been reported as a top driver to contribute to OSS in motivation surveys (Lakhani and Wolf, 2003; Hars and Ou, 2004; Ghosh et al., 2002; Hertel et al., 2003; PS51). However, according to the stratified analysis on Gerosa et al.’s (PS51) study, no women reported this motivation to join (and one–6%–stayed because of this), while 7% of the men who joined were motivated by fun and 20% continued because of it.

The opposite trend is observed for Reciprocity, Kinship, and Pay. Reciprocity appeared as one of the top motivators for women in Gerosa et al.’s (PS51) work—39% of the women reported this as motivation to continue (versus 15% of the men). In general surveys (Lakhani and Wolf, 2003; PS51), reciprocity is not among the top motivators. Regarding Kinship, while 39% of the 226 women who answered the FLOSS survey joined because of this motivation, 31% continued because of it (PS5). Most (64%) of the 22 women who answered Prana et al.’s survey (PS16) select a project in which friends and colleagues also participate. As part of kinship, peer parity also plays a role in women’s motivation (PS50). David and Shapiro (PS50) found that social connections with other developers influence women’s choices. However, no women from Gerosa et al.’s (PS51) study joined OSS because of kinship. Kinship is not top-ranked in general surveys (Hars and Ou, 2004; Lakhani and Wolf, 2003; Ghosh et al., 2002), but this trend has been changing with the rise of social coding platforms (PS51). Regarding Pay, Prana et al. (PS16) indicated that payment is a greater incentive for women than men (64% vs. 35%). This was echoed by women interviewees from Balali et al. (PS18). However, the difference was not noticed in Gerosa et al.’s (PS51) work, in which men and women equally reported money as a reason to continue contributing (14% and 11%, respectively).

For other motivations, we found that women follow a similar trend as reported in the general literature. Learning, for example, has been frequently reported as a key motivation to contribute to OSS (Lakhani and Wolf, 2003; Hertel et al., 2003; Hars and Ou, 2004; Ghosh et al., 2002) and most (68%) of the 226 women who took part in the FLOSS 2013 survey joined for Learning, and 65% continue because of it (PS5). The same happens for Altruism, which is a common motivator in OSS (PS51; Ghosh et al., 2002) and is relevant for women as well—37% of the women who took the FLOSS 2013 survey reported that they joined to share knowledge (PS5) and 22% from another work continued because of it (PS51).

Similar rates were also found for Career, Own-Use, and Reputation. Regarding Career, only 4% of the 226 women who took the FLOSS 2013 survey and none who took Gerosa et al.’s (PS51) survey reported joining to improve their careers. Similar rates were found for men (5% and 8%, respectively). For Own-Use, David and Shapiro (PS50) found that women are motivated by their employment-related needs and one third (6 out of 18) of the women from Gerosa et al.’s (PS51) reported this motivation to join OSS. This is in line with previous research that shows that women are more motivated to use technology for what it enables them to accomplish (Burnett et al., 2010). Reputation was not a top motivation for contributors from all genders in the early 2000s (Lakhani and Wolf, 2003; Ghosh et al., 2002) and still is not. Only 4% of the men from from Gerosa et al.’s (PS51) joined because of reputation, while no women reported it.

Finally, an interesting case was Ideology, which was top-ranked in general surveys from the 2000s about motivation to join OSS (Lakhani and Wolf, 2003; Hertel et al., 2003; Hars and Ou, 2004; Ghosh et al., 2002). Ideology is usually captured by motivations such as “software should be free for all,” “free to modify and redistribute,” or “OSS should replace proprietary software.” (Von Krogh et al., 2012). In the recent survey by Gerosa et al. (PS51), this motivation has dropped some and was mentioned as a reason to join by only 11% of the women and 11% of the men. When considering only women, we can see that there was also a drop compared to the FLOSS survey, in which ideology was mentioned by 28% of the women (PS5).

Reciprocity, Kinship and Pay are motivations especially relevant for women. On the other hand, enjoyment and fun motivate more men than women. Altruism, learning, and own-use motivate both men and women; career and reputation are low-ranked for both genders, and ideology was relevant in the past but has lost importance for both genders over the years.

3. What types of contributions do women make in OSS projects?

The OSS landscape has changed since the early 2000s to include the participation of ever more people and companies. Project-centric roles are becoming more established, and OSS projects increasingly include community-centric roles, which relate to areas beyond programming (Trinkenreich et al., 2020). Understanding how women contribute to OSS projects can help attract both code developers as well as those interested in being part of OSS via non code-centric roles.

While the primary type of contribution in OSS projects is related to code development, the roles available in OSS projects go beyond the project-centric roles (Trinkenreich et al., 2020). They include many people who work “behind the scenes” to drive and sustain the community (PS11). We categorized the activities reported by primary studies according to a framework of OSS roles from a previous study (Trinkenreich et al., 2020) and with two perspectives: coders or non-coders, and project-centric or community-centric (Fig.7). Afterward, we present details about women’s contributions as coders and non-coders reported by primary studies.

Only 31% of the 226 women who answered the FLOSS survey (PS5) contribute to OSS projects as code developers, and 24% perform coding in parallel with other roles.

Core Developers. The classic hierarchical model of coders in OSS development communities is described as a core-periphery structure, with a small number of core developers and a large set of peripheral developers (Nakakoji et al., 2002). Core developers are code contributors involved with an OSS project for a relatively long time who make significant contributions to guide the project’s development and evolution. Due to their relevant contributions and interactions, core developers often play leadership roles in OSS projects (Ye and Kishida, 2003). Canedo et al. (PS6) found women as core developers in only 5.24% of the 711 GitHub analyzed projects. Of all the core developers, only 2.3% were women. From a dataset of 683,865 code review requests from ten popular OSS projects, Bosu and Sultana (PS9) found that women comprise a maximum of 10% of core developers across all ten projects. Following a classification of commit types from Hattori and Lanza (Hattori and Lanza, 2008), Canedo et al. (PS6) concluded that women who are core developers contribute more with corrective and reengineering commits than forward engineering and management commits. Moreover, when describing the commits, women present a more detailed message explaining their contribution changes than men. (PS15) evaluated the interactions in projects Angular.js, Moby, Rails, Tensorflow, Django, Elasticsearch and found that women who are core developers are more likely than men to interact with other contributors, evolve similarly to men within the project, and, though underrepresented, contribute to building sustainable social capital for OSS.

Preferred Technologies. From the 711 projects analyzed by Canedo et al. (PS6), women represented 8.8% of the core developers in projects that are based on Scala programming language, 8.7% when based on CSS, 6.3% when based on TypeScript, 5.6% when on based Swift, and only 1% when the project is based on PHP and Shell programming languages. Still, from the same data, even projects written in TypeScript (10.93%) have at least one woman as a core developer, and 2.17% of projects using PHP have at least one woman as a core developer. Considering the data from the six projects analyzed by El Asri and Kerzazi (PS15) (Angular.js, Moby, Rails, Tensorflow, Django, Elasticsearch), women represented at least 4.8% of core developers for the projects based on Python, at least 4.5% for C++, and at least 4.2% for Java. From the 158,464 of pull-requests for which Terrell et al. (PS4) could identify gender, women had a greater rate of accepted pull requests in Ruby, Python, and C++. We present the percentages of women as developers for each programming language as reported in the primary studies in Fig 9.

Code reviewers. Peer code review is a practice in software engineering. A code developer submits the code produced to another person (peer) to evaluate and find possible errors before merging the code to the project codebase (Bacchelli and Bird, 2013). According to the GitHub dataset of six projects evaluated by Paul et al. (PS13), women are more likely to write reviews expressing sentiments in the text to another woman than to a man during code reviews. Huang et al. (PS14) used medical imaging and eye-tracking to evaluate the visual and cognitive processes and patterns of neural activation followed by reviewers while performing code reviews. Authors found that women spent significantly more time analyzing pull request messages and author pictures (regardless of their identity) than the code itself when performing code reviews.

3.2. Non-Coders

Almost half (45%) of the 226 women who answered the FLOSS 2013 survey take part in non-coding activities, and 24% perform code-related activities in parallel with other roles (PS5).

OSS Community Manager. After analyzing the career pathways followed by 17 contributors, Trinkenreich et al. (PS11) presented a set of community-centric roles, including community founders and managers, strategists, mentors, writers, license managers, treasures, and advocates. The contributors who play these roles are usually “hidden figures,” who are not visible when analyzing the data from projects’ repositories or coding platform websites. 11 (out of the 12) women interviewed in this study play community-centric roles.

Project Leader/Manager. More than half (51.49%) of the 226 women who answered the FLOSS 2013 survey participate in community leader, coordinator, or administrator roles, while only 5% of those women coordinate more than three projects (PS5). Women in OpenStack who play leadership roles represent 7% of the project committee members, 8% of the project team leaders, 9% of the project board directors, 7% of the technical committee, 8% of the working group leaders, and 23% of project ambassadors (PS12).

Women are more present in community-centric than in project-centric roles; almost half make non-code contributions. However, they often play both coding and non-coding activities in parallel. Very few projects have at least one woman as a core developer.

4. What challenges do women face when contributing to OSS projects?

Previous work investigated challenges faced by OSS contributors who are mentors and newcomers (Steinmacher et al., 2015; Balali et al., 2018; Mendez et al., 2018; Steinmacher et al., 2016). Some of them report gender bias as a challenge (Balali et al., 2018), and others report barriers faced by women (Dias Canedo et al., 2019). Anecdotes about gender bias appear across the literature (Terrell et al., 2017; Kofink, 2015), and women have been reported to feel that such biases are to blame for their contributions’ comparatively low acceptance rate (Canedo et al., 2020). We aggregated the scientific evidence about challenges and gender bias. Understanding the nuances of women’s challenges to contribute to OSS can help communities plan strategies to mitigate these challenges and thereby attract and retain more women.

Women mainly face socio-cultural challenges when contributing to OSS (PS21), which can also influence their decision to leave an OSS project (PS13). From the 37 women who answered Powell et al.’s (PS28) survey, 50% indicated they had witnessed gender-based discrimination within the OSS community either online, in meetings, or in class, and 50% said they had experienced harassment online or offline. Gender-related incidents can be so severe that they motivate women to leave an OSS project (PS1). Indeed, in another survey (PS21) women reported that they drop out when the OSS project does not prioritize diversity. Leaving an OSS project is a decision that impacts more women than men—according to Qiu et al. (PS46), women are 27% more likely to disengage from GitHub. Understanding the reasons behind the decisions to step out of a project can help create strategies to increase retention in OSS. Kuechler et al. (PS23) suggest that women drop out because the OSS project is not aligned to their motivations or due to the unappealing and hostile social dynamics in projects.

We summarize the challenges found in the literature, which were all socio-cultural, in Table 5 and Fig. 10 and mark with an asterisk (*) the ones reported as a challenge that ultimately can cause women to leave OSS. Next, we present and explain each of them.

Lack of Peer Parity. Most women (72%) feel outnumbered and 24% feel alienated (PS28). Women reported to feel more comfortable and accepted by their same gender counterparts (PS18) and feel frustrated when there is no peer parity (PS1). This problem worsens in medium-size projects in which all contributors are men, as they may form a clique that a woman contributor could have difficultly breaking into (PS27). “It is not so common to find many girls in technical teams” (PS6), and in the face of this lack of parity women reported feeling invisible in larger men-dominated groups (PS24).

Non-inclusive communication. Discriminatory expletives, swear words, and negative critiques often used in code reviews and mailing lists may be insulting to women. The negative workplace experience of encountering words that are demeaning to them in the mailing lists can cause women to leave OSS projects (PS13). Awkward communication styles (PS18) in acrimonious talk about which code piece should be incorporated (PS22), and terms usually associated with men (e.g., “guys”) can demotivate women (PS27).

Toxic Culture. Incidents of symbolic violence and harassment against women can hinder their access to the community, such as when men decide to ‘hire that one because she is hot’ (PS24). Geek-saturated communities like Slashdot are often unwelcoming and hostile environments (PS25). Additionally, women are sexualized in OSS (PS8), facing judgment, abuse, hostility, and discrimination (PS42). While hurtful and offensive talk is openly addressed to them, women are obliged to remind men not to “stare and point” at them (PS22). From the 13 women who answered a questionnaire sent to contributors of 15 OSS projects, 38% had suffered some incidents of sexism, including sexist statements or assumptions, being ignored, insinuations that they had it easy because they were women, and being simultaneously held to higher standards than men and underestimated (PS21). Moreover, the authors found that women had trouble being taken seriously and needed to prove themselves (prove-it-again (PS3)). According to (PS23), public flaming and aggression can be enough to distort women’s participation, as they may already be hesitant about how they will be received.

Impostor Syndrome. Women tend to be risk-averse (Dohmen et al., 2011) and have low computer self-efficacy (Burnett et al., 2010, 2011; Cazan et al., 2016; Hartzel, 2003; Huffman et al., 2013; Singh et al., 2013), which can affect their behavior with technology, causing women to be less confident in their ability to complete tasks and blame themselves if there is a problem (PS29; PS30; PS31). Women find it challenging to directly translate competence to confidence without social attraction (being liked by the other community members in terms of having a rapid increase of followers). Consequently, initiating a pull-request to a new repository can be problematic due to women’s competence-confidence gap (PS32). Even understanding that confidence is an essential factor when entering OSS (PS28), women face a lack of self-efficacy (PS21; PS18). “Despite having proved [their] competency in certain areas of the code/project, [their] opinion is rarely or never asked for” (quotation from Vasilescu et al. (PS1)). Still, Imtiaz et al. (PS3) found that women tend to be more restrained than men in general. Despite being knowledgeable and professionally well-settled, women may be more reluctant to publicly display their work (PS1).

Community Reception Issues. Women reported to feel restrained when communities nullify them when they do not have enough skills to provide contributions on their first day (PS20). When trying to find a mentor, upon discovering their mentee’s gender, men mentors can treat the relationship as a dating opportunity (PS22). This makes finding a mentor an arduous task, which includes attracting attention and breaking into a close-knit OSS community (PS21). Many women use fake GitHub accounts and hide their gender, “so that people would assume [they] were male” (PS1).

Stereotyping. Pre-existing stereotypes (PS8; PS28), gender roles, and “macho” attitudes can cause gender inequalities in OSS communities (PS24). Women are boxed into specializations despite their manifest protest against it, as the legal case against the front-end/back-end distinction has shown (PS26). Additionally, men often treat women as if they were their mothers, asking for advice about how to dress and behave and then refusing to enter into a technical dialogue thereafter (PS22).

Work-life Balance Issues. Women that participated in Lee and Carver’s (PS21) study reported a lack of time and family responsibilities. Only women from this study, and not from other studies, reported family responsibilities as a challenge.

Gender-Biased Peer-Review Even in a population of core developers, one-third of the 36 women who answered Canedo et al.’s (PS6) survey reported they believe that reviewers had not accepted at least one of their contributions due to gender bias. Moreover, 11.4% of the women participants recognize gender bias while someone assesses their contributions. Although women can have merge acceptance rates nearly equivalent or slightly higher than men (PS4; PS10), according to Terrell et al. (PS4), there is a bias against women’s contributions when the gender is identified. The authors found that women have a 12% lower acceptance rate when they explicitly identify themselves as women, comparable to 3.8% for men who disclose their gender. However, authors found that women tend to have their pull-requests accepted at a slightly higher rate (78.7%) than men (74.6%) when not identifying the gender, regardless of experience level. While less experienced developers making their initial pull-requests do get rejected more often, women generally still maintain a higher rate of acceptance throughout. Bosu and Sultana (PS9)’s study corroborated this bias that women face by using three additional metrics (first feedback interval, review interval, and code churn per comment). The study analyzed ten projects and found explicit biases against women in three of the analyzed projects (Android, Chromium OS, and LibreOffice). Women had lower code acceptance rates than men, and had to wait longer to receive initial feedback for their code changes and to complete code reviews. The code submitted by women also had lower churn per comment in both Android and Chromium OS. The study showed that Android and LibreOffice stand out in prominent gender biases, where women had 10% lower acceptance rates then men, and review intervals that last three times longer than men. On the other hand, three other projects indicated biases favoring women (oVirt, Qt, and Typo3).

Women face many socio-cultural challenges that include: missing having other women around (lack of peer parity), encountering offensive language in mailing lists (non-inclusive communication), suffering symbolic violence and harassment (toxic culture), avoiding initiating a pull-request due to lack of confidence (impostor syndrome), facing challenges finding mentors (community reception issues), being boxed into specific roles (stereotyping), sharing time between work and family (work-life balance issues), and biases or lower acceptance rates of contributions when they explicitly identify themselves as women (gender-biased peer review).

5. What strategies were proposed to support women’s participation in OSS projects?

Recommendations of strategies to improve diversity are scattered and rarely widely adopted. OSS communities need a concise view of the different types of actions to select the ones that are viable and appropriate for their needs and for the challenges they face. Strategies include actionable mechanisms that OSS communities can take and combine to create a more inclusive environment for women.

We summarize the proposed strategies in Fig. 11 and Table 6. Next, we present and explain each of them. All strategies had been mentioned as a way to mitigate at least one challenge presented in Sect. 3.4. However, there was no strategy reported to mitigate the challenge Work-Life Balance Issues nor Gender-biased Peer Review.

Promote awareness of the presence of peers. Approximately half (54%) of the women respondents of Powell et al.’s (PS28) study said they would be more inclined to participate in OSS if there were more women involved. Promoting awareness about the rate of women can help to attract more women, minimizing the feeling of alienation (PS28). This awareness should include a measurement of women’s participation and the type of contributions (PS12; PS39). According to participants of Calvo’s (PS24) study, the communities can create parallel spaces in which the proportion of women is above 50% so as to create more diverse environments under the values of mediation and care.

Promote women-specific groups and events. The community managers interviewed by Calvo (PS24) mentioned that they promote schoolgirls’ events to inspire vocations and empower girls who may opt for an OSS career. This strategy should promote activities exclusively for women in those spaces, highlighting their presence, as women tend to be invisible in larger groups in which men are the majority (PS24). For those women already interested in OSS, promoting women-only groups, spaces, and events (PS6; PS8; PS24; PS42; PS28; PS47) fosters discussions and supports networking and empowerment (PS42). Moreover, it provides a safe space for expressing feelings and opinions (PS24) and revealing their identities (PS47). Although effective, Singh and Brandon (PS47) found that only 3% of the 350 projects they analyzed have women-specific spaces—including websites, IRC Channels, dedicated blogs, collection/list of resources, dedicated Facebook pages, and local meet-ups.

Promote inclusive language. The insights provided by Qiu et al. (PS27) include avoiding gender pronouns that assume that people are [all] one gender or one demographic. For example, using ‘guys’ is common, and it can imply that contributors are men (PS6).

De-stereotype the OSS contributor. The women interviewed in Singh’s (PS8) study recommended to “leave the stereotypes out the door.” The frustration caused by stereotypes was expressed by one of the women surveyed by Canedo et al. (PS6): “Stop treating women developers as ’women developers’ and start treating them as developers.” Powell et al. (PS28) suggests showing less discrimination and more inclusion to tone down the male-dominated atmosphere and promote participation. Calvo (PS24) and Vedres and Vasarhelyi (PS26) suggest avoiding the feminization of specific assignments, like those relating to community building tasks; OSS communities should re-classify types of work that used to be packaged in masculine-feminine stereotyped specialties.

Encourage and welcome women. Singh’s (PS8) findings show that being less judgmental and appreciating diverse teams is essential to supporting and encouraging women. Indeed, as Beach (PS41) discusses, people need to feel supported, accepted, and encouraged. This encouragement may even come from other women (PS16). Powell et al. (PS28) suggested starting by encouraging small steps as an incentive to women to submit bug reports and share their input, increasing their self-confidence (PS25), which was echoed by two participants of Canedo et al.’s (PS6) study, “the solution is to build confidence” and “not to fear when contributing.” Encouragement is also the goal of some initiatives presented by Parker’s (PS25) FLOSSpolswww.flosspols.org, which offers recommendations on how to solve the gender gap; these initiatives include WOWEM, a gender equity and OSS research and education project; and LinuxChix, a community for supporting women in Linux. There is no value in encouraging women to be there if the environment is hostile. To welcome women, one of the community managers who participated in Barcomb et al.’s (PS39) study recommends making the community friendlier in general.

Promote women to leadership roles (empowerment). A way to empower women is to have them in senior roles (PS6), project governance (PS38), and where appropriate (PS8), as mentioned by one woman participant of Prana et al.’s (PS16) study: “More women reviewers. More women are acting directly on the governance of large OSS projects.” Some community managers indicated that their communities created decision-making positions and ensured that women led public activities (PS24). Catolino et al. (PS36) suggested that a way to avoid the proliferation of community smells is to involve women in positions where they can mediate discussions and improve the communication of sub-communities. According to Singh (PS8), promoting women to positions of authority shows the project respects their contributions.

De-bias tools. Most (73%) of the barriers that affect software professionals demonstrate some form of gender bias (PS29). Indeed, bias in tools and infrastructure can hinder women newcomers from joining OSS (PS29). One way to de-bias infrastructure and tools is by applying the GenderMag technique. GenderMag uses personas and a specialized Cognitive Walkthrough (CW) to systematically evaluate software and make them more inclusive of women’s cognitive styles (PS29; PS30; PS31). This technique’s precision was proven by a study that showed that the GenderMag technique helped to identify 81% of the issues (Burnett et al., 2016).

Recognize women’s achievement (visibility). When recognizing women’s achievements, the community provides the social attraction that women seek to overcome their competence-confidence gap (PS32). Communities can show recognition by increasing the visibility of women (PS16), listing them as great contributors whenever they deserve it (PS42), and publicly celebrating their achievements in blogs, project homepages, and social media (PS3; PS16). Another way to increase visibility is to organize events where speakers are women (PS24; PS47). These simple actions inspire more women to participate (PS16; PS24).

Prepare Mentors to Guide Women. As mentioned before, OSS projects are usually men-dominated, which may deter women (PS22; Turkle, 2005). Mentorship can help women newcomers find the assistance and support they need (PS23; PS24; PS39). One way to do so is to have women mentor other women (PS8; PS42). Singh (PS8) highlights that when mentoring women it is necessary to guide them on different aspects. While men need to change their behavior and projects need to implement systemic changes, Singh (PS8) posits that women also need to be trained to ignore disruptions and not be easily bothered by criticism or insults; the mentor needs to be extra supportive, friendly, respectful, and encouraging (PS8).

Create and enforce a Code of Conduct. Developing a code of conduct for the community (PS3; PS8; PS16; PS24; PS27; PS33; PS37; PS47) helps to mitigate Tightrope effects The term Tightrope is usually associated with the circus, where a performer walks on a stretched rope. Following the analogy, the term can refer to the narrow band of socially acceptable behavior for someone, in our case, women (PS3). by assisting communities in articulating acceptable behaviors for all members (PS8). The code of conduct comprises the collective norms of a community, as mantras that shape the culture of collaboration (PS37) as well as the community’s expectations and values to create a friendly and inclusive community (PS8; PS33). While having a code of conduct will not prevent sexism, it indicates to any men who display sexist behaviors that such actions will not be tolerated in the project (PS21). However, according to Robson’s study (PS34), just creating a code of conduct will not increase women’s participation. The author showed that projects that introduced a code of conduct in their history saw women’s participation increase almost at the same rate of projects without a code of conduct. Projects increased from 2.37% to 3.81% after a code’s introduction. Projects without codes of conduct, comparing gender diversity within similar periods yields, had an increase from 4.10% to 5.53%. The average increases were 1.44%, and 1.43%, respectively, which means creating the code of conduct did not help increase women’s participation. Robson (PS34) posits that the code of conduct needs to be enforced among the project members. Communities should put mechanisms in place to implement the code and show that violations have consequences (PS8; PS33).

The strategies that OSS communities can employ to increase women’s participation include providing awareness and statistics about women contributors, inclusive language, women-specific groups and events, de-stereotyping the OSS contributor, encouraging and welcoming women, having women in leadership, de-biasing tools, recognizing women’s achievement, preparing mentors to guide women, and creating and enforcing a code of conduct.

Discussion

In this section, we discuss our results, strengths of evidences of the reported strategies to increase women’s participation, gaps in literature, and opportunities for future research.

The leaky OSS pipeline. Section 3.1 reports that there is a large gender disparity in OSS contributors (Kofink (PS10) found less than 5% of pull-requests authors were women). However, the gender disparity is less pronounced in the initial stages (e.g., as students of Google Summer of Code). There is attrition of women contributors as they move through the different stages of the “joining script” (Von Krogh et al., 2003)—where people start outside the project as readers and passive users; then move to the project periphery as bug fixers, bug reporters, peripheral developers, and active developers (developers without commit rights); and finally enter the project as core members or project leaders (Nakakoji et al., 2002)—the leaky pipe phenomenon. This attrition can be a consequence of the several socio-cultural challenges faced by women during the process. As we presented in Section 3.4, women face gender bias in communication, acceptance in the community, and lower contribution acceptance rates when they explicitly identify themselves as women. While mentorship events enhance (women) participants’ sense of competence and increase the chances of future contributions’ values (Silva et al., 2020a), these programs alone are insufficient as women do not stay long enough to become project leaders. The majority of the challenges that women face or the reasons for women leaving OSS (see Section 3.4) are socio-cultural in nature and not related to technical skills. Indeed, as we presented in Section 3.4, women have similar or even higher rates of merge acceptance than men. Besides the appropriate technical skills, Bosu and Sultana’s study (PS9) suggested that women are as productive as their colleagues who are men in an inclusive OSS project, with some women developers showing more productivity than the average men developers. Therefore, strategies that help create an inclusive environment geared towards retaining and mentoring women are needed to fix the leaky pipeline.

Impostor syndrome amplified by toxic culture. Impostor syndrome is a psychological concept about a pattern of behavior wherein people (even those with adequate external evidence of success) doubt their abilities and experience a persistent fear of being exposed as a fraud (Mullangi and Jagsi, 2019). As we presented in Section 3.4, the literature shows that when women join OSS, the rate of acceptance of their contributions is similar to—if not higher than—men’s. The Impostor Syndrome in OSS can be amplified by the hostile and Toxic Culture that pervades communities, which are two of the challenges reported by primary studies presented in Section 3.4. The impostor syndrome disproportionately affects women and other minority groups, who often lack sufficient role models of success (Mullangi and Jagsi, 2019). Combining actions that include training for allies, plus both reactive and proactive mechanisms to enforce the code of conduct among the project members (PS34) can help mitigate the toxic culture, as we present in Section 4.3.

Impact of gender stereotyping. Research shows conflicting findings about the effects of diversity on team performance. Although low diversity can enhance mutual trust and effectiveness, the demographic similarity may also lead to stereotyping, cliquishness, and conflict (de Gilder and Wilke, 1994; Molleman and Slomp, 2006). Regardless of the cause, according to a survey of 5,500 GitHub users (Zlotnick, 2017), women more often than men encounter language or content that makes them feel stereotyped. Stereotypes manifest the common expectations about members of certain social groups. Both descriptive (how women are) and prescriptive (how women should be) gender stereotypes and the expectations they produce can compromise a woman’s career progress (Heilman, 2001, 2012). Even before starting a career, stereotype threats represent one of the significant barriers to underrepresented groups engaging in Computer Science education. Implicit stereotypes about gender and STEM have profound effects on girls’ and women’s interest, confidence, and persistence in STEM education and careers (Dasgupta and Stout, 2014; Dasgupta, 2011).

A fear of gender stereotyping can lead women to hide their gender (PS1) and create pseudonyms to avoid judgment (PS21). This behavior was also observed by Ford et al. (Ford et al., 2019) in online communities, where participants use a “gender neutral alias for websites like technical communities, because [they] get better help when asking questions or answering them.” One of the strategies presented in Section 3.5 is De-stereotyping the OSS contributor, which differs from using a neutral username to hide gender. In fact, Canedo et al.’s (PS6) study showed that users who do not reveal their gender suffer an even more severe disadvantage in survival probability. Although it prevents discrimination by categorical gender, avoiding gender identity can also lead to a lack of trust and exclusion from projects and ultimately cause a higher exit rate of such users. Even when stereotyping is minimal, it can still make a difference. People’s attitudes, beliefs, and behavior are often shaped by factors that lie outside their awareness (Banaji and Dasgupta, 1998; Greenwald and Banaji, 1995). Considering that even minimal social cues may activate negative stereotypes early in informational processing (Wu et al., 2020), De-stereotyping the OSS contributor is crucial for women to start seeing themselves playing the role of developer and not just men. This strategy is aligned with the suggestion of women interviewed in the Blincoe et al.’s study (Blincoe et al., 2019), who considered that changing the typical image of software engineers as IT geek men is a way to reduce the gender gap.

2. Considering women contributor’s motivations

Motivation to join vs. project culture. Since women have social motivations (e.g. Kinship) as we presented in Section 3.2, and the reported challenges are also social, as we presented in Section 3.4, there is a conflict between their expectations and reality, which can explain why women are not joining or staying in OSS projects. When women join an OSS project expecting to find other women (PS50) and friendly colleagues (PS16), but instead find Lack of Peer Parity and face a Toxic Culture, this directly conflicts with their motivations. One place where toxicity can manifest in communications is via comments in code reviews and in mailing lists. Code reviewers may need education support to articulate their review comments in a way that builds relationships (Bosu et al., 2016). One strategy could be providing review templates that help developers in using inclusive words and employing empathy. The lack of peer parity can be alleviated by attracting more women, which can be accomplished by recognizing different types of contributions (PS11), for example recognizing contributors who participate by answering questions and discussing issues (Trinkenreich et al., 2021; Ducheneaut, 2005). Moreover, communities can foster peer communication through women-only (and their ally) groups and events, such as R-Ladies and other safe spaces (PS6; PS42; PS48).

OSS as a career pathway. As Section 3.2 presents, only 4.07% of the 226 surveyed women from a FLOSS 2013 study joined to increase their job opportunities. After becoming contributors, this motivation increased almost six times (going to 25.79%) (PS5). We argue that this represents the “shifting belief” that women have in OSS toward building a career, which increases only after overcoming the barriers to join and become contributors. The multiple roles presented in Section 3.3 are both related to the technical (project-centric) and non-technical (community-centric) side of the projects. Therefore, an awareness of the different roles and career pathways that exist in OSS can attract women with diverse backgrounds and expertise to OSS by showing them the multitude of trajectories to success (PS11). Programs like Google Summer of Code, as well as other OSS-academic liaisons, can improve awareness of career opportunities provided by OSS. Further, given that 54% of the women who contribute to OSS devote less than 5 hours per week, this means that the majority do not making a living from OSS. The fact that work-life balance issues are a challenge that women face (see Section 3.4) and being paid to contribute is a relevant motivation (see Section 3.2), OSS projects could offer part-time jobs to attract women who are not yet participating.

3. Choosing appropriate strategies to mitigate challenges

The literature has suggested several strategies to mitigate challenges faced by women and increase their participation in OSS (Section 3.5). However, these strategies have not yet been evaluated in the field. Therefore, to help communities in selecting or prioritizing strategies to plan interventions, we analyzed the strengths of evidence of the publications that reported them. To grade the strength of evidence, we followed a process similar to Dybå and Dingsøyr (2008b), using the GRADE (Grading Recommendations Assessment, Development and Evaluation) working group definitions (Group, 2004), which grades the overall strength of the evidence as high, moderate, low, or very low. The GRADE approach is widely used by secondary studies in software engineering to evaluate the strength of evidence found in the primary studies (Dybå and Dingsøyr, 2008a; Alves et al., 2010; Alexandre et al., 2014; Silva et al., 2015; Guinea et al., 2016; Gurbuz and Tekinerdogan, 2018).

GRADE suggests categorizing the strength of evidence of the studies by combining four key elements: study design, study quality, consistency, and directness. Randomized experiments are evaluated at the top of the hierarchy (high) and evidence from observational studies and expert opinion are at the bottom of the hierarchy (low). In addition, the initial overall grade can be increased by the evidence base or decreased depending on the seriousness of the study’s limitations. The strength of evidence is defined by GRADE as the extent to which one can be confident that an estimate of effect or association is correct. While inconsistent, imprecise, sparse, or biased reported evidence may decrease the grade of an experiment, strong evidence of association from two or more high-quality studies may increase the grade of an observational study (Group, 2004; Dybå and Dingsøyr, 2008b).

Study Design: All the primary studies were case studies that reported strategies recommended by participants and none evaluated interventions through controlled experiments. Thus, as per GRADE, the evidence from these observational studies is classified as “low.” Consequently, our initial categorization of the total evidence based on (primary) study design is also low.

Consistency relates to the similarity of estimates of effect across studies (Group, 2004). Since the outcomes of primary studies were not presented in a comparable way and reporting protocols vary according to the study, evaluating the synthesis of quantitative results could be misleading. However, having several studies grouped into each strategy (see Fig.11 reflects positively on the consistency of the studies and we conclude that, in general, strategies are consistent.

Regarding Directness—or the extent to which the people, interventions, and outcome measures are similar to those of interest (Group, 2004)—we found that most studies had another goal besides presenting the strategy(ies). However, little effort has been made to present evidence about implementing a particular strategy to increase women’s participation in OSS projects. Most of the strategies were suggested as part of interviews or discussed by authors based on other findings. The only three strategies that formed part of the main goal of the papers were: use of the GenderMag technique ((PS29; PS30; PS31), enforcing a code of conduct ((PS34)), and providing women-only spaces (PS8). From those three, only Robson’s study (PS34) evaluated women’s participation with and without the reported strategy. However, this study showed that many confounding factors may have influenced the results. Our judgment was thus that there are major uncertainties about the directness of the included studies.

Finally, although the overall quality of the studies was high, with well-described methods and data collection, the threats to validity were presented but not always addressed. Therefore, in the absence of any evaluation or superficial assessment of the effectiveness of the proposed strategies, the quality of the results (effectiveness of the strategy) is graded as very low. Combining the four components of study design, consistency, directness, and study quality, we found that the strength of the evidence about the strategies to increase women’s participation is low and uncertain. Therefore, while many primary studies have shown the different challenges that women face, there remains a gap in research about the success of the proposed strategies to mitigate these challenges.

3.2. Combining synergistic strategies.

One option for communities looking to improve diversity is to combine synergistic strategies. This might be especially useful since the literature has identified many different strategies, which have low strength of evidence. OSS communities can start by implementing simple, but structured actions combining ideas from more than one strategy. For example, by publishing success stories of women in the media, OSS communities can promote awareness of presence of peers to attract more women and also recognize women’s achievement (visibility) to retain women who are already contributors. Considering that this media exposure can include women’s posts and pictures, this action also helps with the strategy of de-stereotype the OSS contributor, which has been associated with images of men in technical textbooks (Makarova and Herzog, 2015; Lee, 2018) and search results (Kay et al., 2015). Another action that can use more than one strategy is to create a women-only forum, which is part of the strategy to promote women-specific groups and events. When moderating and analyzing the messages from women to implement feasible changes to problems that are being actively discussed, this action also acts to encourage and be welcoming to women by offering mentorship or inviting women to contribute to specific activities. Another action that can be be adopted toward multiple strategies is to create and enforce a code of conduct by providing online training on enforcement and being transparent about the punishments for those who violate the code of conduct. There can be a training for contributors in general, another for mentors to (prepare mentors to guide women), and a third for allies to advocate for women and act as “collaborators, accomplices, and co-conspirators” (Melaku et al., 2020). The content of the training can include practical examples of acceptable and non-acceptable behaviors. Communities can use mining tools to identifying gender pronouns in messages of mailing lists, pull-requests, and code reviews and help to promote inclusive language.

4. Research Opportunities

This subsection discusses the gaps in the literature that may be explored in future research.

Evaluation of recommended strategies to increase women’s participation: Although several strategies to increase women’s participation have been proposed in the literature (PS3; PS6; PS8; PS9; PS12; PS16; PS23; PS24; PS25; PS26; PS27; PS28; PS29; PS30; PS31; PS32; PS33; PS34; PS35; PS36; PS37; PS38; PS39; PS40; PS41; PS42; PS47; PS48), few works present scientific evidence about their effectiveness. For instance, Tourani et al. (PS33), Imtiaz et al. (PS3), and Vandana and Brandon (PS47) relegate the evaluation of the effectiveness of the “code of conduct” to future research, this is despite the fact that it is one of the most cited strategies to promote women participation. Izquierdo et al. (PS12) discuss the difficulty of evaluating the effectiveness of strategies, as communities need to have consistent measurements before (baseline), during, and after their implementation. The authors reported that although OpenStack created the Women of OpenStack Working Group (which included educational sessions, professional networking, mentorship, social inclusion, and enhanced resource access), the OpenStack Foundation lacked baseline information about the involvement of women.

Theories to explain why women leave or avoid OSS projects: The literature also reports a diverse set of challenges faced by women (PS13; PS18; PS20; PS21; PS22; PS23; PS24; PS25; PS26; PS27; PS28), but few make a theoretical connection to why women leave (or avoid) OSS projects. Theoretical understandings can help create more effective, longer term solutions. Some studies have analyzed motivation to participate in OSS projects (Lakhani and Wolf, 2003; Ghosh et al., 2002; Hars and Ou, 2004; Hertel et al., 2003), but only a few report women’s motivation, and none go deep in this analysis. Moreover, the literature lacks research exploring why women leave OSS, their motivation to avoid participating in OSS, and why a large portion of women who study STEM do not join OSS projects. One of the few studies that have used theory to explain these phenomena is Qui et al. (PS46); they found that social capital can support the long-term engagement for both men and women in OSS projects, and that when team members have more diverse programming language backgrounds, women are less likely to leave the project early.

Applying the method to study other minorities (in different domains): The lack of diversity affects different OSS and other domains in STEM. While in this study we focus on women in OSS, researchers can leverage this study’s structure to investigate how the literature is positioned regarding the participation of other minority populations in OSS or even different domains.

5. Implications for Practice

Encouragement for women newcomers: Women newcomers can become aware of the different types of contributions made by other women through our study. Although not restricted to the activities presented here, women can be inspired by other women’s success and motivations to participate, gain awareness of the challenges reported by other women so they can be prepared to face similar ones, and prioritize participation on projects that follow one or more reported strategies.

Opportunities for OSS communities: OSS communities that seek to foster gender diversity can combine and implement the strategies to increase women’s participation, as we presented in the Discussion section. Moreover, being aware of the characteristics of women contributors, the communities can target their efforts on them by, for example, promoting events at universities (as most of the contributors are at least undergraduate students).

Opportunities for education: Being aware of the challenges that women face, educators can address the underlying issues causing these challenges in the classroom to improve students’ (all gender) awareness of biases and discuss possible mitigation actions. This research can also inform educators who adopt contributions to OSS projects as a medium to teach software engineering (Pinto et al., 2017).

Related Studies

In this section, we present secondary studies about challenges and strategies to increase women’s participation in STEM and in software engineering in particular.

Women’s participation in other cultures and domains. Analogous to OSS projects, social and cultural context also influences women’s barriers in the medical profession and their ability to rise to leadership positions (Ramakrishnan et al., 2014). Women have played a significant and active role in many contemporary armed rebellions (where men are often presumed to be the default gender), and even are frequently involved in leadership roles (Henshaw, 2016). By analyzing the career trajectories of women executives across a variety of sectors, Glass and Cook (Glass and Cook, 2016) concluded that while attaining promotion to leadership is not easy, serving in a high position can be even more challenging. Although women can be more likely than men to be empowered to high-risk leadership positions, they often lack the support or authority to accomplish their strategic goals. As a result, women leaders often experience shorter tenures compared to men peers (Glass and Cook, 2016). Similar to software development teams, where women are instrumental to reducing community smells (Catolino et al., 2019), in international relations, the collaboration between women delegates and women civil society groups positively impacts and brings more durable peace when negotiating peace agreements (Krause et al., 2018). The challenge of Work-Life Balance that we presented in Section 3.4 is a general challenge faced by women who aim to work in Japan, where the low numbers of women in medicine reflect the prevalent societal belief that careers and motherhood do not mix (Ramakrishnan et al., 2014). In contrast, Scandinavia has similar numbers of men and women physicians, which has coincided with the emergence of progressive work–life policies, the belief that women can combine motherhood and employment, and changing expectations of work-life balance. Historically, Sweden was the first country to establish paid parental leave for fathers in 1974, and its National Labor Market Board has developed statements since 1977 encouraging men to contribute to childcare responsibilities (Haas and Hwang, 2009).

Challenges faced by women and strategies to increase women’s participation in STEM. The literature shows that the gender gap is largely present in the science, technology, engineering, and mathematics (STEM) fields at all education levels and in the labor market (Fatourou et al., 2019). The barriers faced by women in STEM fields, presented by McCullough (McCullough, 2011), are similar to some of the challenges we presented in Section 3.4, including discrimination and implicit bias (Toxic Culture), lifestyle choices, and family obligations (Work-Life Balance Issues), and lack of role models and mentors (Community Reception Issues).

Regarding strategies to increase women’s participation, the W-STEM project (Garcia-Holgado et al., 2019) seeks to create mechanisms to attract and guide women in Latin America in STEM higher education programs, including actions to monitor gender equality in enrollment and retention (as we presented in the strategy promote awareness of the presence of peers). Another strategy proposed by both Garcia-Holgado et al. (Garcia-Holgado et al., 2019) and Moreno et al.’s (Moreno et al., 2014) study is to disseminate scientific and technological culture from an early age, promoting STEM studies vocation and choice for girls and young women in secondary schools (as we presented in the strategy Promote women-specific groups and events). Moreover, the women who participated on Moreno et al.’s (Moreno et al., 2014)’s study corroborated the strategy of promote inclusive language and the need to avoid chauvinistic attitudes, and to de-stereotype the CS student, suggesting that the general image of the student majoring in CS should change and not ascribe to the nerd stereotype, which is typically ascribed to men.

We also found secondary studies related to diversity in other domains. Kazmi (2014); Maheshwari (2021) show that women in academia and industry, despite their impressive performance, face work-life balance issues and the impostor syndrome (Kazmi, 2014), but can be supported by mentorship to overcome the challenges and advance in their career (Maheshwari, 2021).

More specifically for Computer Science, Pantic and Clarke-Midura (2019) found that individual (pre-arrival), institutional, and societal factors interplay to affect women’s commitment and retention to the Computer Science program. Felizardo et al. (2021) found that research contributions from women in secondary studies have globally increased over the years, but are still concentrated in European countries. Although their findings are similar in terms of analyzing the factors that impact women’s participation, none of them addressed specific motivations or challenges, or focused on strategies to increase women’s participation in OSS.

Other literature surveys about gender diversity in software engineering. Gender (with a focus on women) is the most explored aspect of diversity in software engineering literature (Menezes and Prikladnicki, 2018; Silveira and Prikladnicki, 2019). Spichkova et al. (Spichkova et al., 2017) analyzed the role of women within software architecture literature and found that it is understudied. In a more general literature review in Software Engineering, Rodríguez-Pérez et al. (2021) highlighted that researchers have been exploring the gender bias problems in software engineering more than presenting the solutions to mitigate these problems. Our study complements these studies as it focuses on analyzing the literature to identify strategies proposed to mitigate the challenges faced by women in OSS.

The study closest to our work is a systematic literature review (SLR) conducted by Canedo et al. (Dias Canedo et al., 2019). While their study investigated the problems causing women’s lack of engagement in software development in general, our analysis focused on the challenges faced by women in OSS projects (Section 3.4). Our study covered a larger spectrum of studies (51) as compared to Canedo et al. (24). Additionally, some of the studies used by Canedo et al. (Dias Canedo et al., 2019) did not qualify in our inclusion criteria, as these analyzed women participation in STEM in general. When comparing the results, only two challenges were common between our work and Canedo et al. (Dias Canedo et al., 2019)—Toxic Culture and Impostor Syndrome. Canedo et al. (Dias Canedo et al., 2019) also presented possible solutions to increase the engagement of women in OSS. Our work expands this list (Promote inclusive language, Encourage and be welcoming to women, de-bias tools, Recognize women’s achievement (visibility) and Prepare mentors to guide women). Finally, in our work we analyzed the motivations that drive women to contribute (Section 3.2) and further analyzed and associated the challenges faced by women in OSS (Section 3.4) with the strategies to mitigate those challenges (Section 3.5) as proposed in the primary studies (Fig. 12).

Considering that women are still underrepresented in OSS and projects seek more diversity to increase productivity and enhance the quality of delivered software, it is crucial to understand the profile of women who contribute to OSS, including their motivations and challenges, as well as strategies to attract and retain them. Therefore, we believe that the consolidated relationships presented in this literature review are necessary and could inform both practitioners and researchers.

Threats to Validity

We used the checklist provided by Ampatzogloua et al. (Ampatzoglou et al., 2019) to identify and categorize threats to validity and corresponding mitigation actions.

Study inclusion/exclusion bias. There may be relevant papers in the population that are not in our sampling frame, e.g., studies that covered women’s participation in OSS published in outlets that are not indexed by the target search engines or excluded by the filters we used (see Section 2.1.1). Also, there may be relevant articles in the sampling frame that are not in the sample; for example, there may be papers covering women’s participation in OSS never use the terms we included in the search string. We took several steps to mitigate these threats, including using additional terms besides “women” in the search string (e.g., “diversity” and “heterogeneous team”), backward and forward snowballing, and having the list of primary studies reviewed by the most prolific authors. Although in this paper we use the term “women” and “men” as a shorthand for people who self-identify as such, including all dimensions of gender, such as transgender and non-binary identifications, there can be dissimilar participation of those from the different dimensions of women that were not reported by the primary studies.

Researcher bias and Repeatability. We used an open research question (and search string) to find all forms of participation by women in OSS. This openness can bring subjectivity to the data extraction and repeatability. While the first author of this study led the execution of steps to filter and select the primary studies (see Fig 2, the other two researchers with extensive experience in conducting qualitative analysis and literature studies participated in defining the protocol with phases and steps (see Fig 1), inclusion and exclusion criteria, and discussed the data in weekly meetings during the recording process. Additionally, one senior researcher cross-checked a portion of the extracted data. During the weekly meetings, we discussed and adjusted codes and categories until we reached agreement. In the meetings, we also checked the consistency of our interpretations.

Robustness of classification. We used existing classification schema from the literature to explain two categories of participation: the types of contributions that women make in OSS projects (see Section 3.3) and the motivations that drive women to participate in OSS projects (see Section 3.2). The selection of those two initial classification schema poses a threat, since they might not be enough to explain the phenomena. At the same time that we had these two schemas to guide our analysis, we constantly assessed the possibility and need to extend them. Therefore, although we did not find any new category of motivation to extend Von Krogh et al.’s (Von Krogh et al., 2012) framework of OSS motivations, we extended the classification used for types of contributions with new roles that arose from the primary studies.

Conclusion

In this paper, we investigated women’s participation in OSS projects in terms of frequency and demographics, their motivations to participate, the types of contributions women make, the challenges they face, and the proposed strategies to mitigate those challenges.

Our literature mapping has shown severe gender disparity in OSS, with women’s representation being at about 10%. Further, the women who participate are generally volunteers who can devote less than one workday a week to OSS. Although present in community-centric roles, women are less likely to be authors of pull requests and core developers, with many making non-code contributions. Gender biases exist in OSS in several places. For example, when submitting a pull request, women have high rates of merge acceptance, but lower when they explicitly identify themselves as women. Women also face social challenges such as a lack of peer parity, non-inclusive communication, a toxic culture, impostor syndrome, community reception issues, stereotyping, work-life balance issues, and bias against gender-biased peer review. OSS communities that seek to increase women’s participation can mitigate these challenges by providing awareness about the presence of other women, promoting inclusive language, organizing women-specific groups and events, de-stereotyping the OSS contributor, encouraging and welcoming women, placing women in leadership, adopting de-biasing tools, recognizing women’s achievement, preparing mentors to guide women, and creating and enforcing a code of conduct.

As future steps, we plan an in-depth analysis of women’s motivations, challenges, and reasons that would make them leave. We will propose strategies to attract and retain more women to OSS communities, as a long-term plan.

Acknowledgements

This work is partially supported by the National Science Foundation under Grant Numbers 1815486, 1815503, 1900903, and 1901031.

References