What research design will contribute new insights?

Karin Grasenick | February 2019

A well-elaborated methodology ensures that the diversity aspects you are interested in are adequately investigated and interpretable data are collected.

In your research questions and hypotheses, pay attention to include the sex(es) or other relevant diversity aspects of the population under investigation. Avoid terms that overgeneralize (e.g., patients, subjects, citizens, consumers) and rather use specific terms (e.g., female subjects). (Nieuwenhoven & Klinge 2010, p.318, European Commission 2011)

Niewenhoven and Klinge (2010, p. 319) provide further questions to operationalize and include sex and gender: “Sex can be included in a study in many different ways, and the choice will have a great impact on the analysis.

• Is it merely a prognostic factor or also an effect modifier?
• Does it need to be controlled for, or will this obscure interesting findings?
• Does a certain research question or design call for sex-disaggregated models, or are dummy variables better suited?

1. Gender is less easy to reduce to variables that can be included in a statistical analysis, but its explanatory power can be enormous.”

For instance, sex hormone levels might influence brain activation during task performance in psychological tasks. When comparing women and men in this context, phases of the menstrual cycle and the associated hormone levels should be considered. Furthermore, education, practice, or skill level might be influencing factors as well that should be controlled for (Jäncke 2018).

Guidelines for performing and interpreting rigorous sex and gender subgroup analyses are discussed by Aulakh and Anand (2007) (in the context of randomized clinical trials). For all kinds of subgroup analyses, it is important that the statistical power is large enough.

Determining an adequate sample size can be a challenge. Quantitative studies often require large numbers of participants (especially when effects are small and variation is high). In qualitative in-depth studies that require more time per participant, samples are much smaller. Choosing a within-subject design, rather than a between-subject design, might be helpful in some cases to combine the different needs of quantitative and qualitative approaches. (“[…] Melding these two numerical needs is a challenge but one that reveals a deeper understanding of the reasons for variation from a mean. This is especially important to delve into if science is to tell us something real about biology in all its variation. What seemed most revealing is to use a within-subject design and compare the different measures across a single person treating each measure as a different perspective on the same question, rendering quantitative methods as qualitative.” Einstein 2012, p.11)

Nieuwenhoven & Klinge (2010, p.318/19) provide a set of questions regarding sex and gender-sensitive methods that allow gathering sex-disaggregated data (this also applies to other diversity aspects):

• Is it substantiated why women or men are included or excluded?
• Is it necessary or possible to collect sex-disaggregated data?
• Is it necessary to validate an instrument that is being developed for both sexes?
• Is the existing instrument being used validated for both sexes? If not, should it still be used for both sexes?

Taking into account participants’ perspectives and needs by testing and reviewing the study design with potential user groups can be of high value and identify blind spots. An iterative practice that includes constant, reciprocal interaction with participants and constant interrogation of your own methods can enhance qualitative studies, in particular (see e.g. Einstein 2012).
“For example, researchers who studied divisions of labour in service call centres found that most employees with direct contact to customers were women (Russell, 2008). These women typically used software based on managers’ assessments of their needs and not on direct study of their workflow. Engineers who observed how these women worked were able to redesign software in ways that ultimately boosted productivity (Maass et al., 2007).” (European Commission 2013, p.113)

Triangulation of methods is useful to cover the full range of different effects/aspects, cover different perspectives, and can help to serve as an internal control when no control group is possible. E.g., studying the neurobiological effects of female genital mutilation (FGM), Gillian Einstein combined physiological measures with qualitative interviews. This allowed her to ask her questions from multiple perspectives (Einstein 2012, p.18):

• Find out how the participant feels about the question; what is it like for them?
• How do the environment in which the question is being asked and the person asking, affect the participant’s account?
• How do standardized measurements and physiological responses relate to what the participant is saying—how does it appear?

In biomedical research, the inclusion of female mammals might be required. This is more obvious in human clinical trials. In a 2011 review, it was discovered that male bias in human studies has declined (but still exists), whereas it has increased in non-human studies in the last 50 years. In animal studies, male biases have been discovered particularly in neuroscience, pharmacology and physiology, but also behaviour and behavioural physiology; and female biases in reproduction and immunology. Studies involving humans show fewer fields with male biases (interdisciplinary biology, neuroscience, physiology, pharmacology, and behaviour) and others with female biases (reproduction, endocrinology, and behavioural physiology) (Beery & Zucker 2011, p.3)

• 1. 1. 1. If possible, include both sexes (or other relevant diversity aspects, e.g. age) in your sample. If only one sex (or one specific group regarding other diversity aspects) is studied, indicate this in article titles and explain the reasons for the exclusion (Beery & Zucker 2011)
        2. Female mammals are often considered too intrinsically variable (due to hormonal cycles), which allegedly decreases the homogeneity of study populations and makes it too costly and complex to routinely include them in research projects. However, it is exactly because of this heterogeneity why detailed study is necessary. Furthermore, meta-analyses suggest that “the long-held assumption that the estrous cycle of female mice renders them more variable than male mice require reappraisal” (Beery & Zucker 2011, p.5). Beery & Zucker (2011, p.7) further conclude: “Females can be studied irrespective of estrous or menstrual cycle state without a substantial increase in outcome variance for some traits (e.g., Mogil and Chanda, 2005; Meziane et al., 2007). Alternatively, when traits are known to vary as a function of the estrous or menstrual cycle, or one suspects sex differences, comparison of males with two or more groups of females at known estrous cycle stages is a viable and recommended option (Becker et al., 2005).” Thus, hormone variations can and should be incorporated into study design. In the Gendered Innovations Report (European Commission 2013), it is summarized as follows (p.122):
           1. Sample naturally ovulating women at different phases of the menstrual cycle (or female animals at different phases of the estrus cycle).
           2. Take into account the widespread use and effects of exogenous hormones, such as oral contraceptives, menopausal hormones and androgens.
           3. Sample women at various points of pregnancy and post-partum.
           4. Collect data on early and late peri- and post-menopausal status in studies of middle-aged women.
        3. Always specify and disclose on publications the sex of experimental animals, tissues or cell lines (“[…] every mammalian cell has a sexual signature and basic cell chemistry and organ structure may differ between females and males” Beery & Zucker 2011, p.5). In addition, for experimental animals, record their age and weight, and determine females’ reproductive status and ovarian cycle phase as accurately as possible (Genderbasic, Holdcroft 2007).
        4. In animal studies, think about the possible use of different species (vs. a focus on rats and mice) (Beery & Zucker 2011).
        5. Consider intersecting variables that might influence research results and their interpretation in selecting study samples and match your female and male groups accordingly. For example, observed statistically significant differences between men and women’s knee anatomy led to the development of a gendered knee prosthesis. However, body height (which intersects with sex, i.e. on average, women are smaller than men) is a more important factor for the selection and fitting of prostheses than sex. (European Commission 2013, p.111f)