At Desmos, we spend the majority of our day thinking about ways to support teachers. To refine our thinking and ground our decisions in the reality of classroom work, we created the Desmos Fellowship, a yearly application-based program from which we select 40 educators to join us for a weekend of conversations and mutual learning at our headquarters in San Francisco (all-expenses paid). There is no obligation beyond that, though many Fellows stay in touch.
Desmos employees are also growing in their awareness of their unconcious biases, particularly their biases towards race and sex. This post describes our efforts in reducing those biases in our Fellowship selection process so we can create a fellowship that represents the diversity of the teachers we want to support.
1. We made sure our application used inclusive language.
The recruiting company Lever wrote convincingly that checklists of qualifications on an application deter women from ever completing the application. According to a Hewlett Packard study, men feel confident enough to apply for a job if they meet only 60% of the criteria, while women don’t feel confident applying unless they meet 100%.
As an example, “You must be a clear communicator” is a criterion that risks alienating teachers who have been told repeatedly and wrongly that they don’t matter.
For this reason, we replaced the checklist of qualifications with a description of what Desmos Fellows will actually do. An “impact description,” such as “During the Fellowship Weekend, you’ll help us learn more about the advantages and challenges of using Desmos in diverse settings” gives applicants a clearer sense of what they’ll do, and ideally, reduces their anxiety about applying.
2. We measured what mattered to us and only what mattered to us.
We wanted to make sure our Fellowship cohort reflected the diversity of the people who applied, so we asked each applicant to describe their identity. Specifically, we asked about their race and gender using categories we adopted from CultureAmp.
We also looked for applicants who were more-than-casual Desmos users. While we don’t expect anything close to expertise from applicants, our program is built for people who are already conversationally fluent in our work.
We understand that every new question we ask is an opportunity for people from groups historically excluded in math, education, and technology to decide they aren’t qualified to apply. So we tailored our application questions closely to our purpose of supporting diversity.
Here is a subset of those questions and their rationale:
| Question | Rationale |
|---|---|
| Why do you want to be a Desmos Fellow? | We want to learn from people who are committed to students and their educational experiences, and we want to know the different ways in which our Fellowship might support this work. |
| What makes you or your teaching context unique? How does technology support or enhance those aspects of your teaching? | We want to learn from people who recognize the negative effect that digital technology can have on a student’s math education and who are thoughtful about those liabilities. |
| What is something you’d like to see Desmos do differently or better? | We are eager to meet people who will hold us accountable to our highest standards and who will also tell us when those standards aren’t high enough. |
3. We tracked the diversity of the cohort, monitoring for bias at each stage of evaluation.
Idil Abdulkadir, one of our Desmos Fellows, evaluated her school’s award program for bias by asking, “Who is missing? Why?” If groups of people are missing, that reflects bias in our selection process rather than any lack of capacity on their part.
For example, in previous applications, we asked how many Desmos activities a teacher had used with students, a question which was biased in favor of teachers with a class set of computers and against teachers working with fewer resources, which are exactly the kind of teachers our team wants to understand and support. For this reason, we removed the item and instead asked teachers to “submit something that helps us learn about your interest or expertise with Desmos and tell us why you selected it.” We also gave examples of what applicants might submit, such as an authored activity or graph or a video explaining an aspect of Desmos that the applicant liked.
In previous applications, we also asked how many years of experience a teacher had working with adults in professional development settings. That question, and others like it, produced a cohort that was disproportionately white compared to the pool of applicants. After internal deliberation, we realized we were essentially asking, “Have you been promoted by your employer?” which meant we were inheriting bias from every applicant’s employer. As a result, we removed that question.
To reduce the bias that people often attach to Black-sounding names, we evaluated the applications blind to names. We also double-scored the applications, which gave members of the selection committee an opportunity to examine sources of conscious and unconscious bias in conversation with each other.
We used this iterative process to monitor for bias at every stage of this year’s application process.
4. We scored holistically in our final application rounds.
We are a company that loves mathematics, but we know that the answers to the most important questions are often not numbers.
For the last two selection rounds, we set numbers aside and considered applications on a holistic basis. We used numeric, rubric-based scoring to help narrow 600 applicants to 80. We then read across entire applications rather than reading the same question from every application. This method helped us get a more complete picture of an applicant’s experiences, background, and interests.
We also took the opportunity here to reduce biases that had accumulated throughout the selection process. After a holistic reading of the 80 applications, we selected a cohort that would help us understand the diversity of the schools, teachers, and students we hope to support. In particular, we considered geography, grades taught, the socioeconomic status of their school, their racial, ethnic, and gender identity, and their experience supporting students in diverse settings.
5. We recorded our resolutions for next year.
We watched bias slip into every space we allowed it in our processes, and we made resolutions to do better in the future.
In past years, when a given round’s selection committee was majority male, it produced a disproportionately male selection. We began our process this year focused on eliminating bias, and still, when a given round’s selection committee was majority female, it produced a disproportionately female selection.
We can’t rely solely on our good intentions to reduce bias. So we are systematizing our approach to reducing bias in this selection process, in hiring, in our approach to events, and in the products we make. (We’re grateful to Desmos Fellow Nicole Hansen for alerting us to the need for systematization.)
It’s likely impossible to systematize the organic conversations that led us to examine our biases, but it is very possible to systematize the processes for reducing it. That’s this post, along with a longer list of item corrections and modifications we’ve recorded for next year’s Fellowship!
We hope this benefits you and your thinking about bias, and we’d love to learn from you as well. Find us on Twitter, learn more about the Desmos Fellowship, or check out our careers page to learn about ways to partner with us.
BTW: We found these resources extremely helpful in our thinking about reducing bias, and we hope you will too:
- Lauren Baucom’s journal of last year’s Fellowship Weekend.
- The #EducatorEquity Pledge
Next Post: Just as classroom teachers try to make their classrooms welcoming places for all students, we have been learning how to make our events more welcoming for all educators, particularly those who have been historically excluded from math education spaces. Next week, we’ll share some of our learnings and ongoing questions.