2000-2001 Evaluation Report: Executive Summary

MentorNet (www.MentorNet.net), the E-mentoring Network for Women in Engineering and Science, is a nonprofit [501(c)(3)] organization working to further women’s progress in scientific and technical fields through the use of a technology-supported mentoring program. MentorNet aims to advance women in science, and enhance engineering and related sciences, by promoting a diversified, expanded and talented workforce. In partnership with colleges and universities, corporations, professional societies, and government labs and agencies, MentorNet is international in scope, serving students all over the world. MentorNet has been in existence since 1997.

In 2000-01, MentorNet paired 2000 students with 1913 professionals who volunteered to serve as mentors – a number of the professionals were paired with two students. These 2000 e-mentoring pairs represented students from 70 participating campuses and mentors from 690 corporations, government labs and agencies, and other employers. All participants were asked to complete the year-end questionnaire.

This evaluation report analyzes data collected about the 2000-01 MentorNet program year. The evaluation program was administered and conducted by MentorNet staff in conjunction with evaluation consultants. The data were collected using a web-based questionnaire presented to the participants at the end of an academic year-long program. The analyses were conducted in order to assess the participants’ satisfaction with the program and the value of participation. In addition, analyses were conducted to inform program development and improvements.

Of participants who could be reached via email by the end of the year, 43% of the students and 55% of the mentors answered the year-end survey. The sample of students and mentors who responded to the survey was as follows:

Students:

66% were undergraduates
78% were engineering majors
55% were Caucasian or of Western European descent
10% self-identified as part of an underrepresented minority group (African-American, Hispanic/Latino-a, American Indian)

Mentors:

71% were women
58% were employed at an organization that was sponsoring MentorNet
40% had earned bachelor’s degrees
31% had earned Master’s degrees

Both students and mentors responded positively to a set of questions that addressed participants’ satisfaction and ease of communication with the program. On a five-point scale, with 5 being the most positive response, the average student ratings on all of the following items were greater than 3.75 and the average mentor ratings on all of the following items were greater than 3.25:

Their satisfaction with their one-on-one e-mentoring experience
Their satisfaction with the MentorNet program
The quality of the “match” between them and their e-mentoring partners
The comfort with which they felt they could ask questions of their e-mentoring partners
The comfort with which they could respond to questions from their e-mentoring partners

Using factor analysis, we combined these five items with one other item and developed two constructed variables, one from the student data and one from the mentor data. We used these two constructed variables to assess overall satisfaction with MentorNet. These constructed variables were used in the analysis, which determined the individual, match, and program characteristics associated with higher overall satisfaction.

When asked whether they would recommend participation in MentorNet, students and mentors responded as follows:

92% of the students would recommend MentorNet to a friend.
94% of the mentors would recommend to a colleague.

Students’ assessment of the value they received from participation in MentorNet was also positive. Fifty-percent or more of the students reported that the MentorNet experience had positively affected the following:

Their self-confidence about their ability to succeed in their field (50% responded affirmatively)
Their belief that they would enjoy working in industry or a government lab or agency (52%)
Their desire to pursue a job in their field (52%)

The majority of mentors and students exchanged two to four email messages per month. In addition, on average, the mentors spent 18 minutes per week participating in MentorNet and the students spent 19 minutes per week participating in MentorNet.

The students’ and mentors’ open-ended responses also provided insight into the value associated with participation. Students most frequently stated that having the opportunity to learn about the industry or their field was the most valuable aspect of participation. Mentors reported that the most valuable aspect of their MentorNet experience was providing help to someone else, most often regarding the students’ career plans or other aspects of the students’ experiences.

We also conducted analyses to determine which groups of participants, and which program variables, were related to positive outcomes with the program. We found that:

Students’ educational level was related to satisfaction, with graduate students and their mentors having the most favorable experiences.
Mentors’ gender was related to positive experiences, with students paired with female mentors reporting a more positive experience.
Match Total Quality was correlated with outcomes, with e-mentoring pairs that had a closer match reporting a more positive experience.
Match round was related to positive experiences, with pairs that were matched earlier in the program reporting a more positive experience.
Gender was related to mentors’ recommendations of the program, with the proportion of male mentors who would recommend participation in MentorNet to a colleague higher than the proportion of female mentors.

In general, mentors reported higher overall ratings for the program when they were in e-mentoring pairs with closer matches (based on Match Total Quality, an objective measure assigned to the e-mentoring pairs at the beginning of the program) and paired in matches that were launched earlier in the program. Specifically, these mentors had better overall satisfaction ratings and were more likely to recommend MentorNet to a colleague. Of note is that the closer matches were associated with the earlier match rounds. The higher participant ratings for the earlier matches could be due to the closer matches, to having a longer time to cultivate the e-mentoring relationships before the students’ winter break, or to having less lag time between application and matching. Then again, a combination of these three factors could account for the association.

While the results of this evaluation addressed many important questions, it also identified new avenues for investigation, which will be addressed in future evaluations of the program. We defined involvement to mean whether or not the e-mentoring pairs stayed in contact for the full program year. Based on the response rates and questionnaire results, we calculated involvement to range from 46% to 94% for all the participants who were matched at the beginning of the year. We suggest that future evaluations include a response bias follow-up study that will help to narrow this estimate.

MentorNet provided an extremely positive service to the majority of the respondents to the survey. Respondents rated their experiences highly and as having a positive impact on their lives. An overwhelming majority would recommend MentorNet to a friend or colleague. As MentorNet continues to grow, it will be able to offer this valuable experience to increasing numbers of students. In addition, MentorNet may need to consider ways of ensuring that the program intervention, the targeted populations, and the desired outcomes are all aligned. Issues around involvement, matching, and match timing will continue to be important factors as the program goes forward.