SET Committee Report: May 2008
Word Doc
Report to ALC from Committee on Student Evaluation of Teaching
Prepared by Jackie Dewar and Margaret Kasimatis
Context: After holding two workshops in the Center for Teaching Excellence (CTE) in Fall 2006 titled “How Can/Do We Evaluate Teaching at LMU” and conducting a follow-up survey, a report from CTE to Faculty Senate and the CAO recommended a committee be appointed to review/revise the student teaching evaluation form. This recommendation was reinforced in discussions at the 2007 Academic Leadership Workshop, in which participants expressed strong dissatisfaction with the current teaching evaluation instrument. A committee was convened in late Summer 2007 and began meeting every two weeks throughout the 2007-8 academic year.
Committee Members (Spring 2008):
Richard ‘Sonny’ Espinoza, Paul Humphreys (replaced Michael Miranda due to sabbatical), Linda Leon, Jennifer Pate-Offenberg, James Roe (Chair) with Jackie Dewar and Margaret Kasimatis serving as resource persons to the committee
Actions:
- Developed and distributed guidelines for interpreting student evaluation form data to deans, chairs and the Committee on Rank and Tenure
- Developed a pilot form that attempts to take into account best teaching practice, but remains pedagogically neutral
- Pilot tested the form in Fall 2007 in 14 sections of courses, both grad and undergrad and representing all schools and colleges (Total number of students = 278)
- Ran an initial statistical analysis comparing particular questions on the old form with certain questions on the new form
.
- Completed the analysis of responses to an open-ended question at the end of the pilot form asking students to comment on the pilot form as compared to the standard form
- Held two focus groups with students to examine and compare the forms but they did not actually use them (one with 8 grad students; one with 8 undergrad students)
- Recruited additional pilot testers for Spring 2008 with the intent to increase the number of data points to at least 30 so as to be able to do additional testing (e.g., internal consistency)
Outcomes:
- Initial statistical analysis shows that certain questions on the pilot form revised to be pedagogy-neutral still capture key instructional dimensions that were on the old form (such as organization and clarity)
- The graduate focus group indicated a preference for the new form
- The undergraduate focus group was more mixed
- The initial qualitative analysis of the 278 forms indicates overwhelming preference for the new form by the actual users of the form
Next Steps:
- Analyze data from the Spring pilot
- Survey faculty who were in the pilots regarding their satisfaction with the form
- Complete the data analysis and report and hold faculty forum(s) in early Fall 2008
- Distribute guidelines for interpreting student evaluation form data to all faculty
- Develop a pool of optional questions that are directed to particular pedagogies (e.g., lecture, performance class, etc.) for use by departments or colleges
- In 2009 make final recommendation to Faculty Senate for action