Using rubrics for program assessment: Example
A history program wants to use direct evidence of student learning as part of their assessment for the learning outcome, “Students will employ evidence to craft arguments about the factors that led to significant social change.” Below is a description of the steps followed by the history department’s assessment committee.
1. It was determined from the curriculum map that students are asked to craft arguments about factors leading to social change in courses taken during the junior and senior years. It was decided that second semester senior work should demonstrate accomplishment of the outcome. For this reason, the capstone course was selected as a place to look for work products.
2. Within the capstone course there were two assignments that involved demonstration of the outcome: a debate (between two students) and a term paper. The assessment committee selected the term paper for evaluation.
3. Two members of the assessment committee worked with a faculty member who teaches the capstone course to develop a rubric for the learning outcome. This group decided that the learning outcome contains four component skills. The group worked together to develop five categories of performance descriptors for each component skill.
4. During the spring semester that the history department agreed to collect the term papers there were 62 students enrolled in three sections of the capstone course. Since this was a large graduating class, the assessment committee decided that they would apply the rubric to a sample of the papers instead of the entire set. Half of the papers were randomly selected to be a part of the sample.
5. The assessment committee asked two members of the faculty to apply the rubric. Both faculty members were trained in using the rubric before they applied it to the term papers.
6. After the faculty members returned their completed rubrics, the assessment committee averaged the scores given by the two faculty raters for each student on each of the four component skills on the rubric. Next, a mean score was calculated for each of the four components.
7. At a faculty meeting the assessment committee presented the four component means in a bar graph. Three of the four component means were above the criterion mean score of 3.75 (on a 5 point scale). The fourth component, using diverse perspectives of participants in the historical period, fell below the criterion with a mean score of 2.99. The faculty agreed that this was an area that they could work to improve student learning.
8. At the faculty meeting where the results were presented the faculty brainstormed ways to improve student learning for the component skill in question. The assessment committee then met to consider the ideas and come up with a suggested plan. The plan was presented at the next faculty meeting, where it was discussed and voted on.