Traditional Student Evaluations of Teaching (SETs) are feedback forms returned by students at the close of a course. Institutions intend that data from these forms be used to improve the quality of teaching and as an assessment of quality of teaching for deciding faculty promotion and tenure decisions. Although it is recognized that students can offer valuable information on the appropriateness of teaching quality, it has also been recognized that these traditional SETs are likely to have negative effects on the quality of teaching. These negative criticisms are quite extensive and range from "dumbing down" of courses to restrictions on academic freedom. One patently obvious criticism is that the information given by one group of students at the end of a course cannot be used to improve the teaching on that course. Similarly, it can only be useful to future students to the extent that future groups of students are similar to the feedback group and to the extent that the course and teaching remain similar. However, courses and teaching methods hopefully evolve and the constituent subgroups of a student cohort can change considerably from one year to the next.
This paper introduces an alternative method of allowing students to assess the quality of teaching that circumvents many of the problems associated with traditional SETs. In particular it allows feedback to be used for optimizing teaching quality during the course for the whole class, for individuals or for identified subgroups of students within the whole group. The feedback is quick and cheap to process - as it requires only eight ratings from each course member.
The paper outlines the method and the theory behind it. These three objectives - kills, understanding and attitudes - are emphasized to a determined amount in the teaching and assessment of the course. Feedback forms used during the course give data on the lecturer’s and students’ expectations for change in these objectives. This data allows for calculations of the alignment between the lecture’s and the students’ expectations for change. The theory is that academic success is maximized when students and their lecture are working towards the same changes. The theory is re-validated with each course by correlations of alignments with results, which show that in-course alignment predicts postcourse academic success. This paper describes how the data are also used during the course to determine the changes that will best align in-course student/lecture expectations. The educational importance of this alignment method is that it offers a cheap, efficient and effective alternative to the widespread problematic use of traditional SETs for quality control of teaching in tertiary institutions.
Introduction
This article briefly reports an alternative system for assessing quality teaching in tertiary institutions and focuses on the student feedback part of the system. The traditional method of assessing quality of teaching has been by questionnaires that ask students to anonymously rate the quality of teaching on a 4 or 5 point Likert scale from strongly disagree to strongly agree. In the literature the use of these forms is called Student Evaluations of Teaching (SETs). SETs have been used in universities for more than thirty years to assess the quality of teaching and as an indicator of successful teaching for promotion and tenure decisions. Unfortunately, their use has been accompanied by many counter-productive effects such as discouraging innovative teaching, and deterring instructors from challenging students. Although their outcomes are intended to improve teaching, a major negative effect of also using them to varying degrees for promotion and tenure decisions has been to contribute to the lowering of academic standards. Results of analyses of SETs and expected grades suggest that instructors can "buy" better evaluations via more lenient grading.
The Alignment method
Cohen (1994) has introduced to education the term "instructional alignment", meaning the alignment of teaching, assessment and objectives. Instructional alignment has been found to compare favourable with the use of other commonly used strategies intended to improve learning performance, such as criterion-referenced tests, curriculum-based measurement and direct instruction.
Elia (1994) and Walker (1998) found that instructional alignment had an unusually high positive learning effect producing substantial improvement on the achievement levels of disadvantaged and low performing low socio-economic level school students.
It should be noted that critical thinking is expected to be promoted by teaching and assessment of professional competence. Similarly to common applications of Bloom’s taxonomy, which also emphasise different levels within domains, course objectives and content are used as vehicles for emphasizing the desired degrees of Skills, Understanding and Attitudes. What are aligned are changes expected by the lecturer and changes expected by the students in each of these three process objectives.
These forms are confidential, not anonymous. When students enrol, the process objectives are explained with generalised examples related to teaching and assessment. As part of their orientation they take a test to earn the right to be considered as informed assessors. At the start of each course, their lecturer gives subject specific examples as part of the introduction to the course. It has been found that exemplary university teachers find their own different effective teaching dimensions and strategies to achieve excellence. Hence, although staff development units may advise, the teaching techniques for attaining these goals are left as a matter of informed professional choice to the lecturer. Using data from the forms, individuals’ alignments can be calculated and grouped to find the mean alignment of any student sub-group of interest - males v females, experienced v novice students, etc.
Validation of the theory
When the courses are over and the academic results are compared with the alignment scores, it is possible to validate the theory for each course, and for each sub-group of students taking each course by correlating the "Alignment of Scope" with "Academic standards" and by correlating the "Alignment of Proportions" with "Enjoyment of learning".
The student changes were calculated in the same way and each compared with the lecturer's changes. Scope is calculated for each student as the total of the student/lecturer absolute differences in the raw numbers given for the three objectives. Proportion alignment for each student is calculated in a similar way but using proportional changes calculated from the six numbers. Also, the more aligned students and lecturer were on Proportion then the more the students enjoyed the course. These results agree with the theory.
It will be noticed that the Modern Language students were more aligned in both Scope and Proportion than were the History students. Correspondingly, we find that the mean academic results and enjoyment of the Modern Language students were higher than those of the History students. Although the sizes of the sub-groups were small, these comparative sub-group results are also in agreement with the alignment theory. Traditional SETs are a "post mortem" assessment, collected at the end of the course when it is too late to use this feedback to help the students who made the assessments. The data collected in-course can be processed by the same type of sensitivity analysis to calculate the optimum changes that should be made by the lecturer during the course.
Administrative decision point assessment of quality teaching
The minimum possible alignment scores illustrate the best teaching/learning that is possible with these subgroups of students and reflects the fact that students are not all equally amenable to required educational changes in Skills, Understanding and Attitudes. To give the lecturer some protection from such intransigence the decision point measure of quality teaching is taken as the actual alignment less this minimum/best possible alignment.
A novel research application of the alignment method measures, for the first time, the differential effort that teachers expend in teaching mixed ability students. Educators generally accept that teachers expend more effort teaching "less-able" students than in teaching the "more-able" students in their classes. For example, as students improve they become less dependent on teacher assistance and higher ability students use their time more productively while waiting for teacher assistance.
Conclusion
This article has only touched on the classroom assessment use of the Alignment Method. It has not discussed the staff and course development aspects of the method or the many benefits the method is designed to offer for Quality Assurance compared to traditional SETs. This paper demonstrates an alternative method of using students' evaluation of teaching (SETs) that circumvents many of the problems associated with traditional SETs. In particular, it shows how in-course feedback, consisting of only eight ratings, can be used to optimize post-course academic attainment.
Jeff C. Palmer is a teacher, success coach, trainer, Certified Master of Web Copywriting and founder of https://Ebookschoice.com. Jeff is a prolific writer, Senior Research Associate and Infopreneur having written many eBooks, articles and special reports.
Source: https://ebookschoice.com/skills-understanding-and-attitudes/
