Skills, Understanding and Attitudes

Traditional Student Evaluations of Teaching (SETs) are feedback forms returned by students at the close of a course. Institutions intend that data from these forms be used to improve the quality of teaching and as an assessment of quality of teaching for deciding faculty promotion and tenure decisions. Although it is recognized that students can offer valuable information on the appropriateness of teaching quality, it has also been recognized that these traditional SETs are likely to have negative effects on the quality of teaching. These negative criticisms are quite extensive and range from "dumbing down" of courses to restrictions on academic freedom. One patently obvious criticism is that the information given by one group of students at the end of a course cannot be used to improve the teaching on that course. Similarly, it can only be useful to future students to the extent that future groups of students are similar to the feedback group and to the extent that the course and teaching remain similar. However, courses and teaching methods hopefully evolve and the constituent subgroups of a student cohort can change considerably from one year to the next.

This paper introduces an alternative method of allowing students to assess the quality of teaching that circumvents many of the problems associated with traditional SETs. In particular it allows feedback to be used for optimizing teaching quality during the course for the whole class, for individuals or for identified subgroups of students within the whole group. The feedback is quick and cheap to process - as it requires only eight ratings from each course member.

 

The paper outlines the method and the theory behind it. These three objectives - kills, understanding and attitudes - are emphasized to a determined amount in the teaching and assessment of the course. Feedback forms used during the course give data on the lecturer’s and students’ expectations for change in these objectives. This data allows for calculations of the alignment between the lecture’s and the students’ expectations for change. The theory is that academic success is maximized when students and their lecture are working towards the same changes. The theory is re-validated with each course by correlations of alignments with results, which show that in-course alignment predicts postcourse academic success. This paper describes how the data are also used during the course to determine the changes that will best align in-course student/lecture expectations. The educational importance of this alignment method is that it offers a cheap, efficient and effective alternative to the widespread problematic use of traditional SETs for quality control of teaching in tertiary institutions.

 

Introduction

 

This article briefly reports an alternative system for assessing quality teaching in tertiary institutions and focuses on the student feedback part of the system. The traditional method of assessing quality of teaching has been by questionnaires that ask students to anonymously rate the quality of teaching on a 4 or 5 point Likert scale from strongly disagree to strongly agree. In the literature the use of these forms is called Student Evaluations of Teaching (SETs). SETs have been used in universities for more than thirty years to assess the quality of teaching and as an indicator of successful teaching for promotion and tenure decisions. Unfortunately, their use has been accompanied by many counter-productive effects such as discouraging innovative teaching, and deterring instructors from challenging students. Although their outcomes are intended to improve teaching, a major negative effect of also using them to varying degrees for promotion and tenure decisions has been to contribute to the lowering of academic standards. Results of analyses of SETs and expected grades suggest that instructors can "buy" better evaluations via more lenient grading.

The Alignment method

 

Cohen (1994) has introduced to education the term "instructional alignment", meaning the alignment of teaching, assessment and objectives. Instructional alignment has been found to compare favourable with the use of other commonly used strategies intended to improve learning performance, such as criterion-referenced tests, curriculum-based measurement and direct instruction.

Elia (1994) and Walker (1998) found that instructional alignment had an unusually high positive learning effect producing substantial improvement on the achievement levels of disadvantaged and low performing low socio-economic level school students.

 

It should be noted that critical thinking is expected to be promoted by teaching and assessment of professional competence. Similarly to common applications of Bloom’s taxonomy, which also emphasise different levels within domains, course objectives and content are used as vehicles for emphasizing the desired degrees of Skills, Understanding and Attitudes. What are aligned are changes expected by the lecturer and changes expected by the students in each of these three process objectives.

 

These forms are confidential, not anonymous. When students enrol, the process objectives are explained with generalised examples related to teaching and assessment. As part of their orientation they take a test to earn the right to be considered as informed assessors. At the start of each course, their lecturer gives subject specific examples as part of the introduction to the course. It has been found that exemplary university teachers find their own different effective teaching dimensions and strategies to achieve excellence. Hence, although staff development units may advise, the teaching techniques for attaining these goals are left as a matter of informed professional choice to the lecturer. Using data from the forms, individuals’ alignments can be calculated and grouped to find the mean alignment of any student sub-group of interest - males v females, experienced v novice students, etc.

 

Validation of the theory

 

When the courses are over and the academic results are compared with the alignment scores, it is possible to validate the theory for each course, and for each sub-group of students taking each course by correlating the "Alignment of Scope" with "Academic standards" and by correlating the "Alignment of Proportions" with "Enjoyment of learning".

 

The student changes were calculated in the same way and each compared with the lecturer's changes. Scope is calculated for each student as the total of the student/lecturer absolute differences in the raw numbers given for the three objectives. Proportion alignment for each student is calculated in a similar way but using proportional changes calculated from the six numbers. Also, the more aligned students and lecturer were on Proportion then the more the students enjoyed the course. These results agree with the theory.

 

It will be noticed that the Modern Language students were more aligned in both Scope and Proportion than were the History students. Correspondingly, we find that the mean academic results and enjoyment of the Modern Language students were higher than those of the History students. Although the sizes of the sub-groups were small, these comparative sub-group results are also in agreement with the alignment theory. Traditional SETs are a "post mortem" assessment, collected at the end of the course when it is too late to use this feedback to help the students who made the assessments. The data collected in-course can be processed by the same type of sensitivity analysis to calculate the optimum changes that should be made by the lecturer during the course.

 

Administrative decision point assessment of quality teaching

 

The minimum possible alignment scores illustrate the best teaching/learning that is possible with these subgroups of students and reflects the fact that students are not all equally amenable to required educational changes in Skills, Understanding and Attitudes. To give the lecturer some protection from such intransigence the decision point measure of quality teaching is taken as the actual alignment less this minimum/best possible alignment.

 

A novel research application of the alignment method measures, for the first time, the differential effort that teachers expend in teaching mixed ability students. Educators generally accept that teachers expend more effort teaching "less-able" students than in teaching the "more-able" students in their classes. For example, as students improve they become less dependent on teacher assistance and higher ability students use their time more productively while waiting for teacher assistance.

 

Conclusion

 

This article has only touched on the classroom assessment use of the Alignment Method. It has not discussed the staff and course development aspects of the method or the many benefits the method is designed to offer for Quality Assurance compared to traditional SETs. This paper demonstrates an alternative method of using students' evaluation of teaching (SETs) that circumvents many of the problems associated with traditional SETs. In particular, it shows how in-course feedback, consisting of only eight ratings, can be used to optimize post-course academic attainment.

 

Jeff C. Palmer is a teacher, success coach, trainer, Certified Master of Web Copywriting and founder of https://Ebookschoice.com. Jeff is a prolific writer, Senior Research Associate and Infopreneur having written many eBooks, articles and special reports.

 

Source: https://ebookschoice.com/skills-understanding-and-attitudes/

Teaching To The Test

The phrase "teaching to the test" commonly means the practice of using a state-mandated test as a guide in deciding what to teach and how to teach it. However, this simple definition understates the complexity of the issue. On one hand, teaching to the test can be a case of the tail wagging the dog, where the needs of the test becomes more important than the teaching. It can even indicate an attempt to subvert the testing process, to beat the system. But seen in a positive light, teaching to the test can describe purposeful efforts to teach students knowledge and skills that have been established as important and included in mandated standards and assessments.

 

Why has this become an important issue?

 

Almost every state now has mandated tests for students. More and more, test scores are used for accountability-to make decisions about school accreditation, staff job security or pay, and student promotion and graduation. As the tests have became more high-stakes, the practice of teaching to the test has also increased dramatically. School personnel want their students to succeed and show what they know on the tests, and they often feel pressure to use any means available to raise scores. However, while families and the general public are demanding higher standards and higher scores, there is increasing concern, sometimes very vocally expressed, that the time and effort spent teaching to the test is educationally shortchanging students.

 

What's wrong with teaching to tests?

 

There's nothing wrong with teaching the general content and skills included on a test, as long as the test is assessing the "right" things and asking students to demonstrate their knowledge in ways that parallel real-world applications. The problem often develops when a test does not match standards for what students should know and be able to do, covers a very narrow set of objectives from the broader base of knowledge and skills included in standards, or includes mostly items that focus on recall of isolated facts. In cases such as these, both experts and practicing educators fear that teaching to the test may:

 

-    narrow or distort the curriculum;

-    emphasize use of short-term over long-term memory;

-    discourage creative thinking;

 

When is teaching to the test appropriate?

 

In general, the better the test, the more it can be used as a guide for good instruction. There is much less controversy about teaching to the test when the test itself:

 

-    reflects solid content standards;

-    assesses a broad range of knowledge and skills;

 

How can we teach to the test the right way?

 

-    Legitimate teaching to the test is not instruction targeted at specific items that will appear on the test, or that appeared on last year's version. Instruction can, however, appropriately be targeted to the general content and skills that will be assessed.

-    Although subjects specifically included on state-mandated tests (often mathematics and reading) should be taught, schools should not shortchange other core subjects such as social studies and science, or enrichment courses such as art and music.

-    Students should be made familiar with the format of the tests, including the types of questions. But even if the tests emphasize a single format (for example, multiple-choice questions) students should also be given experience with other types of test questions, such as short-answer and essay questions.

-    Teaching to the test can involve a moderate amount of instruction in test-taking skills, but these skills should be generalizable so that students can use them on other tests as well. For example, teachers might remind students to read directions carefully and give them practice doing so, but students should not spend much time practicing very specific test-taking activities, such as filling in answer bubbles.

-    Teachers should use a variety of instructional methods (for example, some that emphasize critical thinking skills) even if the state-mandated tests themselves stress memorization and recall.

 

So, is it okay to teach to the test?

 

There is no easy answer.

 

Both educators and the public agree that it is important to help students master the knowledge base established in national, state, and district standards. Students should also be helped to acquire the skills needed to accurately demonstrate their knowledge and skills on whatever assessments are used.

 

In deciding how much to teach to a state- or district-mandated test, educators should think about the quality of the test itself. The better the content and format of the test, the more useful it will be as a guide to high-quality instruction. In a perfect world, teaching to the test would automatically provide students with solid instruction. But schools operate in the real world, and the tests themselves are not always perfect. Educators find themselves weighing the goal of high student scores against broader educational needs, with student achievement hanging in the balance.

 

Why has the use of assessment data gained attention recently?

 

Over the last ten years, standards and assessments have played a growing role in public schools. Most states now require schools to give their students an array of specific standardized tests that are often used both to rate schools and districts and to make decisions about student promotion, retention, or high school graduation. Whether you see the increase in mandatory testing as a blessing or a curse, the fact remains that assessment plays a big part in education today. On the positive side, educators are developing increasingly more sophisticated ways to use assessment data to improve instruction. This application of data-based decision making can be a powerful tool at the district, school, and classroom levels.

 

How can data-based decisions improve instruction?

 

Whether an assessment is mandated at the state level, required by the district, or developed by a school or a teacher, it will yield results that indicate what individual students or groups of students know. These results can be used to improve instruction in a variety of ways. Here are just a few examples:

 

-    The test results of an individual student, in particular his or her responses to specific test items, can indicate the skills the student has mastered and the skills in which he or she needs more instruction. Students can be provided with extra help in areas where they are weak, preventing them from falling behind.

-    Class results can show individual teachers areas that most students did not master, so that teachers can find ways to teach those areas more thoroughly or effectively. They can also show areas where teachers are doing a good job, areas most students are mastering.

-    Assessment results can provide a focus for school improvement plans based on how well most students scored in various subjects.

 

Is there any evidence that using assessment data can improve instruction?

 

Current research provides plenty of evidence that assessment data can be a powerful tool to guide instruction. For example, a new study found that schools that were successful in closing the achievement gap for minority students tended to make sophisticated use of assessment data to examine student performance. Similarly, in a study of six school districts that have been successful in raising student scores on state-mandated tests, all of these districts began their improvement efforts by carefully reviewing test data. And after studying successful schools to determine why they were doing well, we identified "using data to drive improvement" as one of the keys to success.

 

How does a school create a data-based assessment system?

 

The first step in using assessment data to improve instruction is to identify the key questions-to ask what the school or school district wants to learn from the data. Some possible questions are: How are our students performing in relation to standards? Why are our students performing at the level they are? What can we do differently in order to increase the number of students meeting or exceeding standards?

 

Other important steps in the process of using data to guide instruction are:

 

-    collecting and organizing assessment data;

-    examining student performance data to identify strengths and weaknesses;

-    considering what the causes of strengths and weaknesses may be;

-    making plans to address needed areas; and

-    using data to monitor improvements.

 

What are the components of an effective data-based decision making system?

 

According to research, most schools or districts that have effective data-based decision making systems do the following three things:

 

-    They employ a data expert (for example, a Director of Program Improvement and Data Analysis) at the school or district level. This person helps principals and teachers understand assessment results and make decisions about how to use data to improve instruction.

-    They provide staff development to help principals and teachers learn how to use data effectively, and give teachers time to analyze and discuss assessment results in building, grade-level, and departmental groups. Most data-driven districts and schools are finding that they need to reorganize their schedules to provide opportunities for teachers to work together.

-    They have access to good data. Educators need to become aware-and to make their communities aware-of the quality of the assessments they are using. The "best" assessments are those that have been carefully aligned with standards, because these assessments provide information about knowledge and skills that students should be mastering. The results of these assessments then need to be presented in a "disaggregated" form to provide information about the performance of specific student groups.

 

Schools that have embedded use of assessment data into their instructional improvement efforts have also found that one annual test does not provide enough information to allow teachers to adjust their instruction to students' changing needs throughout the year. Many districts have developed additional, short assessments that are directed to the specific needs of their schools and students and that parallel the state-required assessments.

 

 

Jeff C. Palmer is a teacher, success coach, trainer, Certified Master of Web Copywriting and founder of https://Ebookschoice.com. Jeff is a prolific writer, Senior Research Associate and Infopreneur having written many eBooks, articles and special reports.

 

Source: https://ebookschoice.com/teaching-to-the-test/