Comments on Teaching Evaluation

By C. H. Samson, 30 June 1997

General Observations

· The same questions related to how to achieve excellent teaching and how to evaluate it recur with some frequency in academic institutions, leading to intensive studies, reports, and recommendations that may be adopted to some degree, usually with the result that in a few years continual modifications occur that lose sight of the original rationale.

· New efforts to answer these questions are organized and undertaken with little regard to prior studies. It is certainly true that conditions change such that previous work may be outdated. I suggest that in many cases, however, similar-if not the same-ground is covered and some benefits might be realized by not ignoring previous findings.

· Teaching effectiveness and evaluation are obviously not simple issues, and certainly merit careful study. There are different points of view from which to approach these issues, and it is relatively easy to adopt a single point of view that does not really produce a generally satisfactory solution.

 A suggested Approach

 Without trying to offer a solution to the questions, in my opinion these steps make sense toward finding the overall best approach:

 Mission Establishment 

1. Establish a general mission for the study, e.g., "To determine the most effective approach for achieving effective teaching/learning at Texas A&M University, recognizing the University's institutional mission related to teaching, research, continuing education, and public service." It seems to me that it is very important not to address how to evaluate teaching in isolation from the more fundamental question of how to achieve effective teaching/learning. 

Existing Process Representation 

2. Represent in some fashion-flowcharting, perhaps-the existing process for achieving effective teaching/learning.

3. Identify the significant stakeholders related to why teaching/learning effectiveness is important. These might include, for example, students, faculty/staff, administration, employers, the State of Texas, as well as others.

 Diagnosis of Existing Process Deficiencies 

4. Diagnose any deficiencies in the existing process. Identify (by brainstorming, perhaps) possible causes of these deficiencies. 

5. By appropriate means, from the array of possible causes identify those causes that are considered to be real causes of deficiencies. Use of focus groups could be one approach.

 Remediation of Deficiencies 

6. Take note of the diagnosis of process deficiencies: 

a. Make the best possible determination of the attributes of graduates desired by various stakeholders. Establish an "attributes model" that best satisfies all stakeholders. 
    1. Establish the best "teaching/learning model" consistent with the "attributes model." Embedded in the establishing of the desired "teaching/learning model" are several important "subquestions," such as: 
    • What are the criteria for effective teaching?
    • How can teaching effectiveness be measured (assessed) against these criteria?
    • How can teaching effectiveness be continuously improved?
    • What are the other elements of the desired teaching/learning model that need to be addressed-laboratories; computers; administrative support; teaching loads; teacher-student-administration interaction; library enhancement; class size; teaching media; faculty/staff/administration professional development; student evaluation; administration evaluation; documentation of what is expected of students, faculty, staff, and administration; a clear statement of College mission, vision, value system, operating philosophy, and goals; others? 

Implementation 

  1. Implement the new process. 

    2. Prior Texas A&M Studies 

    It may be of some historical interest that some 33 years ago President Rudder appointed a Faculty Advisory Committee with the following charge:  

    The Advisory Committee will furnish the Committee of Deans by September 15, 1964, recommendations for faculty member evaluation from which a final plan can be developed by the Committee of Deans. 

    Specifically, the Advisory Committee is asked to develop general guidelines for (1) employment of new personnel in the academic ranks, (2) promotion of academic personnel from one rank to another, (3) merit salary adjustments within ranks, (4) periodic department head evaluation.  

    During its study, the Committee obtained a considerable amount of material, including related material from other institutions and organizations, as well as previous studies at Texas A&M. 

    The Committee comprised members selected by each of the four colleges, four members elected at large by the faculty as a whole, and a chairman-who happened to be myself-appointed by President Rudder. Between June 1 and September 21, the Committee met 21 times, with each meeting averaging something over two hours in duration. In addition, various subcommittees undertook special studies for the Committee. 

    The Committee did propose several recommended actions. My recollection is that these were adopted by the Academic Council, implemented, and followed for several years. By no means am I suggesting that these recommendations are ideal, or that they were even ideal at that time.

    The recommendations covered these items: 

    A. Employment of New Personnel in the Academic Ranks

    1. Recommended Procedure

  2. Rank Clearance of a New Employee
  3. Rank Assignment of a New Employee
  4. Qualifications for Research and Extension Rank
  5. Employment of Graduates of Texas A&M

B. Promotion of Academic Personnel from One Rank to Another

  1. Criteria for Rank
  2. Rank Clearance for Promotion
  3. The Individual's Annual Report
  4. Staff Achievement Record
  5. Student Evaluation Reports
  6. Department Advisory Committee
  7. Rank-Distribution Ratio

C. Merit Salary Adjustments within Ranks

D. Periodic Department Head Evaluation

  1. Term of Appointment
  2. Department Head Achievement Record
  3. Faculty Member Feedback Report

E. Related Considerations

  1. Private Office Facilities
  2. Laboratory Facilities
  3. Library
  4. Participation in Learned Society Activities
  5. Teaching Loads
  6. Well-Defined Teaching Loads

Sample forms were provided. 

It may be of some interest to quote briefly from a portion of the report: 

"First of all, the guidelines recognize that in the absence of objective criteria for measuring performance there is no substitute for administrative judgment. The guidelines simply undertake to make these judgments as systematic and comparable as possible. Second, they recognize the limited ability of administrative judgment to differentiate. To quote Douglas McGregor (The HUMAN SIDE OF ENTERPRISE, 1960, p.82) again, 'Using fairly simple procedures, and some safeguards against extreme bias and prejudice, it is probably fair to say that we can discriminate between the outstandingly good, the satisfactory, and the unsatisfactory performers.' . . . They [the guidelines] seek to assure a systematic flow of information to the centers of decision-making rather than to prescribe the process by which decisions are to be made."  

A copy of this report is in the Evans Library. 

Subsequent to this study, there was an initiative by students to become more active in teaching evaluation, and as a result President Rudder continued the Committee augmented with four students. This led to some modification of the evaluation process that provided for student input. In arriving at the modified approach that accommodated student input, some the issues discussed were:  

Concerns of Faculty: 

  • How could the process be prevented from providing excessive weight to the limited ability of students to evaluate teaching?
  • How could the process avoid having student evaluations become the primary basis for promotion and merit increase?
  • How could the temptation of faculty to "curry" good evaluations at the expense of good teaching be avoided?
  • How could a faculty member discriminate between meaningful student ratings and those of students disgruntled because of a poor grade?
  • It is not right to be criticized without knowing the one who is criticizing.
  • Often several years after students graduate, they consider their best teachers to be those they would not have so recognized during students' college years. 

Concerns of Students: 

  • If student opinions reached only the individual faculty member, those faculty who "needed the information most would simply deposit the ratings in the circular file. Unless the ratings get to at least one decision maker, they won't help where help is needed most."
  • Students are afraid to be identified if they make negative evaluations for fear of grade retribution.
  • Students pay for their education, and therefore have a right to have input.
  • Students should have some opportunity to select teachers who are regarded as effective and fair. 

Resources 

Much resource material is available that could assist the College Committee in its study. These include:

  • Published material.
  • Studies by other institutions.
  • In-house material. For example, several years ago a College of Engineering Master Teachers Panel developed through a brainstorming exercise an array of possible obstacles to effective teaching. A similar exercise was done by students in a graduate class in quality management. I have the results of both these exercises if they would be of any value. Also available from an exercise in another graduate class is a statement of mutual responsibilities of teachers and students in producing an effective teaching/learning relationship.

Return to the Lohman homepage

© 2001 The Lohman Professorship all rights reserved. Last modified