Summary Notes by J. T. P. Yao, 8/27/98

There are approximately 446 references in this 148-page book. It is concerned with performance indicators and their implementation.

"But with the 1980s and the public’s perceived erosion of quality in education, the goals for higher education shifted in almost epidemic proportions to a focus on quality – by the academy and the public alike. From January 1983 to December 1990, for example, ERIC lists 13,541 publications explicitly referring to ‘quality’ in education."

"National goals for higher education in the 1980s – indeed, to the present – remain much the same as they were in the 1960s and 1970s, with an emphasis on equity, access, and student financial need. But even at the federal level, at least by the late 1990s, performance evaluation in the form of peer reviews and assessments by external agencies can be expected to intensify."

"A study of faculty workload in 1993 conducted by the Ohio Board of Regents found that teaching load had dropped 10 percent in the previous decade. … Paradoxically, as faculty workload ostensibly declined, the workload of the ‘average’ American was increasing. … A study of over 4,000 tenure track faculty at a wide range of institutions reached the following conclusions:

• The more time you spend on teaching, the less compensation.
• The more hours in class per week, the lower the pay.
• The greater the time spent on research, the higher the compensation.
• Faculty who teach only graduate students get paid most.
• The greater the number of refereed publications, the greater the income.

The pervasiveness of the incentive system can be aptly illustrated by a review of the criteria for the SUNY Distinguished Teaching Professorships, which are intended to reward the system’s best teachers. The criteria require – and the system-wide committee rigorously enforces – ‘a substantial record in research and publications.’ In contrast, the Distinguished Professorship, which requires a substantial national record in publications and research, requires no evidence of good teaching skills. … But what is truly amazing is that there are so many faculty in research universities who care about teaching despite the virtual absence of rewards for doing so."

"Although in one study ‘educational quality’ was improved initially as a result of the instructor’s research, it subsequently declined, and the final results were an increase in the cost per student, less quality time for undergraduates, and a proliferation of support services. … Spurred by such evidence, public policy makers are seeking more public accountability, particularly in relation to undergraduate education. … Any ‘significant and sustainable’ increases in productivity must come through the learner’s productivity; that is, students’ ‘demonstrated mastery of a defined body of knowledge or skills,’ for example, by using a set of prescribed measures or indicators for individually paced learning. The key ingredients for such a system are (1) a set of clear, comprehensive, and measurable learning objectives; (2) assessment instruments; and (3) moving the student on to a new set of objectives after mastery and measurement of the initial assignment."

"An indicator system must be flexible to permit evaluation of different kinds of institutions, using available data where possible to make the process more cost-effective. Multiple indicators are necessary to capture adequately the complexity of institutional performance. Finally, the designers of the system must remember that indicators are simply that – indicators of success, not precise measurements – and must be continuously improved to approximate a stated goal. Indeed, indicators as a system are best used to check a unit’s health and progress. These systems, when combined with other efforts like accreditation, program review, and peer review can become a respected form of monitoring quality. A good performance indicator system can best be used as one of many instruments to monitor quality assurance, enhancement, and effectiveness. But performance indicator models have additional uses beyond this role. … Performance indicators have five primary uses: monitoring, evaluation, dialogue, rationalization, and allocation of resources."

"The ongoing ten-state project by the Education Commission of the States (ECS) has identified certain general areas around which state goals and expectations are developing: effectiveness, efficiency/productivity, access, quality of undergraduate education, equity/diversity, state economic development needs, and contributions to reform of elementary and secondary education. … The following list includes the common core of performance indicators identified by the ten states in the ECS project:

• Enrollment/graduation data by gender, ethnicity, and program.
• Degree completions and time to degree.
• Persistence/retention rates by grade, ethnicity, and program.
• Number/percent of accredited eligible programs.
• Remediation activities and indicators of their effectiveness.
• Transfer rates to and from two- and four-year institutions.
• Pass rates on professional exams.
• Job placement data on graduates and graduates’ satisfaction with their jobs.
• Faculty workload and productivity in the form of student/faculty ratios and instructional contact hours.

Other lists, however, include entirely different indicators (e.g., see the following list).

1. Overall revenue structure.
2. Overall expenditure structure.
3. Excess (deficit) or current fund revenues over (or under) current fund expenditures.
4. Percent of freshman applicants accepted and percent of accepted freshman who matriculate.
5. Ratio of full-time equivalent student to full-time equivalent faculty.
6. Instructional grant aid as a percent of tuition and fee income.
7. Tenure status of full-time equivalent faculty.
8. Percent of total full-time equivalent employees who are faculty.
9. Maintenance backlog as a percent of total replacement value of plant.
10. Percent of living alumni who have given at any time during the past five years."

While the above list contain "critical success factors," these two lists "further signify the conflict between state and institutional definitions of ‘success.’"

"Limited resources, rising expectations, and a growing awareness of the interconnectedness of educational systems have implications for higher education and how its stakeholders will respond to heightened calls for public accountability. … It includes revitalizing a sense of academic integrity and, albeit reluctantly, providing more collective responsibility through a coherent, quality curriculum with an effective cadre of faculty. … Rankings based on reputation appear to have served more symbolic than informational expectations of management. … The ‘clubiness’ of higher education, a tradition of highly abbreviated site visits, and the potential for politicizing self-study teams in some regions of the United States remain a problem."

"Four recent trends will help shape the future of the academy, … First, higher education will continue to be viewed as a key asset. Second, accountability in the K-12 arena will remain more specific and focused, while higher education will continue facing a crisis of confidence from policy makers. Third, the dominant focus will return to undergraduate education, especially the general education component of the curriculum and graduates’ skills. Fourth, higher education will remain driven by the market and highly sensitive to community support."

Impediments to further development includes:

• How to convert hundreds of data to "information that decision makers can use remains problematic."
• Issues "of the validity and reliability of data generated from existing indicators must be addressed systematically."
• A "leap of faith exists between concerns over input or contextual characteristics and outcomes. … Sorting out the uses of indicators by various stakeholders could yield greater visibility of process measures but litter integration with most existing uses of performance indicators."
• Faced "with temptation of complexity, added administrative costs, resistance from faculty, and a less forgiving political culture, the probability increases that by the turn of the century most indicator systems will be viewed as too costly and ineffective as primary management tools.
• A "hands off mentality has long lingered as part of the academic culture in the United States."
• Despite "greater technological capacity to improve the efficiency of performance indicator systems, the focus of the indicators remains an issue."

"In sum, maintaining the balance between demands for public accountability and internal quality assurance in the last half of the 1990s will not be easy. … The key to strike a balance between campus autonomy and government accountability for all activities. To that end, the experiment is worth continuing."

Return to the Lohman homepage