"Progress of the Engineering Education Coalitions, by H. R. Coward, C. P. Ailes, and R. Bardon, SRI International Report, May, 2000 [Referred to by Dr. Dan Abrams], 41 pages, the report is available in its entirety on the Internet at http://www.nsf.gov/cgi-bin/getpub?nsf00116

Summarized by J. T. P. Yao, 7/24/00

"… Over the course of three annual competitions, NSF funded six Coalitions. Program goals included:

• A dramatic increase in both the quality of engineering education and the number of degrees awarded in engineering, including those to women and underrepresented minorities;
• The design, implementation, evaluation and dissemination of new structures and approaches affecting all aspects of undergraduate engineering education;

• The establishment of new linkages among all types of U.S. engineering institutions, large and small…

SRI International, working with COSMOS Corporation, conducted a … study... The elements of the study consisted of:

1. Archival review;
2. Field visits to the lead and one other member institution in each Coalition;
3. Focus group interviews with students using Coalition-developed products;
4. Telephone interviews with the Coalition leader at institutions not visited;
5. A survey of faculty participants in Coalition projects at the various member institutions;
6. A survey of deans of engineering schools and colleges not involved in a Coalition.

The time-frame of the study was essentially the prospective midpoint of the Coalition's life span - 1997-1998 - the point at which they were being reviewed for the second phase of funding, depending on which competition had funded the individual Coalitions." [Six Coalitions as follows: ECSEL (1990-94, 7 universities); Synthesis (1990-94, 8 universities); Gateway (1992-96, 8 universities); SUCCEED (1992-96, 8 universities); Foundation (1993-97, 7 universities); and Greenfield (1993-97, 5 universities).]

"… Aside from the unusual organization of the Greenfield Coalition, among the other five Coalitions, all of the new courses demonstrated the variety of features desired by NSF… Several Coalitions had instances of external funding that supplemented NSF's funding. However, it was difficult to ascertain the extent of 'true' ... Thus, there were occasions when the field visit team was told that faculty were not encouraged to participate in a Coalition's activities because it detracted from their research productivity. … Industry has definitely been involved and appreciative of Coalition efforts, but its level of commitment and involvement is limited. …"

"With regard to the dimensions and complexity of the tasks confronting the Coalitions, it should be noted that the Coalitions were confronted with three difficult processes simultaneously: scaling up…; institutionalizing…; and coalescing plans and actions across institutions. …"

"… The paucity of assessment information suggests that it is minimally used for continuous improvement. Further, … the assessments have not been well implemented because the … institutions have generally been unable to overcome such conditions as:

• Difficulty in defining common data collection instruments …;
• Unacceptably low response rates, …
• Infrequent use of or lack of access to archival sources …
• Inability to define non-Coalition groups of students or faculty for comparison purposes; and
• Lack of useful database to capture more easily … students' … information.

Even when such conditions have been overcome, the assessment results cannot be easily interpreted. …"

"… In the study, the faculty outcomes of interest were observations of any shifts from traditional, passive, lecture-based instruction to newer models of active, collaborative learning… Evidence of such changes in culture was spotty and possibly premature to expect at this stage. … Nevertheless, although evidence of desired culture changes at the time of the study was limited, the field visits and surveys did provide some evidence of changes taking place. These could hardly be characterized as strong trends, but there seemed to be desirable, if relatively recent, developments in four areas.

The first and possibly most important pattern across Coalitions was the increased valuing of contributions to the practice of engineering…

A second consistent theme across all Coalitions was the development of cross-institutional and not just cross-discipline perspectives and interactions…

A third potential cultural shift involves outreach to non-engineering students and departments…

A fourth shift is evidenced by isolated examples (and, indeed, counter-examples) of shifts away from a research-only culture of promotion and tenure…"

"On the positive side, a number of respondents [campus-level PIs] pointed to the utility of travel funds to actually observe what was going on elsewhere. … It was also useful for Coalition faculty to observe where culture of change had emerged at other institutions. … On the down side, it was clear that both intra-Coalition interaction was difficult to achieve outside of meetings arranged by NSF … Campus PIs reported that interaction with institutions outside the Coalition Program was nearly non-existent."

"The telephone interviews replicated observations from the field visits with repeated admissions that dissemination was slow to come, behind schedule, and generally disappointing. … Faculty enthusiasts enjoyed projects developing new courses, projects, and other materials. The problem of getting these documented derived from many factors. The conflicting pressures on faculty, the small amounts of money provided individual projects at the end of the pipeline, and faculty skills at writing and presenting materials outside the classroom all contributed to dissemination problems. … The results of the survey of non-Coalition engineering schools seems to confirm that, aside from a couple of textbooks and a few CD-ROMs, Coalition materials seem idiosyncratic and hard to adapt. … it has legitimized scholarly publication concerning curriculum reform, but this takes precedence in the reward structure and tends to preempt any efforts that most faculty are prepared to undertake to work up Coalition products for dissemination. … Perhaps the broadest, most important and frequently noted point is that dissemination must be seen as a process - not just the distribution of a product. …"

"Reform efforts can suffer because of the need … to mesh iterative efforts to improve courses. The constant 'tweaking' of one first-year course was cited as an example of why it was difficult to move on to the next level. … It is hard enough to gain buy-in in one's own institution, much less to get an outside institution to enter into the process of adopting things developed elsewhere."

"The challenge most commonly identified was that significant, widespread reform of engineering education is harder than originally thought… Coalition courses are typically expensive to maintain. This makes them hard to transfer, as noted under dissemination, and they may prove hard to sustain after NSF Coalition funding ceases. Administrators are not eager to pick up these expenses. ... Beyond cost, there was a broader unease about the prospect of a breakdown of the Coalition culture and a falling away of people with the loss of focus that the Program gave. … Some local K-12 involvement was feasible, but broad efforts were precluded by other pressures, especially in research universities."

"… There is, in fact, widespread concern about how the momentum initiated by the Program will be sustained when Coalition funding is phased out. … Experiences ranged from an institution's deliberately discouraging participation by untenured faculty to being able to cite cases in which individuals were specifically helped in obtaining tenure through their Coalition work. … Pressures are particularly strong at schools with ambitious administrations that place emphasis on advancing the school's research reputation. … The need for enthusiastic, energetic, and charismatic leadership was constantly mentioned as critical to getting a Coalition started the maintaining its momentum. However, such leadership is not sufficient without support, both in terms of policy, encouragement, and supplementary funds from both engineering school and university administrators."

"The most common observations [by students] were that Coalition-related courses were considered to require more work, could be hard on Grade Point Averages, and could put a separate Coalition cohort at a perceived disadvantage in comparison with their classmates. … At the same time, it was common for students who made such observations to insist also that, hard as the effort had been, they recognized what it had done for them and that it often provided them with educational advantages over their classmates that outweighed the costs in terms of time, effort, and GPAs. …"

"… Particularly at research-intensive universities, there was a conflict between the faculty member's interest in curricular change and a university culture that was at odds with the reality that developing and using Coalition materials and teaching Coalition courses required extra time and effort in the classroom, especially in computer-based courses. … Coalition money often represented the means for substantial investment in computer equipment and software, but the pressures of obsolescence are constant. The students showed great acuity in their software reviews and expressed frequent frustration with the inability of new curricula to keep up with changes or new development in commercial programs."

"Teaming was widely recognized by the students as an essential part of their education and something that was a real asset in their encounters with industry. … There were frequent complaints of the difficulty of gaining faculty intervention when serious problems within a team surfaced. … Establishing leadership and how to deal with slackers dominated student comments. …"

"… The random sample of 302, drawn from lists of participating faculty provided to SRI by the Coalitions (a total of 1151 faculty names, the sample representing 26%), was given a choice of responding via the web or printing out survey to preview it and/or submit it in hard copy, or, through an SRI contact number, to receive a hard copy by mail. The overall response rate was slightly over 50% when the survey was closed. From the 152 responses received to the survey, 18 (12%) indicated that they had never participated in Coalition activities. Eight of these (5% of those responding) were not even aware of the Coalition Program's existence. …"

"Of the 132 valid responses from individuals involved in the Coalition program at their respective institution, 117 (89%) held professorial rank at some level, while the remainder held administrative or other non-teaching positions. With more than 75% of the respondents holding normally tenured positions, the data suggest that tenure-track junior faculty do not appear to be significant participants in Coalition activities… Almost two-thirds (64%) of faculty members reported redesigning or modifying courses or educational materials with faculty members within their Coalition, while only about a fourth (24%) had done so with individuals from other Coalitions. … Industry involvement in Coalition curricula most often took the form of a visiting lecturer in courses or labs. Among those teaching courses, 42% of the faculty reported having a guest speaker from industry, but only 13% had included industry guests in Coalition-related labs. … "

"Hardcopy (e.g., textbooks or paper modules) remains the dominant type of media used in Coalition labs and courses, with 77% of faculty reporting their use in Coalition courses, and 63% using them in their labs. E-mail is also used by more than half of the faculty in courses (60%), as are World Wide Web sites (52%). Multimedia modules, computer diskettes, and video are all used by over 20% of the faculty teaching Coalition courses."

"Dissemination of materials across Coalition institutions was relatively uncommon. Only 17% of faculty utilized World Wide Web sites developed at another Coalition school in the courses they taught, with 15% reporting their use in labs. …"

"The student workload for Coalition courses and labs was considered by faculty to be significantly higher compared with non-Coalition classes. The majority (69%) thought that students had either much more work or somewhat more work in the new classes. … Half or more also thought that the Coalition courses had had a positive impact on students' depth of understanding of subject matter (58%) and retention in engineering majors (50%). …"

"… While most reported that innovative teaching and curricular reform activities are currently valued about the same as before the Coalition, about a third thought that innovative teaching is currently valued more than prior to the Coalition in promotion decisions (36%) and in tenure decisions (28%). …"

"… Of the 193 Deans to whom questionnaires were distributed by a combination of e-mail and web-based administration, 126 valid responses were received, for a response rate of 65%. … Two-thirds of the responses (65%) were completed by the deans of engineering themselves, and another fourth (25%) for completed by the associate/assistant deans for academic affairs… Roughly twenty-five to thirty percent of the schools also reported that curricular reform activities and efforts are being weighed more heavily in promotion and tenure decisions than five years ago. While again, the vast majority reported no differences from five years previously, almost none reported that such activities are weighed less than five years earlier. …"

"Recommendations

System-wide reform … takes more than five or even ten years…

Despite possible impediments, one force strongly favoring reform is the gradual implementation of ABET's Engineering Criteria 2000…

There is much still to be accomplished along such lines as changing academic culture to integrate engineering education and research, enhance interdisciplinary and team-oriented teaching, and provide greater institutionalization of reforms …"

Return to the Lohman homepage