Davis review

When research is not enough

Glyn Davis

Derek Bok
Our Underachieving Colleges: A Candid Look at How Much Students Learn and Why They Should Be Learning More
Princeton University Press, $52.95 hb, 413 pp, 0691125961

On a cold, grey day in February this year, economist Larry H. Summers announced his resignation as president of Harvard. Though some undergraduates gathered in Harvard Yard to wave signs saying ‘Stay Summers Stay’, the rift with faculty and the governing board proved too much. Summers issued a dignified letter to the Harvard community, shook hands with well-wishers, and disappeared.

Much media commentary on his resignation suggested a hostile faculty had campaigned against a visionary leader. Lawyer Alan Dershowitz called the outcome a ‘dubious victory for the politically correct’. An Op-Ed writer in the Wall Street Journal described the move as a ‘coup d’École’, citing clashes between Summers and professors over a petition to divest from Israel, comments about the under-representation of women in science, and clashes with prominent black faculty.

Such analysis restates a now familiar critique of American higher education. Captured by shadowy forces of ideology, professors allegedly resist leadership from above and criticism from students below. Summers becomes a convenient symbol of the alleged ungovernability of the American university, a man whose plans for revising curriculum and a new campus to accommodate growth sparked a revolt of the tenured.
Since Harvard remains an endless source of copy, no doubt books will appear soon to dissect the Summers years (2001–06). These will likely show a more complex narrative at work, and may also highlight the often unhappy history of leadership at the richest and most famous university on the planet. President Abbott Lawrence Lowell (1909–33) once described Harvard administration as ‘tyranny tempered by assassination’. His predecessor, Charles William Eliot (1869–1909), was asked to define the single most important attribute for a successful college president: ‘The capacity to inflict pain,’ he answered confidently.

One former Harvard president unlikely to comment on the fall of Larry Summers is Derek Curtis Bok (1971–91). Though seventy-six and long retired from university administration, Bok resumed the role of president in July, as the Harvard Corporation scrambled to find a long-term successor.

Yet Bok’s most recent book, Our Underachieving Colleges, published just before Summers strode out to face the media scrum, signals clearly his concerns about American higher education. Bok is unmoved by conventional pieties about universities held to ransom by tenured radicals. He rejects the doomsday tone of much recent writing on higher education, with its claims of mediocrity and subversion by postmodernists (has a passing intellectual current ever been credited with such apparent influence?). Such polemics, he argues, are long on rhetoric but short on evidence.

Instead, Bok is worried by a pervasive conservatism in higher education, specifically the refusal by professors to confront evidence about poor teaching. His concern is less content than process — without more attention to classroom experience, America’s myriad colleges and research universities will not meet their obligations to students or communities.

This is brave territory for Bok to explore, because assertions about teaching quality inevitably run into problems of data. The unreliable nature of teaching quality measures is a core issue for universities: since only research can be confidently counted and ranked, research league tables dominate discussions of performance. With few proxies of overall quality available, students may select a university that focuses all its resources on research performance rather than on pastoral care and classroom excellence. Reputation is no guarantee of a great student experience.
Bok is careful to survey beyond America’s élite universities to a wide array of institutions. Yet he need look no further than Cambridge, Massachusetts, for examples of dissatisfied students. Take, for example, Privilege, by recent graduate Ross Gregory Douthat, published in New York last year. In often startling prose, Douthat catalogues his cumulative disappointment with the Harvard undergraduate experience: students obsessed with networking (‘a Harvard education is a four-year scramble to ingratiate oneself’); professors concerned with protecting research time; incoherent electives; grade inflation; and classes taken by young teaching assistants desperately concerned about their career prospects. ‘I give out mainly As,’ one TA tells Douthat and his classmates. ‘A few Bs maybe. But I don’t like to, you know?’

Bok understands the pressures that would leave teaching little valued in institutions that claim to be great centres of education. He argues that academics and university leaders ignore evidence about the futility of many time-honoured teaching methods. Professors resist taking instruction; they see teaching as innate, rather than a skill to be learned and developed. As Bok observes, ‘Since faculty members normally keep abreast of published work in their fields, the content of their courses tends to be reasonably up to date. The same cannot be said of their teaching methods.’

Lectures remain the most popular form of instruction in American colleges, despite overwhelming evidence that students learn best through active learning — problem-based learning, study group discussion and laboratory exploration. Bok cites studies to show that most students cannot recall factual content from a lecture within fifteen minutes of class concluding. One study from the University of Texas, at indicates that professors, on average, spend eighty-eight per cent of available teaching time lecturing, and just five per cent talking with students. The same literature suggests that only eight per cent of professors consult the literature on teaching research.

Bok is concerned about more than classroom experience. He argues that curriculum design in many four year colleges fails to achieve the aims of a generalist education, concerns such as the ability to communicate, character building, citizenship, living with diversity and preparing for a global society. Faculty press early for specialisation, with unexpected consequences for broader graduate attributes:

Unfortunately, far from reinforcing other aims of undergraduate education, many concentrations appear to have the opposite effect. Thus Alexander Austin’s study of 24,000 undergraduates revealed that majoring in engineering was associated with declines in writing ability, cultural awareness, political participation, and a commitment to improving racial understanding. Majoring in education proved to be negatively associated with self-reported growth in analytical and problem-solving skills, critical thinking, public speaking and general knowledge. Other concentrations were associated with declines in critical thinking (fine arts) or civic engagement (science) or writing ability (science).

Why would teachers rely on teaching methods and curricula known to be inadequate? Bok identifies a number of causes: the craft training system that sees junior academic staff learn by imitating existing professors; the efficiency of large lectures over small classes; time pressure on staff to excel in research; the inherent conservatism of a profession that sees no need to change; and the inability of university leadership to make teaching quality a pressing issue. Advocates of reform are caught by the difficulties of measuring performance, the lack of national standards, and the reality that ranking systems such as the influential US News and World Report pay scant attention to measures of classroom quality. All incentives run strongly toward the status quo.

Not everything is gloom, of course. Bok finds examples of excellent teaching practice, and of professors willing to learn and innovate. One striking example, inevitably, draws on Harvard. Eric Mazur taught a quantitative introductory physics course to undergraduates for some years using the standard approach of lectures, distributed class notes and set textbooks. His student evaluations seemed sound, so Mazur saw no need for change until, by chance, he read an article suggesting that physics students in courses such as his rely on memory, rather than understanding, to solve assigned problems.

Somewhat affronted, Mazur tested his students. He was disappointed to find little grasp of underlying scientific principles, and decided to change fundamentally his teaching approach. Instead of distributing a lecture summary after class, Mazur now required from students a short paper prior to his lecture. This identified what students had understood from the textbook and what remained puzzling, and so allowed lectures to focus on problem areas. In class, Mazur abandoned the one-hour lecture, speaking instead only briefly at the start of the class, before setting short exams on key physics concepts. Students were invited to discuss their findings while he compiled scores and identified continuing problems to be solved in class or carried forward to tutorial groups.

Evaluation soon recorded a significant improvement in understanding. Bok reports Mazur’s undergraduates doubling their understanding of physics concepts compared with earlier cohorts. Mazur published his method and results in academic journals. Yet ‘his techniques have yet to penetrate more than a small fraction of the quantitative science courses across the country, leaving most students still at risk of getting through by memorising material they do not truly understand’.

Such attention to research on teaching, Bok suggests, is distressingly rare. As he observes with gentle irony, ‘faculties seem inclined to use research and experimentation to understand and improve every institution, process, and human activity except their own’. American universities remain teaching institutions but not ‘learning organisations’ committed to self-examination. Bok quotes surveys suggesting that nearly ninety per cent of college professors consider their teaching ‘above average’.
To make a difference, Bok seeks robust indicators of teaching quality. ‘If nothing much turns on the results, faculty members will ignore the tests.’ At present there is ‘no compelling pressure to improve undergraduate education’. Professors — like everyone else — find close evaluation unwelcome and uncomfortable. Bok argues that only when teaching scores appear in university rankings, and so influence student choice, will reform follow. While Bok sees a role for philanthropic foundations in encouraging closer attention to teaching quality, and for governing bodies to ask awkward questions, he looks to universities to solve their own problems. ‘Useful reforms can come only from within the universities,’ he warns. ‘Ultimate success or failure … will depend on the faculty.’

Australian governments have not been so patient. Under Education Minister Brendan Nelson, teaching scores became the basis for scarce additional funding. The first round of the Learning and Teaching Performance Fund in 2005 distributed $54.4 million in funding to fourteen universities, based on a composite index of measures, weighted toward student satisfaction scores. The announcement sparked a long and unresolved controversy over the validity of scores and the fairness of outcomes; a Department of Education, Science and Training advisory group, carefully drawn from winners and losers alike, is now reviewing the methodology.

Whatever the merits of the Nelson decision to award dollars based on contested teaching scores, his action focused attention on teaching quality more sharply within Australian institutions. Teaching evaluation has a long history in Australia — many measures Bok seeks for American colleges are already commonplace locally. A new funding source has heightened interest. Since universities are filled with clever people, this has inspired some cuteness with rules and measures. But it has also made Academic Boards ask pointed questions about courses and teachers with poor appraisal results. Many Australian universities now require new academic staff to undertake teaching training, and encourage mentoring, feedback and structured reflection on teaching practices. Such measures are reported through statutory requirements that would be resisted fiercely as infringements on academic freedom in the United States.

A renewed focus on teaching raises important questions about the purpose of a university. International recognition systems stress research above all other activities. Indeed, the lure of the Shanghai Jiao Tong index, which ranks universities exclusively on research performance as measured by science-weighted US-based indices, is hard to resist. The Jiao Tong index has produced a single international ranking and so changed forever competition between universities, whatever the widespread pretence to indifference. ‘We don’t pay much attention to such rankings,’ a senior academic at Heidelberg University told me last year. ‘But then we’re number sixty-four in the world, so we don’t need too. I notice your institution is only eighty-second. Why am I talking with you?’

There are other ways to think of universities, better games to play. In contemporary Western practice, an ideal university excels at three core activities: research of course, but equally learning and knowledge transfer. When these three strands are tightly wound together, each reinforcing the other, greatness follows. No single ranking can measure this triple helix, but together these goals, working together for researchers, students and community, define the purpose and ambition of higher education worth taking seriously.

Bok finds himself in the unusual position of being able to practice what he has preached. Our Underachieving Colleges is a superb book, driven by tough questions, impatient with opinion and nostalgia, concerned to test propositions against evidence. Since Bok’s subject matter is American, his solutions reflect the political culture in which American colleges work.
Larry Summers does not appear in the index to Our Underachieving Colleges, yet he drew some of the same conclusions. His letter of resignation cites complacency as among the greatest risks facing Harvard. Standing in Harvard Yard, Summers noted surveys showing that ‘the quality of experience we provide our students is not fully commensurate with their quality or the quality of the Harvard faculty’.

Derek Bok the author has identified the sources of conservatism among American professors when teaching the next generation. Now Derek Bok the University President has an unexpected chance, once again, to inspire and to teach.

Glyn Davis is Vice-Chancellor of the University of Melbourne, and was a Harkness Fellow at Harvard during a small part of the Bok presidency.

Download print-friendly version.