Part 1 (2/2)

While some of these additional noninstructional obligations are mandated by the inst.i.tutions that employ facultya”as in the university and department committee meetings that professors often complain abouta” many of these additional activities likely advance faculty careers, but are largely unrelated or only indirectly related to undergraduate instruction. Ma.s.sy and Zemsky have referred to the process whereby faculty gain increased discretionary time to pursue professional and personal goals, while undergraduate education is devalued, as an aacademic ratchet.a Ma.s.sy and Zemsky note: Put simply, those hours not used for teaching courses, for grading papers, or for meeting with students become available for research and scholars.h.i.+p, for consulting and other professional activities, and in most research universities, for specialized teaching at the graduate level. Inst.i.tutional rhetoric about the importance of teaching notwithstanding, we believe that the reductions in discretionary time a.s.sociated with more and better teaching usually are not compensated by additional salary or other rewards, whereas success or failure with regard to other obligations carries significant rewards and penalties a Even when most faculty use their time to meet professional and inst.i.tutional obligations, the academic ratchet still s.h.i.+fts output from undergraduate education toward research, scholars.h.i.+p, professional service, and similar activitiesa”a process that we have termed aoutput creep.a23 Christopher Jencks and David Riesman several decades earlier provided a similar account of faculty movement away from undergraduate instruction at research universities in The Academic Revolution. They noted that the availability of external funding gave successful researchers significant leverage over the colleges and universities that employed them: Since the amount of research support has grown much faster than the number of competent researchers, talented men have been in very short supply and command rapidly rising salaries. They are also increasingly free to set their own working conditions. The result has been a rapid decline in teaching loads for productive scholars, an increase in the ratio of graduate to undergraduate students at the inst.i.tutions where scholars are concentrated, the gradual elimination of unscholarly undergraduates from these inst.i.tutions, and the parallel elimination of unscholarly faculty.24 In recent decades the allure of external funding for research has been greatly enhanced by the growth of commercial opportunities a.s.sociated with research activities in higher education. Federal government legislation, such as the Bayh-Dole Act of 1980, allowed colleges and universities to patent discoveries that had been developed with federal research support and facilitated the growth of university collaborations awith the private sector in the development of the commercialization of new technologies.a25 Colleges and universitiesa”inst.i.tutions that, according to Derek Bok, share with compulsive gamblers the trait that athere is never enough money to satisfy their desiresaa”eagerly embraced these new opportunities to acquire new sources of funding.26 Universities also engaged in these emerging corporate ventures to acquire the symbolic resources that the collaborations conferred. Sociologists Walter Powell and Jason Owen-Smith have astutely observed that athe commercialization of university-based knowledge signals the universityas role as a driver of the economy. Such a lofty status has much more legitimacy and cachet, and makes it possible for universities, especially public universities, to boast their success in creating employment opportunities.a27 Whether one focuses on aoutput creepa occurring as a result of an aacademic ratcheta that individual faculty engage in to expand their professional discretionary time, on the aacademic revolutiona produced by the expanding power of the faculty researcher that Christopher Jencks and David Riesman described in the late 1960s, or on the acommercialization of higher educationa following the Bayh-Dole Act of 1980 that Walter Powell and Jason Owen-Smith examined, one thing is clear: undergraduate education in many colleges and universities is only a limited component of a much broader set of faculty professional interests, and one that generally is not perceived as being significantly rewarded. And if there is any doubt that college professors are less likely than other individuals to focus on material incentives, recent surveys of students and faculty have found that faculty are more likely than students to report that being well off financially is an essential or a very important goal to them.28 We do not believe, however, that financial incentives are primarily responsible for faculty commitment to research. Rather, we believe that given the transformation of higher education, one of the few remaining moral bases for academic life is a quasi-religious commitment to embracing research as a avocational calling.a As Anthony Kronman recently observed, athe equation of scholarly specialization with duty and honor a makes the development of oneas place in the division of intellectual labor a spiritually meaningful goal and not just an economic or organizational necessity.a29 For many faculty, commitment to their own individual research programs is thus understood not as an act of self-aggrandizement or personal selfishness, but rather as a moral imperative that one must pursue and struggle to achieve regardless of inst.i.tutional obstacles.

While faculty distracted by professional interests other than undergraduate instruction share responsibility for the current state of undergraduate learning occurring on U.S. campuses, it is worth emphasizing again that the professoriate respond to incentives established not only by their larger professional fields of scholars.h.i.+p, but also more specifically by higher-education inst.i.tutions and the administrators who oversee the colleges and universities where they are employed. While many U.S. colleges follow governance policies that cede formal control over curriculum and instruction to the faculty as a whole, administrators have the inst.i.tutional authority and responsibility to determine work loads and ensure that faculty are spending sufficient effort on undergraduate instruction as opposed to other legitimate professional activities (e.g., graduate instruction, academic scholars.h.i.+p, and professional service).

If faculty at U.S. colleges can be described as being distracted by professional interests other than undergraduate instruction, it is likely even more the case that contemporary higher education administrators experience inst.i.tutional interests and incentives that focus their attention elsewhere. As former Harvard University President Derek Bok has noted: While (academic) leaders have considerable leverage and influence of their own, they are often reluctant to employ these a.s.sets for fear of arousing opposition from the faculty that could attract unfavorable publicity, worry potential donors, and even threaten their jobs. After all, success in increasing student learning is seldom rewarded, and its benefits are usually hard to demonstrate, far more so than success in lifting the SAT scores of the entering cla.s.s or in raising the money to build new laboratories or libraries.30 We believe that administrators are likely even more distracted than faculty from a focus on undergraduate instruction due to the simple fact that their professional lives (with the possible exception of administrators working in the area of student services) tend to reduce and limit their amount of interpersonal contact with students. After all, faculty on average spend eleven hours per week on teaching and advis.e.m.e.nt activities that to some extent must remind them of the significance of student learning.

One empirical way to highlight the extent to which administrators have allowed higher-education inst.i.tutions to drift away from an undergraduate instructional focus is to identify the staffing and employment changes that those inst.i.tutions have implemented in recent decades. While administrators at colleges and universities with strong traditions of faculty governance can legitimately claim that curriculum and instruction are appropriately considered faculty matters and not administrative responsibilities, decisions around employment structure and staffing are universally considered to be under the purview of administrators. In colleges and universities across the country, not only have part-time instructors increasingly replaced full-time professors, but resources have increasingly been diverted towards nonacademic functions. Sociologist Gary Rhoades has doc.u.mented that over the past three decades, athis group [of non-faculty support professionals] has become the fastest growing category of professional employment in higher education.a31 While some of these individuals have been hired for administrative functions such as human relations, accounting, and regulatory compliance, Rhoades has observed that the most significant increase has occurred in the broad area of student services including admissions, financial aid, career placement, counseling, and academic services such as advising and tutoring that have been rea.s.signed to non-faculty professionals. These amanagerial professionals,a as Rhoades has termed them, have come to comprise anearly 30 percent of the professional positions on campus and more than three times the number of administrative positions.a In related changes, the percentage of professional employees in higher education comprised of faculty has decreased from approximately two-thirds in 1970 to 53 percent by 2000.32 This internal transformation of higher education, while often focused on elevating student services as broadly defined, has implicitly deemphasized the role of faculty and faculty instruction per se at these inst.i.tutions. The nonacademic professionalization of higher education can also be observed in appointments to college and university leaders.h.i.+p positions, as well as their compensation packages. While the vast majority of higher-education leaders continue to emerge from earlier positions in the college professoriate, in recent decades individuals increasingly have been drawn from nonacademic backgrounds and hired through a process dependent on professional search consultants. About one in seven college and university presidents now comes from outside academia; the role of external professional search consultants in the selection process has grown from 12 percent in 1984 to more than half today.33 In addition, administrative positions in higher education have become increasingly well compensated.34 On average, college and university presidentsa compensation in the private sector is approximately $500,000, with many making over a million dollars per year. aWhen you have college presidents making $1 million, youare going to have $800,000 provosts and $500,000 deans,a Patrick M. Callan, president of the National Center for Public Policy and Higher Education has noted. aIt reflects a set of values that is not the way most Americans think of higher education.a35 While there is nothing inherently wrong with well-paid higher education administrative personnel, the nonacademic professionalization of higher education leaders.h.i.+p, and the process whereby it is identified, our concern here is simply about how these changes might affect inst.i.tutional attention to academic instruction. As the sociologist Steven Brint has noted, awe know that the backgrounds of top executives can influence the climate of the firms they lead a If this is true in corporations, is it not likely to be true a fortiori in colleges and universities?a36 Arguably, s.h.i.+fts in the character of administrative leaders.h.i.+p are a.s.sociated with the phenomenon of colleges and universities today becoming much more interested in the fulfillment of nonacademic services and functions, while focusing less on traditional academic instruction.

Indeed, as sociologist Mitch.e.l.l Stevens noted in his recent ethnography of a selective private residential college: aThe College is an academic inst.i.tution, and a justly proud one, but it also is proud of its twenty-eight varsity sports teams, its budding artists and musicians, its community service projects, diverse student body, spectacular campus, and loyal alumni.a37 Colleges and universities have secured their centrality in our society not only by providing credentials that aserve as ever more important cues about worker capability and character,a but also by amaking college life more athletic, more masculine, and more fun.a38 Colleges and universities are not just asievesa that sort and train students, but also aincubators,a atemples,a and ahubsaa”i.e., settings for the development of cultural dispositions, network formation, knowledge production, and inst.i.tutional relations.h.i.+ps.39 Changes in Inst.i.tutional Functions and Ident.i.ties.

Traditionally, U.S. colleges and universities had embraced both academic and moral education as primary inst.i.tutional functions and rationales. While Harvard historian Julie Reuben has shown how colleges and universities over time s.h.i.+fted the approach whereby moral education was inculcated in studentsa”with athe religious stage, falling roughly between 1880 and 1910; the scientific, from about 1900 to 1920; and the humanistic and extracurricular, roughly 1915a”1930aa”these inst.i.tutions defined their organizational missions in large part by embracing the responsibility of providing academic and moral guidance to young adults in their charge.40 Following World War II, however, colleges and universities that were enrolling increasing numbers of students turned away from these functions and embraced more narrowly defined technocratic ends, such as the generation of scientific knowledge and the production of graduates to fill professional and managerial positions. Some observers have largely celebrated these organizational changes. For example, Clark Kerr, former chancellor at the University of California, Berkeley, observed that in these transformed inst.i.tutions athere is less sense of purposea but athere are more ways to excel. There are also more refuges of anonymitya”both for the creative person and the drifter.a41 Other scholars, however, have lamented this transformation, worrying that U.S. higher education does not have aan adequate basis for establis.h.i.+ng a consensus of moral valuesaa”other than support for adiversity and mutual toleranceaa”and thus is ain the midst of a moral crisis.a42 Since the student rebellions of the 1960s, the extent to which collegiate life has embraced nonacademic pursuits has likely been aided and abetted by college administrators and staff who have alargely withdrawn from oversight of manners and morals.a43 While colleges once a.s.sumed a quasi-parental role and struggled with mixed success to ensure athe enforcement of academic and social rules,a educators and administrators have grown aless certain than they once were as to what students ought to be or become, and are reluctant to go to the mat with the young for principles in which they themselves only half believe.a Even if a consensus was reached on the definition of an appropriate and desirable code of student conduct, college administrators and faculty have often found it apolitically expedient to avoid collective regulation of student behavior.a44 Although administrators in recent years on some college campuses have implemented policies to limit and control alcohol and drug use, in most secular colleges there has been little inst.i.tutional responsibility taken for the moral development or social regulation of students. It is thus not particularly surprising that behaviors at odds with academic values, such as cheating on exams, have been demonstrated to have increased significantly in recent decades. In a longitudinal comparison of nine colleges, for example, college students who admitted that they copied from other students on tests or exams increased from 26 percent in 1963 to 52 percent in 1993. Rates of student cheating were particularly high in colleges that had no honor code governing student conduct.45 These developments are not unique to higher education; they have occurred concurrent with broad-based cultural changes in the relations.h.i.+p between youth and education. They occurred, for example, during an historic period where elementary and secondary students had begun to enjoy a wide range of new legal rights and ent.i.tlements that undermined studentsa sense of traditional forms of authority relations.h.i.+ps in education.46 Concurrently, legally mandated supplementary student services in special education programs increased dramatically, redefining earlier a.s.sumptions of individual and inst.i.tutional responsibility for managing studentsa academic and social difficulties. Middle-cla.s.s parents increasingly saw themselves less as collaborative partners with school authorities who were believed to possess legitimate authority in loco parentis and more as aadvocatesa for their childrenas educational needs. Educators became progressively more reluctant to require students to master certain forms of knowledge over other less culturally privileged ones. Students in Ka”12, and particularly in higher education, increasingly became defined as aconsumersa and aclients.a In this context, schools are expected not to provide quasi-parental guidance and social regulation, but instead to meet client needs through delivery of elaborate and ever-expanding services.

The effects of these broad-based cultural changes on higher education were enhanced by federal and state policies that s.h.i.+fted financial support from inst.i.tutions to individuals. As higher-education researchers Sheila Slaughter and Larry Leslie have doc.u.mented, in the early 1970s the federal government began formulating internal policy papers calling for aa freer play of market forcesa that would agive individuals the general power of choice in the education marketplacea as well as specifying alevels and types of student support which will make most inst.i.tutional aid programs unnecessary.a47 At the federal and state level, inst.i.tutional aid programs were increasingly replaced by ahigh tuitiona”high aid policy through which government gave aid to students rather than inst.i.tutions, thus making student consumers in the tertiary marketplace. Inst.i.tutions competed with each other to attract students and their Pell grants.a48 Student aid was essentially structured as an educational voucher. While the G.I. Bill of 1944 provided portable scholars.h.i.+p support for veterans to use at accredited inst.i.tutions, the higher-education reauthorization legislation pa.s.sed in 1972 provided portable financial aid to large numbers of students who were defined as qualified based on income levels. In recent years, this market-based logic has only been further extended by federal policies that have facilitated the growth of college finance models that rely on tax credits and student loans.49 Personal financial investment in higher education has significantly grown with increases in the cost of higher education and an expanded reliance on private credit-based financing. Specifically, from 1978 to 2008, tuition and fees (not including room and board) increased from $9,903 to $25,143 in private four-year colleges and from $2,303 to $6,585 in public four-year colleges in constant 2008 dollars.50 Family and student sources of financing also s.h.i.+fted, with the fastest-growing source of funding being private-sector loans. From 1997 to 2007, private-sector student loans in constant dollars increased almost seven times, from $2.5 billion to $17.6 billion.51 Approximately 60 percent of students graduating four-year colleges have taken out student loans; from 2000 to 2007 the average student-loan debt per borrower increased 18 percent, from $19,300 to $22,700 in constant 2007 dollars.52 In addition to student-loan debts, students during this period also increasingly used credit cards to support themselves and their educational expenses while in college. Undergraduates in their senior year in 2008 on average had $4,100 in credit card debt, with one-fifth of seniors carrying credit card balances greater than $7,000. Moreover, 30 percent of students reported putting tuition costs on their credit cards.53 The a.s.sumption of significant debt during college became typical, as did the hours many students spent in paid employment while attempting to complete their degrees.

Social scientists are just beginning to explore the implications of this s.h.i.+ft for how students are understanding and experiencing their college years. The increased debt burden could potentially serve to impose a new sense of self-discipline on students, and a refocused attention on academic activities. Alternatively, it might lead students to become distracted from their coursework by the importance of paid employment, or it might produce other unantic.i.p.ated consequences. Full-time college students on average today spend five hours more per week working than in the early 1960s, although national data suggests that fewer than one in six full-time students at four-year colleges work more than twenty hours per week.54 In terms of increased debt, an intriguing recent study of students at one selective southern Californian inst.i.tution found that undergraduates had little worry about their ability to find high-paying jobs after college to repay their student loans. Students reported that they defined the purpose of these loans as serving not just as an investment in the future but also as a means to experience fully a collegiate lifea”a personal objective that included a commitment to a student culture characterized by frequent socializing, travel, and entertainment.55 Regardless of how rising costs and increased reliance on loans affect student academic and social behavior, changes in the character of higher-education financing are potentially related to the deepening of consumerist orientations within higher education.

A market-based logic of education encourages students to focus on its instrumental valuea”that is, as a credentiala”and to ignore its academic meaning and moral character. The historical sociologist David Labaree has argued that awe have credentialism to thank for aversion to learning that, to a great extent, lies at the heart of our educational system.a56 Many studentsa lack of commitment to substantive academic learning is consistent with their definition of the situation: aIt is only rational for students to try to acquire the greatest exchange value for the smallest investment of time and energy.a57 Faculty also do not have much incentive to challenge this emerging reward structure, as conflicts with students over these matters potentially can distract from research, lower teacher or course evaluations, and generate administrative problems a.s.sociated with student resistance.

Private colleges and universities, of course have always to some extent adopted market-based orientations and competed for studentsa”just as students have competed for access to elite private education. In recent decades, however, as the market-based logic of higher education has been extended, public colleges and universities have begun to share more in common with their counterparts in the private sector. There are likely many positive consequences a.s.sociated with defining students as consumers and clients as schools become more responsive to articulated individual student needs. Our point here, however, is that there is no guarantee that students will prioritize academic learning at the core of their inst.i.tutional demands. There are many reasons instead to expect students as consumers to focus on receiving services that will allow them, as effortlessly and comfortably as possible, to attain valuable educational credentials that can be exchanged for later labor market success. As historical sociologist David Labaree has noted: The payoff for a particular credential is the same no matter how it was acquired, so it is rational behavior to try to strike a good bargain, to work at getting a diploma, like a car, at a substantial discount. The effect on education is to emphasize form over contenta”to promote an educational system that is willing to reward students for formal compliance with modest performance requirements rather than for demonstrating operational mastery of skills deemed politically and socially useful.58 While colleges and universities have always in part been businesses that have competed to attract students and cater to their individual needs, they also have traditionally seen themselves as enterprises with quasi-parental authority and the responsibility to define appropriate educational goals with regard to academic content, social behavior, and moral development. The balance between these competing inst.i.tutional functions has noticeably s.h.i.+fted in recent decades.

Measuring Learning in Higher Education.

Organizational inertia, the a.s.sumption that students are meeting the academic goals espoused in mission statements, and a lack of external pressure to demonstrate learning have all contributed to a failure systematically to measure and evaluate studentsa gains in higher education. The tide is s.h.i.+fting, however, as concerns about turning out productive workers and not wasting resources become paramount in an era of globalization and fiscal constraints. Learning in higher education was recently placed in the national spotlight by a report of the Secretary of Educationas Commission on the Future of Higher Education ent.i.tled A Test of Leaders.h.i.+p. Reminiscent of the critique in A Nation at Risk of elementary and secondary education in the 1980s, A Test of Leaders.h.i.+p placed the responsibility for the nationas compet.i.tiveness in the global economy on the doorsteps of educational inst.i.tutions. With respect to student performance, the commission noted that athe quality of student learning at U.S. colleges and universities is inadequate, and in some cases, declining.a59 Supporting this claim, it reported on sobering statistics from the National a.s.sessment of Adult Literacy. Specifically, from 1992 to 2003 the percentage of college graduates judged proficient by various literacy measures was relatively low, and by two of those three indicators competency declined (prose, 40 to 31 percent; doc.u.ment, 37 to 25 percent; and quant.i.tative, 31 percent at both time points).60 While a debate has since ensued on the definition of proficiency, the commission nevertheless used the results from this study to urge improvement and increased accountability to monitor student learning in higher education.61 The commission also identified a lack of transparency and accountability with respect to inst.i.tutional performance in general and student learning in particular. aDespite increased attention to student learning results by colleges and universities and accreditation agencies, parents and students have no solid evidence, comparable across inst.i.tutions, of how much students learn in colleges or whether they learn more at one college than another,a its report noted. aSimilarly, policymakers need more comprehensive data to help them decide whether the national investment in higher education is paying off and how taxpayer dollars could be used more effectively.a62 From our standpoint, the evidence of student and organizational culturesa inattention to learning and high levels of societal investment makes discussion of higher educationas accountability both largely inevitable and in certain respects warranted. We are deeply skeptical, however, that externally imposed accountability systems will yield desirable changes in educational practicesa”for reasons that we will discuss in the concluding chapter of this book. More immediately, as social scientists we raise two additional core reservations regarding such endeavors. First, it is not clear that the state of knowledge in the field is adequate to the task. Specifically, as we will discuss in detail below, there is only a very limited tradition of social scientific efforts to measure learning rigorously across individuals and inst.i.tutions in higher education, and even less of a scholarly research corpus that attempts to identify individual and inst.i.tutional factors a.s.sociated with improved postsecondary student performance. Given these limitations, it is doubtful that the implementation of an externally imposed accountability system would yield outcomes that would be either meaningful or productive.

Second, while the question of how much students in particular colleges are learninga”or, whether they are learning anything a.s.sociated with academic knowledge at alla”is worth pondering at a societal and regulatory level, in terms of applied social science research designed to improve inst.i.tutional policy and practice, it is the wrong question. Rather than asking whether students are learning anything at college and designing accountability regimes to address the absence of measurable gains at underperforming schools, we need first to identify the specific factors a.s.sociated with variation in student learning across and within inst.i.tutions. Such an empirical a.n.a.lysis requires that large numbers of students in multiple inst.i.tutions are tracked over time as they progress through college. Longitudinal measurement of test score performance, coursework, inst.i.tutional characteristics, social background, and college experience is needed to build our knowledge of the processes and mechanisms a.s.sociated with student learning. Datasets of this character in elementary and secondary education have existed for several decades and have enabled researchers to address these questions adequately.

To date, however, longitudinal datasets with these features have not existed in the field of U.S. higher education. As social scientists we were tired of waiting on the U.S. government to muster the political will to overcome inst.i.tutional resistance and begin collecting longitudinal data tracking student learning in higher education over time. Our frustration was so great that when an opportunity arose to join a group of innovative pract.i.tioners to collect independent data on this topic, we began building our own dataset that could for the first time systematically identify the relevant individual and inst.i.tutional factors a.s.sociated with student learning in higher education. Our research addresses the critical absence of similar studies by tracking students through a large and representative sample of higher-education inst.i.tutions with objective measures of their learning as well as of their coursework, social background, and experience of life on todayas college campuses.

The Determinants of College Learning Dataset.

Our research was made possible by a collaborative partners.h.i.+p with the Council for Aid to Education,63 an organization that brought together leading national psychometricians at the end of the twentieth century to develop a state-of-the-art a.s.sessment instrument to measure undergraduate learning, and twenty-four four-year colleges and universities that granted us access to students who were scheduled to take the Collegiate Learning a.s.sessment (CLA) in their first semester (Fall 2005) and at the end of their soph.o.m.ore year (Spring 2007).64 Students who consented to partic.i.p.ate in our study not only completed the CLA at multiple points in their college careers, but also responded to surveys on their social and educational backgrounds and experiences. In addition, we collected course transcript data and inst.i.tutional information on high schools and colleges that the students attended. The research in this book is based on longitudinal data of 2,322 students enrolled across a diverse range of campuses. Colleges in our sample include schools of varying size, selectivity, and missions. The sample includes liberal arts colleges and large research inst.i.tutions, as well as a number of historically black colleges and universities (HBCUs) and Hispanic-serving inst.i.tutions (HSIs). The schools are dispersed nationally across all four regions of the country. We refer to this multifaceted data as the Determinants of College Learning (DCL) dataset.

Logistical and resource constraints required our reliance on partic.i.p.ating inst.i.tutions to implement appropriate random sampling and retention strategies. We thoroughly investigated the extent to which students in our sample were indeed representative of students from these inst.i.tutions as well as of U.S. higher education more broadly (this bookas methodological appendix provides detailed comparisons with data from the Integrated Postsecondary Education Data System and the Beginning Postsecondary Students Longitudinal Study). On most measures, students in the DCL dataset appeared reasonably representative of traditional-age undergraduates in four-year inst.i.tutions, and the colleges and universities they attended resembled four-year inst.i.tutions nationwide. The DCL studentsa racial, ethnic, and family backgrounds as well as their English-language backgrounds and high school grades also tracked well with national statistics. For example, 65 percent of DCL students had college-educated parents, as compared to 59 percent of a national sample of traditional-age students in four-year inst.i.tutions. Half of students in both the DCL and national samples earned A or Aa” in high school. Moreover, the four-year colleges and universities in the DCL sample have a proportion of white students and a level of academic preparation similar to those of four-year inst.i.tutions in general. Indeed, the 25th and 75th SAT percentiles of entering students at the DCL inst.i.tutions and four-year inst.i.tutions nationwide are virtually identical. As a likely result of the voluntary partic.i.p.ation required in our study, however, our sample did have fewer men, as well as fewer students of lower scholastic ability as measured by standardized testsa” for example, studentsa combined scores at the 25th percentile of the SAT were lower in our sample than at DCL inst.i.tutions or four-year inst.i.tutions nationwide. Consequently, we believe that any biases introduced into our a.n.a.lysis by the sampling procedures used are likely to be in the direction of leading us toward overestimating studentsa positive educational experiences and inst.i.tutional success.

The Collegiate Learning a.s.sessment.

The Collegiate Learning a.s.sessment (CLA) consists of three open-ended, as opposed to multiple-choice, a.s.sessment components: a performance task and two a.n.a.lytical writing tasks (i.e., to make an argument and to break an argument). According to its developers, the CLA was designed to a.s.sess acore outcomes espoused by all of higher educationa”critical thinking, a.n.a.lytical reasoning, problem solving and writing.a65 These general skills are athe broad competencies that are mentioned in college and university mission statements.a66 Rather than testing for specific content knowledge gained in particular courses or majors, the intent was to a.s.sess athe collective and c.u.mulative result of what takes place or does not take place over the four to six years of undergraduate education in and out of the cla.s.sroom.a67 The developers of the CLA argue that it a.s.sesses abilities distinct from those measured in general education tests such as the Scholastic Apt.i.tude Test (SAT) and the American College Testing (ACT) program. aConsequently, an SAT prep course would not help a student on the CLA and instruction aimed at improving CLA scores is unlikely to have much impact on SAT or ACT scores.a68 While the CLA as a whole is considered by some as state-of-the-art, the performance task component is its most well-developed and sophisticated part. Our a.n.a.lysis, which follows in this book, will focus on that component. The performance task allows students ninety minutes to respond to a writing prompt that is a.s.sociated with a set of background doc.u.ments. The testing materials, including the doc.u.ments, are accessed through a computer. The Council for Aid to Education has published several examples of representative performance tasks that are worth describing here in detail.

The aDynaTecha performance task asks students to generate a memo advising an employer about the desirability of purchasing a type of airplane that has recently crashed. Students are informed: aYou are the a.s.sistant to Pat Williams, the president of DynaTech, a company that makes precision electronic instruments and navigational equipment. Sally Evans, a member of DynaTechas sales force, recommended that DynaTech buy a small private plane (a SwiftAir 235) that she and other members of the sales force could use to visit customers. Pat was about to approve the purchase when there was an accident involving a SwiftAir 235.a Students are provided with the following set of doc.u.ments for this activity: newspaper articles about the accident, a federal accident report on in-flight breakups in single engine planes, Pat Williamsas e-mail to her a.s.sistant and Sally Evansas e-mail to Pat Williams, charts on SwiftAiras performance characteristics, an article from Amateur Pilot magazine comparing SwiftAir 235 to similar planes, and pictures and descriptions of SwiftAir models 180 and 235. Students are then instructed to aprepare a memo that addresses several questions, including what data support or refute the claim that the type of wing on the SwiftAir 235 leads to more in-flight breakups, what other factors might have contributed to the accident and should be taken into account, and your overall recommendation about whether or not DynaTech should purchase the plane.a69 A second performance task that the Council for Aid to Education has circulated is related to crime reduction. The test instructs students that aJamie Eager is a candidate who is opposing Pat Stone for reelection. Eager critiques the mayoras solution to reducing crime by increasing the number of police officers. Eager proposes the city support a drug education program for

<script>