Johns Hopkins University
Middle States Commission on Higher Education
Self-Study Report for the Accreditation Site Visit
Selected Topic Report:
The Challenge of Improving Undergraduate Education
in a Research Intensive Environment

Introduction

This Middle States Commission on Higher Education (MSCHE) Accreditation site visit comes at a critical point for the Whiting School of Engineering (WSE). In spring 2003, Dr. Ilene Busch-Vishniac stepped down after a successful five years as Dean and was replaced on an interim basis by the former Associate Dean for Academic Affairs. The University plans to appoint a new Dean for the WSE by July 2004. This review, therefore, serves as an opportunity for some of the School's strengths and weaknesses to be identified for the incoming Dean. Used effectively, this self-study and accreditation process can reduce the learning period for the new Dean and offer him or her a sense of direction for the pedagogical mission of the school as reviewed by a team of highly-regarded professionals in the field of higher education. The WSE is anxious to engage in this process as it attempts to break into the next level of elite engineering schools.

Ranked as the thirteenth best undergraduate engineering school in the 2004 U.S. News & World Report survey, the Whiting School is proud of the reputation it has garnered, especially considering the School's relatively small faculty and student body size. With 117 tenured and tenure-track faculty members serving approximately 1,370 full-time and part-time undergraduates, WSE professors face the challenge of providing a first-rate undergraduate experience while also maintaining some of the most productive research agendas in the country. It is this presumed struggle for faculty members' time between research and undergraduate education that spurred the Boyer Commission report in 1995 and the Johns Hopkins President's Commission on Undergraduate Education (CUE) in 2002. With the second largest undergraduate body in the University, it is not surprising that the WSE was well-represented on the Commission — with several faculty members, students and administrators serving as members — and that the School has embraced CUE's findings. The WSE has ensured its students and faculty members that it will not postpone the implementation of CUE's recommendations until a new Dean is named. If a CUE- advocated change will improve learning or college life for WSE undergraduates, then it needs to be adopted as soon as it is feasible to do so.

Full-time undergraduate education in the Whiting School is conducted by our 117 tenured and tenure-track faculty members and by approximately 50 lecturers and adjunct faculty members affiliated with the following 13 majors (with ABET accreditation denoted by an asterisk): Applied Mathematics and Statistics, Biomedical Engineering*, Civil Engineering*, Chemical and Biomolecular Engineering*, Computer Engineering*, Computer Science, Electrical Engineering*, Engineering Mechanics*, Environmental Engineering, General Engineering, Geography, Materials Science and Engineering*, and Mechanical Engineering*. The five largest majors, composing 77% of the student body, are as follows: (1) Biomedical Engineering, 505 students; (2) Computer Science, 163 students; (3) Mechanical Engineering, 119 students, (4) Chemical and Biomolecular Engineering, 108 students; and (5) Electrical Engineering, 103 students. Our percentages of female (25%) and underrepresented ethnic minority (7%) students appear to be typical of a highly selective engineering school. The Whiting School also has a very small program in part-time undergraduate engineering education. Approximately 70 students are spread among three degree programs in Electrical Engineering, Engineering Science and Mechanical Engineering.

Standard 1: Mission, Goals and Objectives

The Whiting School of Engineering has benefited greatly from its mission and vision statements as well as its overarching goal and objectives. These principles serve as the cornerstone of the WSE Strategic Plan (Phase II), which has guided the School since its introduction in 2001 by former Dean Ilene Busch-Vishniac.

Details of the Guiding Principles

The mission, overarching goal and objectives were all ratified by the Whiting School Strategic Planning Committee and announced to the community formally through Phase II of the Strategic Plan. They address both full-time and part-time programs in the Whiting School. To ensure that these statements were publicized and widely known, Dean Busch-Vishniac presented the Strategic Plan to External Departmental Committees as well as the Whiting School's National Advisory Council at the same time that she distributed hard copies to all faculty members, presented the plan at a WSE Faculty Retreat, and published the plan on the Internet. The mission explains whom the School serves and what it intends to accomplish by focusing on research and education — the two core functions of the Whiting School.

The overarching goal of placing each department's graduate ranking among the top 15 in the country in its discipline with at least two departments ranked in the top five is the Whiting School's clearest effort to position itself in an external context. While the School refuses to allow external rankings such as those by U.S. News & World Report to affect policy decisions greatly, we have become increasingly aware of the influence that these metrics have on our ability to recruit the top undergraduates, graduate students and faculty.

The Whiting School's overarching goal and the thirteen objectives stated in Phase II of the Strategic Plan are unquestionably outcomes-based and capable of being evaluated. The Office of the Dean recently conducted a thorough "Assessment of Progress in the Strategic Plan" to measure the advancement already made toward its goal and objectives and how much farther the School must go to meet them. Upon completion of the review, we found the following:

The Whiting School has not yet met its overarching goal. In the 2004 U.S. News & World Report undergraduate rankings in engineering, the Whiting School placed one program in the top five in both (Biomedical Engineering, first) and another program is near that range (Environmental Engineering, eighth). But, while all eight eligible programs are ranked within the top 25 in their discipline, only the two mentioned above earned spots in the top 15.

The Whiting School has made significant progress in nearly all of the 13 objectives. Key findings are as follows: (1) The Whiting School has already met Objective 1 — adding 15-30 full-time faculty members within 10 years — by increasing the ranks by 19 since 1999-2000; (2) the Whiting School has acquired space in or made plans to build six buildings in a very short amount of time [Objective 2]; and (3) the Whiting School has greatly increased its services to undergraduate engineering students through an expanded Engineering Advising Office capable of providing not only advising, but also opportunities for international study [Objective 4].

The assessment was equally successful in identifying areas where the School has yet to make significant progress, most notably, our objective of reviewing the undergraduate engineering curricula to incorporate advances in pedagogy and course content [Objective 4]. Here, the influence and momentum of the Commission of Undergraduate Education (CUE) should be useful. By assigning responsibility for assuring the quality of undergraduate education to a senior-level administrator in the Whiting School and appointing a Director of Undergraduate Studies in each department, the School now has formal channels through which to conduct reviews of departmental curricula. To learn of pedagogical advances, the Whiting School will increase its support for faculty and graduate teaching assistants by making the services provided by the Center for Education Resources (CER) available upon request to all Whiting School instructors.

While our continuous self-assessment is extremely valuable, outside assessment can more readily place a school into the external landscape. At the request of President Brody, the WSE underwent a formal external review in spring 2003 shortly before Dean Busch-Vishniac stepped down from office in order to assist the incoming Dean in surveying the School's strengths and weaknesses and in setting strategic directions for the School. The review also serves as a guide for the Whiting School in its attempt to reach the next level of status as dictated by our peers. To achieve the next level, the external review suggests that the WSE embark on a strategy of focused selective excellence. The Interim Dean is currently working with the WSE leadership, under the guidance of the Provost, to determine what preliminary steps can be taken during his interim tenure to prepare for possible changes to be implemented by a new Dean.

Departmental Mission, Goals and Objectives

The Strategic Planning Committee decided that the School mission, goal and objectives could be broad functional statements applied to all disciplines within the School (e.g., increasing focus on undergraduate admissions and advising) because the individual Whiting School undergraduate programs are expected to provide focused student learning outcome statements for the engineering accreditation process. In 1999, the Whiting School's eight undergraduate engineering programs underwent a review by the Accreditation Board for Engineering and Technology (ABET), using the new ABET criteria (EC 2000). These criteria revolutionized the way engineering schools viewed their offerings by requiring that all programs develop student learning-focused "program educational objectives" and "outcomes" very similar to the goals and objectives requirements for MSCHE accreditation. Six of our nine departments offer ABET-accredited programs, and therefore maintain active statements that are equivalent to the MSCHE'S departmental mission, goals, and objectives. Of the remaining three departments, one — Applied Mathematics and Statistics — has now developed focused student-oriented goals and objectives. The other two departments — Computer Science and Geography and Environmental Engineering — will seek accreditation during the next school-wide review (in fall 2005) and will develop goals and objectives during the 2003-04 academic year.

The Part-Time Programs in Engineering and Applied Science (PTE) include three undergraduate majors (in Electrical Engineering, Engineering Science and Mechanical Engineering) with a total enrollment of approximately 70 students. This is a two part undergraduate degree, with students first completing a 70-credit engineering foundation program comprised of core mathematics, science, liberal arts and engineering science at any accredited educational institution. In the second part of the program, students take a minimum of 60 credit hours in courses that lead to mastery in the area of the major.

Goals are set for the PTE undergraduate program as a whole. The two year program has experienced diminishing enrollments and is currently under scrutiny. We seek tighter integration between the part-time and full-time undergraduate programs in order for all WSE offerings to be of similar quality. In the field of engineering, the hallmark of a high-quality program is ABET accreditation. None of the part-time majors currently hold this distinction. Two issues which may preclude this from occurring are: (1) the fact that the majority of PTE instructors are adjunct faculty members, and (2) insufficient lab requirements vis-…-vis ABET standards.

Mission, Goals and Objectives Assessment

Most of the Whiting School's undergraduate education programs are reviewed by three different boards at different intervals. At each of these steps, the programmatic or departmental mission, goals and objectives are assessed. These three boards and their cycle lengths are: (a) the Homewood Academic Council, every four years; (b) ABET, every six years; and (c) MSCHE, every ten years. A brief description of each process follows.

The Homewood Academic Council reviews provide an opportunity for each department to assess its mission, goals and objectives for both undergraduate and graduate programs. The Academic Council examines both education and research. ABET requires each program to have assessed its mission, goals and objectives with input from a range of stakeholders. In addition, ABET reviews require departments to place their mission, goals and objectives in alignment with those of the Whiting School. As recommended by CUE, we will now require all undergraduate programs not accredited by ABET to undergo a self-study every six years on the WSE/ABET timetable. These reviews will include departmental mission, goals and objectives. Lastly, the MSCHE accreditation cycle provides the opportunity for us to re-examine the mission, goals and objectives for the School in the context of a broader University review.

A New Mission and Vision of Undergraduate Education at Johns Hopkins

The Whiting School welcomes the new mission and vision statement for undergraduate education at Johns Hopkins which resulted from the CUE process. It is very much in alignment with our existing approach to educational programs. Critical thinking skills and creativity are at the heart of engineering design. In addition, we take great pride in our preparation of WSE graduates to pursue either advanced degrees or employment as practicing engineers. With regard to the eight specific dimensions, the Whiting School already strives to meet five of them through our ABET accreditation efforts.

The three dimensions that we do not pursue directly are addressed in other ways. For example, the mission statement suggests that our students be familiar with the subject content and approaches to understanding the world carried out by humanists, natural scientists and engineers, and social scientists (dimension 2). Naturally, we require our students to be well-versed in engineering and the natural sciences subject matter, but we also require that they undergo a brief but coherent review of one or more areas in the humanities or the social sciences. Whether they should pursue studies in both of these fields can be considered in future curriculum reviews. While the Whiting School does not specifically require familiarity with the origins and diversity of U.S. and foreign cultures to foster civility and an appreciation for diversity (dimension 3), our students do enroll in humanities and social sciences courses, as mentioned above, and we are making significant progress in expanding our study abroad program for engineers. Lastly, even though the Whiting School does not expressly provide opportunities for all students to develop the habits of community service and civic engagement (dimension 8), two of our programs select senior design projects to serve the needs of the local community and/or the disabled community, and many of our students tutor middle school and high school students through programs sponsored by Homewood Student Affairs and the Hopkins Organization for Minorities in Engineering and Science. Overall, adopting the new mission and vision of undergraduate education at Johns Hopkins will have a strong, positive effect on the Whiting School; it will reinforce our pre-existing emphases and encourage us to consider increasing our curricular and intellectual diversity.

Standard 8: Student Admissions

The Whiting School has experienced vigorous growth and improvement in undergraduate admissions. Since fall 1998, the Whiting School freshman class has increased in size by 14% (F'98-326; F'03-372), without compromising quality, as the mean SAT score for the WSE incoming freshmen has improved from 1368 to 1410 over the same period. The Whiting School recognizes, though, that size and mean SAT score are not the only measures of an incoming class. We are dedicated to increasing the ethnic and gender diversity of our student body, as well as its intellectual diversity as measured by the number of students who show interest in each major. In recent years, for example, nearly half of the incoming class has been composed of Biomedical Engineering majors, risking the health of all programs in the School. Through increased communication and collaboration with the Office of Undergraduate Admissions and the Krieger School of Arts and Sciences, we are making significant progress towards our goals.

WSE Involvement in Admissions

The Homewood Office of Undergraduate Admissions manages the Whiting School's admissions efforts, and the Homewood Office of Student Financial Services administers our undergraduate financial aid. To avoid any potential disconnect between the Whiting School and the process of admissions, Phase II of Strategic Plan called for the WSE's increased involvement in undergraduate admissions. The Whiting School now participates in the newly formed Faculty Committee on Admissions and in the Enrollment Management Team. The former seeks to solicit faculty and School input to guide admissions decisions on topics ranging from marketing ventures to lists of desired admitted student characteristics, while the latter both sets admissions policy and provides the schools with frequent updates on admissions throughout the recruitment and enrollment seasons.

One of the Whiting School's most critical admissions initiatives was implemented with the assistance of these two committees. As the CUE Final Report proposes, the Whiting School seeks to match student enrollments more closely with academic resources. As mentioned above, faculty members in the Biomedical Engineering program had been teaching and advising nearly half of the undergraduate students in the Whiting School of Engineering. While the School strongly supports its flagship program, its success was jeopardizing the intellectual diversity of the engineering community. Two measures were taken. First, the School sought to place a limit on the number of students accepted to the Biomedical Engineering program by requiring prospective students to apply specifically to BME, rather than generally to the Homewood campus. Second, after noticing that students were taking advantage of the no-obstacle-to-transfer policy at Hopkins, we sought to crack down on the practice of students applying with a stated major other than BME, and then transferring into the BME program. Starting with the fall 2003 incoming class, only students admitted to BME may enroll in that major. This is the first time in Hopkins history that a restriction was placed on transfer into a Homewood program. The new policy was successful in dropping the incoming BME class from 160 (fall 2002) to 136 (fall 2003), reversing a very strong upward trend in freshman enrollees.

What softened the effect of the change in admission to the BME program was the introduction of three "bioengineering" options in the Whiting School. The Departments of Chemical and Biomolecular Engineering (the biomolecular option), Materials Science and Engineering (the biomaterials option), and Mechanical Engineering (the biomechanics option) all enrolled students who may have otherwise selected BME as their major. The presence of these programs has helped to spread enrollments more evenly. We now turn our attention to declining enrollments in Computer Science (69 freshmen in fall 2000, 24 in fall 2003) and Civil Engineering (16 in fall 2000, 4 in fall 2003). Preliminary analysis suggests that the former is part of a nationwide response to current economic conditions. The latter might be a part of a trend shared by other similar COFHE engineering schools (e.g., Duke, Columbia, Rice, Penn), but more information needs to be gathered before this can be verified.

Policy Information

The Whiting School relies upon the Office of Undergraduate Admissions to make admissions policies and criteria available to prospective students. Admissions also distributes a two page sheet with expected learning outcomes for students in every WSE major. In addition, every major has a specific advising manual that outlines all major requirements. These manuals are published on the Internet for use by current as well as prospective students. Information regarding placement and diagnostic testing is available in the Arts and Sciences and Engineering catalog as well as on the WSE Academic Affairs website. The Office of Student Financial Services provides scholarship and loan information. The Undergraduate Academic Manual (both in-print and online) contains policies regarding transfer credit. In the Whiting School, the Office of Academic Affairs determines whether a transfer course will be accepted for Hopkins credit, while the student's department decides whether a transfer course meets a specific program requirement.

Admissions Planning Efforts

The Whiting School is interested in whether there are any admissions variables that may be particularly strong predictors of the academic success of its engineering students (e.g., SAT, HS GPA, etc.). When most analyses of this sort are conducted at Johns Hopkins, data for engineering students are not disaggregated from those for arts and sciences students. In the past, the Whiting School has attempted to conduct its own analyses. Some of these attempts are statistics-based and others are more anecdotal — performed by professional academic advisors who have extensive contact with engineering students. While the statistical analyses seem to indicate that performance on math standardized tests (i.e., SAT Math, SAT II Math) are highly correlated with success in the Whiting School, these studies are crude and do not include all potential variables, making the data unreliable and inconclusive. The anecdotal data can be very helpful, but our professional advisors spend a majority of their time with students in academic difficulty, rather than those who have excelled in the classroom and laboratory. With the assistance of the Johns Hopkins Office of Institutional Research, the Whiting School expects to conduct a more formal analysis on all possible variables predictive of success during the 2003-04 academic year.

Diversity Admissions Efforts

Engineering has traditionally been a white (and more recently, Asian) male endeavor. Racial and gender diversity have been difficult to establish. In recent years, the Whiting School entering undergraduate class has been approximately 25% female, slightly higher than the national norm, and 7% minority, which is lower than the national norm. At the graduate level, the WSE is among the highest in percentage of women in our programs, but the percentage of underrepresented ethnic minorities is below the national norm.

The norms, however, are still unacceptable to us. To make engineering a more diverse field, at Johns Hopkins we are taking and will take several measures. We will continue to support two nascent diversity-focused initiatives. First, the Baltimore Scholars Program encourages talented city high school students to apply to JHU by waiving tuition. Second, The Whiting School has created a policy of reserving a percentage of overall funding for financial aid and scholarships for transfer students. The School has found that recruiting applicants from community colleges and other postsecondary institutions can be an effective tool for attracting minority students as well as students from economically and socially disadvantaged backgrounds. The most critical step, however, will be in line with CUE Recommendation 21. We will work with the Office of Undergraduate Admissions to prepare a detailed plan for enrolling ethnic minority and female students. This shall include action steps, funding requirements, and an aggressive timetable.

Part-Time Engineering Admissions

PTE conducts its own recruiting at both the undergraduate and graduate level, and both student bodies are local. With decreasing enrollments in the PTE undergraduate program, admissions has become a focal point of the program planning efforts. PTE has increased its efforts to widen the pipeline from local community colleges to the part-time undergraduate program. Descriptions of all undergraduate courses (with outcomes) are available to prospective students in the PTE catalog. Admissions requirements and transfer credit policies also are published in the PTE catalog.

Standard 9: Student Support Services

Many of the student support services provided to engineering students are shared with arts and sciences students and formally administered through Homewood Student Affairs and Homewood Enrollment and Academic Services. Because these units organizationally are responsibilities of the Vice Dean for Undergraduate Education, they are addressed in the Krieger School's Self-Study Report.

The Whiting School offers critical student support services to its undergraduates in the form of academic and career advising. Both faculty members and professional advisors serve these functions. WSE faculty advisors are, first and foremost, expected to provide degree requirement information and assistance in course selection. There are other academic advising tasks associated with being a faculty advisor, but these two constitute the primary expectations. If a student intends to pursue either employment or a graduate degree within his or her major field, the faculty advisor is then expected to provide guidance or referral to colleagues in these areas as well, upon request.

The Engineering Advising Office handles tasks that can be applied to all undergraduate students within the Whiting School, such as: (a) summer registration for freshmen, (b) orientation for incoming freshmen, (c) semi-annual review of academic progress (i.e., Dean's List and academic probation), (d) graduation clearance for bachelor's candidates, (e) guidance on major selection, and (f) evaluation of requests for exceptions to academic policies outside of the major. Though faculty and professional advisors often work very closely together on individual student cases, one of our objectives is to make the general division of responsibility clear to students. We stress the different roles of the advisors during freshmen orientation and on the Academic Affairs website. In our spring 2003 Advising Survey, we asked students whether they expected a faculty advisor or the Engineering Advising Office to perform some of the tasks listed above, and an overwhelming majority of respondents answered in a way that would indicate that we have successfully conveyed the functional differences of the two types of advisors. For example, 90% of all students expected their faculty advisor to provide degree requirement information, and 93% expected him or her to provide course selection assistance. This is consistent with our system's intent.

Advising Policies

The Engineering Advising Office serves as the repository for WSE undergraduate academic records, and it has strict regulations regarding the maintenance and security of these documents. All records for active undergraduates, including complaint and grievance files, are stored in the WSE Office of Academic Affairs. No students are permitted to enter these files. Records of alumni and un-enrolled undergraduates are stored for two years in a locked room near the Office of Academic Affairs. In the third year, these records are shipped to the archives of the Sheridan Libraries. All documents that identify students in any way are shredded prior to discarding. The Office of Academic Affairs posts all of its academic policies on its website, and all WSE undergraduate programs have published advising manuals which lead a student from point-of-entry to graduation — all but one of these are on the Internet. Policies regarding complaints and grievances can be found in the Homewood Undergraduate Academic Manual, and information regarding the release of student information is available in the Arts and Sciences and Engineering catalog.

Advising Assessment

The Whiting School surveys all of its undergraduate students and alumni regarding advising through WSE Exit Surveys and Alumni Surveys. WSE departments conduct the Exit Surveys which address advising provided by the department-assigned faculty advisor. Departments are expected to analyze the data and employ the findings in a process of continuous improvement required by ABET and MSCHE. The Dean's Office, therefore, does not receive a regular report of these results.

The Alumni Survey is distributed by the WSE Office of Academic Affairs. It addresses both faculty and professional advising and is mailed to graduates two and five years post-graduation. These data show that students are generally satisfied with both faculty advising and professional advising, though, consistent with the findings at peer institutions, the reviews are not overwhelmingly positive. In addition, satisfaction with these services appears to have increased over time. The results of the 2002-03 Alumni Survey tells us that for the Class of 1997, 62% of all respondents were either neutral or satisfied with faculty advising, leaving 38% dissatisfied. For the Class of 2000, the level of dissatisfaction dropped to 25%. Dissatisfaction ratings for the Engineering Advising Office also dropped from 32% for the Class of 1997 to 19% for the Class of 2000. Despite the methodological problems of trying to compare data for alumni who graduated two years ago with those of alumni who graduated five years ago, this is our only opportunity to examine changes over time.

Ultimately, however, these two instruments attempt to address a broad range of topics and cannot cover advising in depth. To gain a richer picture of student satisfaction with advising, and in order for effective service as a faculty advisor to be considered in salary and promotion decisions, an instrument dedicated to advising is necessary. In addition, there must be a way for individual faculty member advising performance to be evaluated. We have addressed the first issue with an Advising Survey distributed for the first time in spring 2003. As stated above, this survey solidified some assumptions about student expectations of faculty advising, but it also opened our eyes to some possible shortcomings. For example, while 70% of all students stated that they either "strongly agree" or "agree" that their advisor did "a good job of advising, overall," the percentage of those who strongly agreed was only 26%. In other words, there is satisfaction with faculty advising among most students, but not at the level of strength that we hope to elicit. In addition, only 59% either agreed or strongly agreed with the following statement: "There is someone at Hopkins (faculty or staff) who is concerned about my progress." This serves to confirm the need for more of a "sense of community" at Johns Hopkins, as CUE seeks to produce for undergraduates.

We are unsure as to how we can effectively include advising assessment results in salary review decisions as proposed by CUE. Our current advising survey does not ask respondents to identify faculty advisors by name. Even if it were to do so, many advisors would go unevaluated (thoroughly or entirely) unless our response rate improves dramatically from the 25% that we saw on the first distribution. This sample size may be sufficient to provide us with a sense of student satisfaction with WSE advising, but it does not permit us to use these survey results as part of the reward system — we need robust data for all faculty members for this to occur.

One weakness we have found in the advising assessment system is our tendency to forget to "close the loop." The Associate Dean for Academic Affairs has access to the Alumni Survey and Advising Survey results, but does not receive the Exit Survey data from the departments. In all cases, consistent with the practices at a highly decentralized university, the Associate Dean trusts that the departments are using these data to improve academic programs. A discussion, however, of these data and changes departments have made in response to findings could benefit the Engineering Advising Office as well as the other departments. We will attempt to elicit this discussion by both adding these results to the annual departmental report submitted to the Dean and by opening the lines of communication between the newly appointed senior-level administrator responsible for undergraduate education with the new Directors of Undergraduate Studies, consistent with CUE Recommendations 1, 2, and 3.

Career Advising

When WSE students seek jobs upon graduation, they are encouraged to request assistance from the Homewood Career Center. If they intend to pursue a position within their major field, they can additionally solicit advice from the faculty advisor. In the 2003 Advising Survey, only 62% of respondents who claimed to have requested advice from their faculty advisor reported receiving "good career advice." In the 2002-03 Alumni Survey, 35% of those who used the Career Center claimed that the office was "not helpful," and 36% claimed that it was only "somewhat helpful." These results are problematic to us. We believe there is confusion among students as to whom they should ask for career advice. Students may be expecting in-depth knowledge of engineering professions from Career Services and seeking assistance in acquiring non-major related positions from the faculty advisor. If this is the case, we must better explain from whom students should seek advice, and we shall increase communication between faculty advisors and the Career Center. Faculty advisors must know the sort of assistance provided by the Career Center, and Career Center's staff must be better informed about engineering careers. We will accomplish this by having Career Center staff meet with the Directors of Undergraduate Studies.

The Whiting School believes that internship opportunities serve a critical function in career services for its students. The WSE Office of Industrial Initiatives has a new website dedicated to internships with approximately 300 vetted offerings specifically for engineering students. Awareness of the site is increasing, but it is not yet pervasive in the School — the spring 2003 Advising Survey indicated that only half of our students "know where to go for help in finding an internship." We will continue to increase awareness of this site through promotional publications, workshops and direct communication with students. We also suspect that our students are unsure as to whether they should seek internship assistance from the Homewood Office of Academic Advising (which serves both engineering and arts and sciences students) or from the Whiting School. As a result, the two parties will meet to discuss possible centralization of services and ways to publicize the type of assistance available.

Advising Training

As CUE Recommendation 13 implies, we cannot expect faculty members to advise our students effectively without any training. In the Whiting School, each program's advising manual provides program-specific requirements. We expect individual departments to train faculty members on University-wide requirements and policies. We cannot be certain, however, that new faculty members in all departments are receiving the same information. As a central location for engineering advising, the Engineering Advising Office will create an Advising Handbook for WSE Faculty Members. This publication will address topics such as the function of a faculty member's signature, the implications of assigning an "incomplete" grade, and other policies that should be normalized across all WSE departments.

Part-Time Engineering Student Services

Whenever possible, we attempt to provide student services to PTE undergraduates in a convenient location. The Dorsey Center site now houses PTE undergraduate enrollment and registration services and computer laboratories. In some cases, it is neither more efficient nor more effective for PTE to offer its own services, and students are directed to use Homewood campus facilities such as the Career Center, the Sheridan Libraries, and the Office of Student Financial Services. Academic advising for PTE undergraduates is primarily the function of the program chair and the vice chair. Students are encouraged to discuss their academic programs with these faculty members.

All PTE services are evaluated in an annual fall survey to graduate and undergraduate students. While there appears to be general satisfaction with student services in PTE, the administration has not distinguished between graduate and undergraduate students in the past, and thus has not been able to directly identify areas of success or improvement for undergraduates. In the future, PTE plans to make this distinction.

Standard 10: Faculty

The faculty of the Whiting School consists of tenured and tenure-track faculty members, part-time and full-time lecturers, and graduate teaching assistants. The WSE employs 117 tenured and tenure-track faculty members (all holding doctorate degrees) who teach approximately 70% of the courses in the WSE. Tenured and tenure-track faculty members in the Whiting School have three main functions pertaining to undergraduate education: (1) advising undergraduates, (2) teaching undergraduates, and (3) overseeing undergraduate research. Faculty advising performance is assessed in the section addressing Standard 9 (Student Support Services). This section, therefore, will only address the last two functions listed — teaching undergraduates and overseeing undergraduate research — for the tenured and tenure-track faculty. Approximately 50 lecturers teach the remaining 30% of Whiting School courses. The WSE also employs roughly 110 graduate teaching assistants, who support the professors rather than lead classes. With very few exceptions, lecturers and TAs have no other responsibilities to undergraduates other than those related to instruction. In the text below, the term "instructor" is usually employed when referring to professors, lecturers and graduate teaching assistants en masse, with exceptions noted.

Assessment of Teaching

The Whiting School uses one primary tool to assess student satisfaction with teaching by faculty members, lecturers and graduate teaching assistants — the end-of-semester student evaluation system dubbed "MERLIN." Whiting School undergraduates are also questioned regarding satisfaction with instruction on two of the Homewood-wide surveys — the College Student Experiences Questionnaire (CSEQ) and the COFHE Senior Survey — and departmental Exit Surveys. Whiting School alumni are polled on instruction in the annual Alumni Survey. Combining all of the assessments, it becomes clear that students are generally satisfied with teaching in the Whiting School. We have no deep concerns regarding current WSE instruction as a whole, and all individual departments receive satisfactory ratings. Student opinion of specific instructors varies, of course. Where we can perhaps improve undergraduate education, however, is by strengthening student-faculty relations — what we view as the cornerstone of the "sense of community" that CUE seeks to elicit. Lastly, an important question is how the Whiting School can improve its assessment of student satisfaction with instruction.

In the MERLIN questionnaire, students are asked three questions regarding an instructor's behavior in class. In all, they are asked to determine the extent to which they agree with the following statements: (1) "the professor gave clear, well- structured presentations"; (2) "the professor seemed interested that the students learn the material"; and (3) "the professor used class time effectively." For all MERLIN questions, a scale of 1-5 is used, with the following sentiments representing each number: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. In the spring 2003 survey, no department's average rating on any of these three questions was below 3.7, and 73% of all department mean ratings were above 4.0. This leads us to believe that, on average, students are satisfied with these critical behavioral traits for instructors.

On the WSE Alumni Survey, we inquire about student preparation in certain engineering-oriented subject areas such as "engineering design," "laboratory/instrumentation" and "modeling," but we ask only one direct question regarding instruction on the Alumni Survey. For that question, instruction is rated very highly. In the 2002-03 distribution, 77% of the Class of 1997 and 77% of the Class of 2000 either agreed or strongly agreed that they "received excellent instruction at JHU." The majority of those who did not agree, in both cases, responded that they were "neutral" and thus did not disagree. While this statement includes instructors from other schools at JHU (primarily the Schools of Arts and Sciences and Medicine), our students take an overwhelming majority of their classes within the School of Engineering and are presumed to have been rating WSE instruction.

As CUE helped us to realize, the sense of community at Hopkins is greatly dependent upon strong student-faculty relations. There is more to this relationship than simply the instruction provided. On the spring 2002 CSEQ survey, WSE undergraduates were asked several questions regarding student-faculty relations. Specifically, students were asked to offer their level of agreement to the following statements: "Faculty members at Johns Hopkins are concerned about undergraduates' academic progress," and "Faculty members have been responsive to my needs." Only 11% of respondents strongly agreed with the former statement, and 16% strongly agreed with the latter. It did not surprise us, therefore, that only 9% strongly agreed that that they "felt a sense of community at Hopkins." This only reinforces the need for the JHU schools, including the WSE, to implement CUE to the fullest extent possible.

Although overall satisfaction with instruction from professors and adjunct lecturers appears to be high (as demonstrated by the MERLIN results), we suspect that satisfaction with Teaching Assistant performance is another area where we can improve. On the MERLIN questionnaire, the departmental range of scores on questions regarding Teaching Assistants was much wider. For example, when asked whether the TA was "deserving of a teaching award," approximately half of the departmental means were in the 3.0-3.9 range, and the remaining means were split between under 3.0 and over 3.9. On the spring 2002 CSEQ distribution, 34% of respondents expressed dissatisfaction with graduate teaching assistants. This may not be a grave concern, but certainly one that we intend to address in the near future. We suspect that our reliance on foreign graduate students who may have some language deficiencies is a contributing factor.

Assessing the Evaluation Tool

As the primary evaluation tool for instruction on the Homewood campus, the MERLIN system plays a critical role in undergraduate education. In fall 2002, the Dean for Enrollment and Academic Services convened a committee to examine the effectiveness of the MERLIN system. The WSE is represented on this committee by its Assistant Dean for Academic Programs. With feedback from students and faculty members, the committee quickly determined that the evaluation system, including the instrument, is likely to be flawed. Low response rates, poorly-worded questions, and an absence of belief among students that the results are used to improve courses all have very strong interaction effects with one another. In spring 2003, the committee hired a course evaluation consultant to assess every aspect of our process: our current instrument, analysis system, survey distribution process (to students), and results distribution process (to faculty members). The consultant met with students, faculty members, Department Chairs, high-ranking administrators, and the Homewood Academic Council. The final review confirmed what the committee had suspected; the system and instrument are both flawed. The data are not collected in a uniform manner, as the online distribution process occurs after the paper survey distribution process, possibly skewing the results. The instrument has a number of survey design errors, and the results are not subjected to any systematic analysis. Results are simply posted online, and faculty members and Department Chairs have no context within which to place these numbers. Courses need to be compared with like courses based on size, level and other variables.

The consultant provided recommendations regarding all aspects of the system. The Whiting School has already begun implementing recommendations on the distribution process, but perhaps the two most important aspects of the system — the instrument itself and the way results are reported to faculty members — are currently under review. The committee has requested additional assistance from the consultant in creating a new survey and hopes to present a revised instrument to the Academic Council in spring 2004. A draft is now under review. To this point, the committee has focused mainly on the instrument and has not yet addressed the reporting mechanism. We expect this to be discussed in early spring 2004.

Undergraduate Research and Faculty

At Johns Hopkins, the first American research university, we expect our faculty to expand the frontiers of knowledge through both basic and applied research. Consistent with the mission of the University, the Whiting School also expects its faculty to perform ground-breaking research. This does not mean, however, that our undergraduate students suffer when faculty members are in the laboratory. At the Whiting School, we do not view research and undergraduate education as a zero sum equation; to the contrary, we contend that a symbiotic relationship exists between the two. While it is certainly true that our research informs our teaching, and vice versa, undergraduate research represents the strongest and clearest link between research and undergraduate education.

Few other engineering schools offer an equivalent level of breadth and volume (per student) of research opportunities for its undergraduates. In the spring 2003 Alumni Survey, 80% of the Class of 1997 respondents and 74% of the Class of 2000 respondents claimed to have conducted undergraduate research while at Johns Hopkins. Nearly half of both classes reported three or more semesters of undergraduate research. In the spring 2002 CSEQ survey, 88% of all students who reported having conducted undergraduate research were either "satisfied" or "very satisfied" with the experience.

Faculty Reward for Teaching

At the conclusion of every academic year, the Whiting School recognizes ten instructors who have demonstrated teaching excellence during the two previous semesters. Five of these awards are dedicated to graduate teaching assistants, while the other five are reserved for faculty members and lecturers. In addition, half of these come with a $1,000 gift. In most cases, it is undergraduate students who select the recipients. While we believe that teaching awards are an appropriate way to recognize outstanding achievement in the classroom and we are proud to name ten recipients each year, we also understand that: (1) there are more than ten outstanding instructors in the Whiting School; and (2) high-achieving faculty members deserve more than one time recognition — they deserve regular compensation for superior classroom performance. Spurred by CUE, which encourages the JHU schools to recognize excellence in teaching "through rewards as well as awards," the Whiting School is seeking to make teaching a more critical part of its compensation model.

The granting of tenure represents perhaps the greatest incentive for any faculty member. As the CUE Final Report states, the Homewood Academic Council (which serves the Schools of Arts and Sciences and Engineering) bears responsibility for determining the proper emphasis on teaching within tenure decisions. At research-based universities, teaching is usually outweighed by research in these types of decisions, and as the first American research institution, Johns Hopkins is no exception to this rule. Nevertheless, the Whiting School willingly assists the Council by providing information on individual faculty member teaching, mentoring, advising and undergraduate research that is used to evaluate candidates for promotion and by acting as advocates for faculty members who teach well and engage with undergraduates constructively. Unlike tenure and promotion decisions, salary decisions are made annually and throughout an instructor's entire career at JHU, and thus may be more effective in rewarding/ensuring consistent classroom performance. Currently, teaching is factored into merit raise decisions in a number of different ways. First, faculty members are expected to provide information on the courses they taught during the previous calendar year. Second, WSE Department Chairs provide input to the Dean on each of their faculty members' teaching performance. This is likely composed of student feedback via exit surveys, MERLIN system results, and other sources. Lastly, the Dean is given a summary of MERLIN results for all faculty members before the merit raises are instituted. Although this appears to be an ample amount of information for teaching to be a strong input factor, we know that the MERLIN system data, as discussed above, are not robust. It is thus with extreme caution that much of this input is used to help determine faculty raises. The Whiting School hopes to make teaching performance a standardized input on every instructor's merit raise decision, but it cannot do so until the evaluation instrument is known to produce reliable and valid data.

Instructor Training

Incoming tenured and tenure-track faculty members are required to attend WSE New Faculty Orientation. These early fall sessions address a faculty member's three primary responsibilities in the School of Engineering: research, teaching and advising. At these events, faculty members are engaged in a broad discussion of research policies and a more focused discussion of grants and contracts. In terms of teaching, the orientation does not explicitly dictate to faculty members how they are expected to instruct a class. Rather, they are provided with a list of resources to contact for assistance with teaching. Faculty members are encouraged to first seek guidance from members of their department.

While the Office of the Dean is confident that faculty members receive adequate assistance from colleagues familiar with deep disciplinary issues, it also wishes to increase the opportunities for WSE faculty to receive assistance in teaching from pedagogy experts. The WSE will contract with the Center for Educational Resources (CER), a technology-oriented teaching center whose mission is being enhanced to support pedagogy, to provide assistance for WSE faculty members who request such help.

WSE graduate students slated to teach undergraduates receive training in a mandatory session for incoming Engineering and Arts and Sciences TAs. This is a three hour session run by the WSE Associate Dean for Academic Affairs and the KSAS Vice Dean for Faculty. Participants are provided with a teaching manual upon completion of the training session. The Graduate Representative Organization (GRO) produces the manual, thus it is a guide written by TAs for TAs. In spring 2004, the WSE and KSAS Dean's Offices will survey attendees in the previous fall TA Training Session regarding how it, as well as the teaching manual, can be improved.

Part-Time Engineering Faculty

Instructors for the PTE undergraduate programs consist of WSE full-time faculty, professional staff from the Applied Physics Laboratory (a division of the Johns Hopkins University) and engineers from area businesses and government organizations (such as the National Institute for Standards and Technology). Credentials of adjunct faculty members are carefully reviewed by the program chairs, the program committees, and by the PTE Associate Dean. Criteria for selection include education as well as professional achievement.

All PTE undergraduate courses are reviewed through an evaluation system similar to the one used by the full-time programs. Students are asked to rate the instructor on issues such as his or her "knowledge of the subject matter," "preparation for class," "responsiveness to student questions and concerns," and "quality of feedback to students." Evaluation results are reviewed by the program chairs and PTE administration. Faculty members who receive outstanding evaluations are awarded teaching distinctions at the end of the year. Very low evaluation scores are brought to the attention of the PTE Associate Dean. Lack of redress of poor teaching performance does result in termination.

Standards 11: Education Offerings

The focus of the undergraduate offerings in the Whiting School are the ABET (Accreditation Board for Engineering and Technology) accredited engineering programs and the applied science programs. In general, the Whiting School seeks to focus on the fundamental engineering science basis to modern technology without ignoring the very real need for engineers to be able to design systems subject to realistic (economic, environmental, social, legal, etc.) constraints. Mindful of the pace of technological change and the permanence of fundamental scientific and engineering principles, we seek to provide the rigorous foundation on which our graduates can build meaningful and successful careers.

Content

Content for each of the WSE programs is dictated by what subject matter knowledge and skills are appropriate for graduates of these departments. With the exception of the Department of Computer Science, all of our programs have published explicit goals and objectives for their program

Not all students enter the field in which they major. In fact, according to our Alumni Surveys, only about half of our students report entering career paths that are closely related to the major. So we at the WSE attempt to determine whether our programs and courses provide adequate time on task and information to learn and practice the knowledge, skills, and abilities relative to their experiences after leaving JHU. In the Alumni Survey we ask graduates to rank their preparation in critical subject matter areas and skills and then the importance of these areas and skills to them in their current career path (Appendix 2-A). Relative to the importance of these items, our students appear to be well-prepared in content areas and less prepared in skills. For example, students claim that their preparation level in physics, chemistry, biology, the humanities, the social sciences and core mathematics outweighs their importance in those areas. On the other hand, oral presentation, leadership/management, time management, teamwork, and technical writing are all skills rated very high on the importance scale, but lower in preparation offered by JHU.

Coherence

Our B.S. programs (11 of 16 total) are offered in a lock-step sequence, as engineering undergraduate programs must be because upper-level courses rely so heavily on the mathematics, physical sciences and engineering sciences. WSE undergraduates frequently build upon knowledge acquired in the previous semesters or even courses taken together in the same semester, such as thermodynamics and multivariable calculus (Calculus III) in mechanical engineering. Each program is carefully designed to be completed in four years, and the vast majority of our students complete their degrees within four years. However, this careful coordination militates against study abroad, and leaves little room for intellectual experimentation.

Rigor

WSE undergraduate programs are considered extremely rigorous. There are at least two issues embedded in the notion of rigor: one is how hard students must work in order to receive good grades, and another is whether our grade distribution is necessarily lower than distributions at peer schools. Although anecdotal feedback always indicates that students perceive themselves to be overworked in the WSE, the number of hours spent on out-of-class academic work was lower in the WSE than for students at our peer institutions in the 2002 CSEQ, but the sample of WSE students who responded to the CSEQ also enrolled in more credits than their counterparts at peer schools (Appendix 2-C). It may be that the heavy course load leads to the perception of academic burden since so many hours are spent in class, but we are not sure how to interpret this discrepancy between what students report anecdotally and the number of hours they report studying. The spring 2002 CSEQ survey offered a comparison of the average self-reported grades in the WSE to those for engineering students at a few of our peer schools. We found that the "majority of grades" earned by WSE students was remarkably similar to that of our peers. For example, the percentage of "A" averages earned was 19% for the WSE, versus 18% for our peers. The percentage of "A- or B+" averages earned was 39% for the WSE and 38% for our peers. This does not prove that our students are not overworked, but it may put to rest the notion that we injure our students' chances at grad school acceptance due to what has been dubbed "grade deflation" in the Whiting School.

Active Learning Opportunities

The WSE offers at least two ways in which students engage in active learning and practice and improve upon skills associated with the field or area of study, as the standard requires. The first is the "capstone" design experience. All ABET-accredited programs require a capstone design experience. In the Class of 2000 Alumni Survey, only 16% of respondents from engineering majors reported feeling either "somewhat unprepared" or "unprepared" in "engineering design" upon graduation (Appendix 2-B). This is clearly one of our strengths. What we do not ensure, however, is that students from non-ABET accredited programs have similar required capstone experiences. As CUE suggests, we will now investigate how all undergraduate programs can institute a similar requirement so that all WSE undergraduates have such a fulfilling experience critical to job acquisition or graduate school.

The second active learning opportunity for WSE students is through undergraduate research. What perhaps sets the WSE apart from other engineering schools is the opportunity for undergraduates to engage in research other than the senior design project. As mentioned under Standard 10, approximately 75% of respondents to the 2002-03 Alumni Survey noted that they had engaged in undergraduate research at Hopkins. This is confirmed by the 70% of seniors who reported the same in the 2002 COFHE Senior Survey. While the alumni noted high levels of satisfaction (88%), the 2002 seniors were not quite as satisfied — only 67% reported being either "very satisfied" or satisfied" with the experience. Perhaps our biggest concern, when it comes to undergraduate research, is the lack of systematic oversight at the school or department-level. Because, by definition, the research experience in faculty laboratories is largely at the discretion of individual faculty members, it is difficult to monitor actively or to judge the quality of the experience. We are in the process of strengthening the requirements for faculty mentoring and student participation in research.

Assessment of Educational Offerings

The Whiting School of Engineering educational offerings are assessed in a number of ways, as displayed by the WSE Undergraduate Student Outcomes Assessment Matrix (Appendix 2-D). All undergraduate departments and thus all programs are assessed by the Academic Council every four years. These reviews focus on both the program in its entirety and drill down to the course- level. Program content is assessed for every program by Alumni Surveys every year, two and five years post-graduation. Every department but DoGEE assesses undergraduate programs every year through Exit Surveys. All departments have either a Visiting Committee or an External Advisory Board, and these meet every year. Lastly, all departments but Computer Science and Mechanical Engineering have an Undergraduate Program Committee.

Standard 12: General Education

All schools of engineering face at least one common challenge: to ensure that their graduates are prepared for engineering licensing exams requiring deep disciplinary knowledge while also providing a broad general education — all without extending the student's time-to-degree beyond that for other programs offered by the institution. Each Whiting School of Engineering department is required to design programs that accomplish this feat (even though relatively few of our students sit for professional licensing exams) by closely examining each required course to ensure that it leads to purposeful learning. Furthermore, the WSE is committed to assessing whether its students obtain both a broad general education as well as deep disciplinary knowledge, though only the former will be addressed in this section.

General Education Requirements

All undergraduate programs in the Whiting School meet the 30- hour general education requirement as stated in this standard. At the core of all WSE undergraduate programs is a School-wide requirement of 18 credits of humanities or social sciences coursework, and at least 6 credits of writing-intensive courses. B.S. programs in the WSE, of which there are eleven, also require at least 15 credits of mathematics and statistics and 15 credits of basic sciences, in addition to engineering requirements. B.A. programs in the WSE, of which there are five, replace some engineering courses with more humanities, social science and writing courses, including foreign language study.

The ways in which WSE undergraduates obtain the specific skills and subject matter outlined by the General Education standard are addressed in the following section.

Oral Communication

The WSE has no specific oral communication requirement for engineering students; however, our undergraduates spend a great deal of time in formal and informal study and design groups. The intimacy of these small groups enables students to hone their oral communication skills. In addition, most WSE undergraduates make formal design presentations that require advanced oral communication. Some departments take an active role in presentation skill development. For example, the Mechanical Engineering department videotapes practice student presentations that faculty members critique for the students. Lastly, in response to ABET requirements, the WSE developed its own Professional Communication Program in 1998 that addresses oral and written communication within a technical context. This program now offers nine different communication courses including Effective Oral Presentations.

Written Communication

All WSE B.S. programs require at least two writing-intensive courses, and WSE B.A. programs require at least four writing- intensive courses. Students can fulfill these requirements in the Professional Communication Program or in the Krieger School of Arts and Sciences. WSE undergraduates also receive valuable writing experience while producing lab and senior design reports.

Scientific and Quantitative Reasoning

WSE students receive preparation in these areas through departmental requirements in mathematics, applied mathematics and statistics, and the natural sciences. All WSE undergraduate programs require students to enroll in at least 15 credits of mathematics or statistics courses, including Calculus I and Calculus II. All WSE B.S. and most B.A. programs also require students to earn at least 15 credits of natural science courses. In most cases, programs require students to enroll in both Physics and Chemistry. Beyond this base level preparation, students utilize high-level scientific and quantitative reasoning in the major.

Technological Capabilities Appropriate to the Discipline

WSE undergraduate programs focus on the fundamentals of engineering as well as the in-depth understanding of their disciplines. With this background, graduates can both undertake the lifelong learning necessary to maintain mastery of rapidly developing fields and engage in the creative process of engineering design. Technological capabilities addressed range from the effective use of engineering software to the ability to synthesize information from different areas to meet design problems with multiple constraints.

Information Literacy

WSE undergraduates obtain information literacy skills through the capstone experience — collecting information on engineering standards from discipline-specific resources and applying it to their design projects. For example, a Civil Engineering student may need the maximum weight that can be supported by a steel beam of a specific thickness and length. These standards are available through resources published specifically for engineers, such as the ASCE Standards, and are available in the library.

Values, Ethics and Diverse Perspectives

As a requirement for ABET accreditation, the WSE ensures that its engineering graduates engage in discussions of professional ethics. This topic is introduced in first-year courses within each engineering program and applied in the senior design projects with considerations of societal values and professional ethics. At this point, similar discussions do not always occur in Computer Science and Applied Mathematics and Statistics. The WSE has no School-wide requirements pertaining to values or ethnic diversity, but students are exposed to the diverse perspectives of different fields through coursework that spans the humanities, social sciences, mathematics, the natural sciences, and engineering. Moreover, WSE students also participate in orientation activities that emphasize the importance of academic integrity, and faculty are urged to devote class time to discussion of academic ethics.

Assessment of General Education

To a great extent, these requirements are embedded in coursework and assessed in homework assignments, exams and course portfolios. Several of these requirements are also assessed in the WSE capstone experience. In addition, we assess our progress through surveys — departmental Exit Surveys, the annual WSE Alumni Survey and miscellaneous Homewood surveys (e.g., the CSEQ). Below is a discussion of recent assessment data addressing each General Education topic. Overall, our students appear to be very well-prepared in content areas (particularly in the engineering sciences) and less prepared in non- engineering (or "soft") skills. We are concerned about the discrepancy that exists between the self-reported estimate of gains for our students and those for engineering students at peer institutions, as demonstrated by the CSEQ results. With nearly uniform underperformance on this survey regardless of the question, we are reluctant to attribute all variances solely to curricular differences. Rather, we suspect that the intense Hopkins culture, in which students are demanding of themselves and their institution, plays a role in these outcomes as well.

Oral Communication

The data indicate that we have made improvements in this area over time. In the spring 2003 Alumni Survey, 62% of the Class of 1997 and 70% of the Class of 2000 responded that they were either "somewhat prepared" or "prepared" in "oral presentation." The percentage of those who selected "prepared" as their answer rose from 21% (1997) to 39% (2000). However, WSE alumni rated the importance of this skill higher than their preparation level (Appendix 2-A). It is useful for us to place these results in the context of our peers as well. In the spring 2002 CSEQ results, 52% of WSE seniors claimed to have made either "quite a bit" or "very much" progress during their time at JHU in "presenting ideas and information effectively when speaking with others." This is close to the 55% of engineering seniors at our peer institutions who responded the same way.

Written Communication

The Alumni Survey data show consistent levels of preparation over time in this area — 75% of the Class of 1997 and 76% of the Class of 2000 claimed to have been "somewhat prepared" or "prepared" by their years at Hopkins in "technical writing." Again, however, our graduates rated the importance of this skill higher than their preparation level (Appendix 2-A). In addition, on this dimension, we do not compare favorably to our peers here in the CSEQ data. Only 34% of WSE seniors responded that they had made either "quite a bit" or "very much" progress at JHU in "writing clearly and effectively," a lower percentage than engineering seniors at our peer schools (46%). New leadership in the Krieger School's expository writing program offers promise of enhanced programs that will benefit WSE students.

Scientific and Quantitative Reasoning

In terms of scientific and quantitative subject matter (which includes reasoning), our students seem to be prepared. The percentage of respondents in the spring 2003 Alumni Survey who reported being "somewhat prepared" or "prepared" upon graduation remained constant in Physics (77% for the Classes of 1997 and 2000), and increased in the fields of Chemistry (63%, 1997; 74%, 2000), Core Math (80%, 1997; 92% 2000) and Applied Math (68%, 1997; 76%, 2000). The CSEQ looks more at the skills involved, and we underperformed our peers here. Sixty-three (63%) percent of our seniors versus 71% of peer seniors felt that they increased their understanding of the "nature of science and experimentation" either "quite a bit" or "very much." Our numbers were similarly lower in reported gains in "analyzing quantitative problems" (WSE, 76%; 84%, peers), but both the WSE and its peers scored well on this measure.

Technological Capabilities Appropriate to the Discipline

For this, we have abundant Alumni Survey data for engineering students ( Appendices 2-A and 2-B), but not as much for students in Computer Science and Applied Mathematics and Statistics. Graduates of the engineering programs reported high levels of preparation in their major field — 79% of the Class of 1997 and 91% of the Class of 2000 reported being either "somewhat prepared" or "prepared." Engineering alumni also reported high levels of preparation in "designing experiments" and "conducting experiments." All students reported adequate preparation in these areas relative to how important they are post-graduation. Our students need better preparation in "modeling" and "software packages" — two technological capabilities that specific engineering programs will need to target for improvement.

Information Literacy

As with all of the other skills listed, a student's ability is honed and assessed in coursework and the capstone design project. There is one question on the CSEQ survey, however, which directly addresses the emphasis placed on "developing information literacy skills (using computers, other information resources)." Students were asked to rate the institution on a scale of 1 (weak emphasis) to 7 (strong emphasis). The WSE earned a score of 5.2, with our peer institutions earning a score of 5.9. While the results are satisfactory, there is room, nonetheless, for improvement.

Values, Ethics and Diverse Perspectives

We must improve our students' preparation in "professional ethics." WSE graduates reported an importance level much higher than their preparation level in this subject (Appendix 2-A). In addition, our scores in these areas are lower than those for our peers as well. Sixty-five (65%) percent of peer institution seniors reported significant gains in "developing your own values and ethical standards," versus 52% of WSE seniors. When asked to rank the strength of institutional emphasis on "developing and understanding and appreciation of human diversity," our peer schools registered a mean of 4.6 versus our 4.3 (on a 1-7 scale), a statistically significant difference, though perhaps not practically significant. Both the WSE and our peer institutions have room for improvement in this area.

In summary, the WSE appears to provide good preparation in course subject matter, but less effective preparation in skill development. In addition, we find some discrepancies between the training provided to our graduates and training received by engineering students at peer institutions. In response to these findings, the WSE will undergo a rigorous assessment to ensure that all undergraduate programs (not just the School as a whole) provide their students with sufficient exposure to diverse disciplines and fields through general education.

One of the ways that we intend to increase the understanding of diversity is by ensuring that opportunities exist for undergraduates to engage in international study. This is quite difficult for engineering students who take courses in a lock- step manner, so we face the additional challenge of identifying programs which mirror our course sequences. However, we believe that study abroad can have a profound effect on a student's academic and personal development. It is therefore worthwhile for us to make these efforts.

Standard 14: Assessment of Student Learning

Undergraduate student learning assessment is approached in two different ways by the Whiting School of Engineering — what students learn in a specific course and what students learn from the academic program in its entirety. Though these outcomes are inextricably linked, as programmatic learning outcomes are largely the sum of individual course learning outcomes, we are able to analyze the approaches separately.

Learning Assessment for Individual Courses

Learning objectives for courses are set by individual instructors and conveyed to students in course syllabi or other documents distributed at the beginning of the semester. Instructors measure student success in meeting these objectives using a number of traditional methods such as in-class performance, homework assignments, exams, laboratory assignments and other class projects. Ultimately, the student is awarded a grade for the course based on a collection of such methods. A student's grade, therefore, represents the degree to which the student has met the learning outcomes set forth at the beginning of the semester. In fall 2001, for example, 79% of grades awarded in the Whiting School were between "A+" and "B-." While we hesitate to offer any assumptions regarding the strength of the correlation between student learning success and the percentage of high grades awarded, we are confident that a positive correlation exists.

Since fall 2001, the Whiting School has asked students to self- evaluate individual course learning in a very broad manner through the end-of-semester course evaluation. This instrument is distributed in every traditional undergraduate class — rarely for independent study enrollment — at the conclusion of every semester. The question asks students to rate their level of agreement with the following statement, "I learned a lot in this course." WSE classes receive extremely high scores on this question, with all departmental averages ranging between 3.9 and 4.2 (with 1=strongly disagree and 5=strongly agree) for spring 2003. While this information does not permit us to conclude that students are meeting the learning objectives outlined by the instructors, it does allow the instructor to confirm assumptions about student learning based on student performance in homework assignments and exams.

Learning Assessment for the Program

As stated under Standard 1, departments are expected to create learning objectives for graduates of their undergraduate degree programs. Programmatic learning objectives for undergraduates are available to prospective and current undergraduates in the following three places: (1) undergraduate program websites, (2) the Arts and Sciences and Engineering catalog, and (3) a one page information sheet on every program distributed by the Office of Admissions. In many cases these are the same objectives devised to meet the requirements of our engineering accrediting board, ABET.

How Programmatic Objectives are Set

For our ABET-accredited programs, broad educational objectives (equivalent to the MSCHE goals) and specific program outcomes (equivalent to the MSCHE objectives) are devised through a series of discussions by the faculty in ABET-targeted meetings as well as year-end faculty retreats. These programs also consult with external review committees composed of alumni, industry executives and academics in the field to evaluate the relevance of their objectives. ABET lists two levels of learning objectives which are used as a base for these discussions. The first is a general set of learning outcomes similar to the MSCHE Standard 12 General Education requirements. These are skills that alumni of any engineering program should have upon graduation from an ABET-accredited program. For example, students are expected to graduate with "the ability to communicate effectively" and "an ability to design and conduct experiments, as well as to analyze and interpret data." The second is a set of specific engineering disciplinary requirements demonstrating that students have academic knowledge bases specific to that discipline (e.g., "a thorough grounding in chemistry and working knowledge in advanced chemistry" for chemical engineering), theoretical skills (e.g., "the ability to apply advanced mathematics through multivariate calculus and differential equations" for mechanical engineering), hands-on skills (e.g., "the ability to perform civil engineering design" for civil engineering), as well as practical knowledge (e.g., "an understanding of professional practices issues such as how the design professionals and the construction professions interact to construct a project," again, for civil engineering).

The outcomes that ABET states serve only as a guide though; students are expected to meet them en route to obtaining knowledge bases and skills more closely related to the mission of the institution, school, and program. Information critical to these discussions, therefore, is a summary of what skills and subject areas our alumni deem important upon entry into graduate school or the workforce. In our Alumni Survey distributed two and five years post-graduation, we ask our graduates the importance of skills such as data analysis and technical writing and subject areas like physics and the social sciences.

ABET sets its objectives by conducting research to determine what skills and subject areas are critical to professional engineers in each sub-discipline. Our programs which are not accredited by ABET, therefore, do not benefit from this research. The results of the Alumni Survey are particularly useful to these programs. This survey provides them with an opportunity to confirm their assumptions as to which skills and subject areas are useful to their graduates. These programs also have the opportunity to consult with external advisory committees regarding the relevance of their learning objectives.

How Programmatic Objectives are Assessed

Progress in meeting programmatic learning objectives is assessed by three different constituencies: (1) the program and its faculty, (2) the students themselves, and (3) prospective employers and graduate school admissions committees. First, our undergraduate programs and the faculty associated with each program assess student progress in an incremental manner through exams and homework assignments and more holistically through engineering design capstone projects undertaken at the end of the senior year. The design project requires students to synthesize basic and advanced coursework with more applied knowledge of the discipline.

Second, students assess their own progress towards meeting learning objectives at the three following junctures, all of which are discussed in further detail below: (1) at the end of each course, (2) at the point of graduation, and (3) two and five years post-graduation. The intended purpose of the MERLIN evaluation is to assess student satisfaction with the course and the performance of the instructor. Over time, however, the Whiting School has added a few questions which assess the extent to which a course advanced the student's progress in meeting a programmatic learning objective. These are particularly relevant to ABET-accredited programs, as they all share some common objectives, but they are undoubtedly useful to other programs as well. For example, we now ask students, to state their level of agreement to the following two questions: "the course improved my engineering design skills," and "the course improved my computing skills." In addition, several of our programs currently distribute one page surveys along with the MERLIN evaluation that measure progress in all of the program's objectives.

In all but one department, graduating seniors are asked whether they have met the stated programmatic learning objectives in department-distributed exit surveys. This is a critical point of information-gathering with a very narrow window of opportunity, as students are either near or at program completion — so their perspective is extremely well-informed — but they have not yet left the Homewood campus, enabling sheer proximity to boost our response rates.

Two and five years post-graduation, all Whiting School graduates are surveyed in the WSE Alumni Survey. As mentioned above, this survey is particularly useful since respondents are now aware of what skills and subject areas are required in graduate school or the workforce, permitting departments to test the relevance of their stated learning objectives. In a cyclical effect, the survey results of previous years help to shape the learning objectives of future graduates, who then confirm or disconfirm the relevance of the same objectives with their own survey results. Students also participate in a battery of surveys administered by the University's Office of Institutional Research, from which programs can glean relevant data on whether students believe they met program-specific or perhaps more broad objectives. Current students are assessed in instruments such as the COFHE Senior Survey and the College Student Experiences Questionnaire (CSEQ). These surveys contain questions asking students to estimate their progress in a range of areas such as oral and written communication and critical and quantitative analysis, the results of which are very useful to our programs

Lastly, one of the Whiting School's core goals, though currently undocumented, is to prepare our alumni for graduate school or a professional career. In addition, we assume that there is a link — the strength of which remains undetermined — between student learning and the selectivity of the graduate program in which an alumnus enrolls or the nature of the position an alumnus accepts in the workforce. These data, provided by students but generated by graduate admissions committees and potential employers, are collected in two ways. In nearly all departmental Exit Surveys, students are asked about their future plans for employment and/or graduate school. In the WSE Alumni Survey, we again ask our alums to list the graduate degrees they have earned, are seeking, or plan to seek, and the types of professional positions they hold and have held in the past. The information is analyzed by the programs.

From Assessment to Change

Faculty members and departments are entrusted with the task of ensuring that students meet the individual course and programmatic learning objectives. Faculty members conduct systematic reviews of courses and programs during annual reviews, also known as faculty retreats. In these meetings, the faculty considers data from the department's Exit Surveys and the WSE Alumni Surveys, and discusses course portfolios. Based on these results, departments improve courses and curricula to better serve their students.

ABET also serves as an active evaluator of whether students meet individual course and program learning objectives, and an agent of programmatic change. The evaluation team is provided with evidence that students meet program objectives in the form of sample homework assignments, exams, transcripts, and survey data. In the 1999 review, ABET found that nearly all WSE programs fully met expectations. In two cases, deficiencies were addressed and corrected immediately, to ABET's satisfaction. In Biomedical Engineering, for example, ABET found that the department had not demonstrated student ability to apply statistics at an advanced level. The Department of Biomedical Engineering immediately instituted an upper-level statistics course requirement.

Lastly, WSE programs undergo broad programmatic reviews every four years by the Homewood Academic Council and self-organized external reviews every year. The Academic Council Review examines the major directions in the field represented by the department, the department's position in the field in terms of interests and coverage, the department's role in both graduate and undergraduate programs of the School and the University, interactions at the teaching level with other departments in the School and in the University at large, the quality of teaching and advising in the department, and the overall reputation and quality of department and comparisons with similar departments at other institutions that are highly regarded. External reviews, encouraged by ABET, provide an opportunity for industry professionals and academics from peer institution to examine the program, its curriculum content, the laboratories and other teaching infrastructure and the ability of the faculty to provide appropriate coverage in the field. Both processes result in recommendations to the departments and to the Dean on a range of actions to be taken, from adjustments in curriculum content to the areas of critical need in the hiring of new faculty members.

Conclusion

In this self-study, we have attempted to address the MSCHE standards while also updating the progress on the WSE's implementation of all relevant CUE recommendations. A snapshot, such as a self-study, captures an organization at one point in time. The WSE evolves too rapidly, however, for this to serve as the whole picture. We must include a vision of the future. In the text below, therefore, we have identified a set of challenges which we must meet in order to continue improving our undergraduate programs.

The Whiting School Strategic Plan (Phase II) outlines a clearly defined mission and overarching goal for the school and lists 13 distinct and measurable objectives. We have made respectable progress in meeting the goal and objectives. At this point, one of the broad strategic decisions that the School must make regards the role of our part-time undergraduate engineering programs. Is this a student population that we wish to serve? If so, how can these programs be linked more tightly with the full- time programs?

Whiting School students appear satisfied with both advising provided by professional advisors in the Engineering Advising Office as well as faculty advising. The course evaluations results are overwhelmingly positive as well. In order for advising and instruction to be made part of the annual salary review process, however (as recommended by CUE), our assessment processes must be improved. We will address the question of how to improve our processes to ensure that both faculty advising and in-class performance are accurately and thoroughly assessed and thus able to be factored directly into the faculty compensation model. Once these logistical obstacles are overcome, we have an even greater task in determining the appropriate emphasis on faculty advising and teaching in the faculty compensation model. The Whiting School is a research- based institution, but teaching and advising are also core mission functions performed by the faculty.

As a highly decentralized school, the WSE must increase its information flow — particularly from the departments to the higher levels of the administration. This will serve to heighten accountability for student learning for both the departments and the School overall. Adding Directors of Undergraduate Studies in each program and requiring that programs provide annual reports will serve as a beginning point. The next step is to find the most efficient way for a great amount of data to be transmitted without causing a loss of programmatic flexibility and autonomy or greatly increasing the workload for our departments.

Our undergraduate students appear to be well-schooled in academic subject matter and engineering skills. They seem to be less proficient in more basic skills that alumni have told us are extremely important once undergraduates enter the workforce. We will address how to increase undergraduates' skill levels without also increasing the number of courses required to graduate.

Overall, we believe we offer undergraduate students a rigorous education characterized by excellent academic programs, good teaching, and extensive opportunities for learning through research. The Whiting School has established a discipline of assessment and is on a fine trajectory of achieving the highest levels of academic quality.

© 2004 The Johns Hopkins University. Baltimore, Maryland. All rights reserved.
Last updated 20Feb04 by dgips@jhu.edu