Academic Affairs

(Revised, 1997-98)
Writing samples will be taken from students in ENG 234: Reading and Writing about Literature (the first class specifically for English majors) and in all senior seminars (capstone courses) each semester. The Assessment Committee will form a team to read and evaluate the samples using a rubric constructed by the committee. This assessment will be administered each semester, and the committee will evaluate the samples, report on the findings, and file the results for later comparison studies. To track each student's progress, the department will compile a portfolio, with student names converted to numbers to maintain privacy, that will include the writing samples and Committee assessments from the 234 and senior-level courses, student self-evaluations of their writing, and a collection of additional writing samples contributed by the student. The Assessment Committee will evaluate the portfolios upon the completion of each student's degree program.

Writing Samples: Writing samples will be taken from students in ENG 234 Reading and Writing about Literature (the first class specifically for English majors) and in the senior seminar classes during the fall semester. The Committee will read and evaluate the anonymous samples using a rubric constructed by the committee. These two bodies of writing will be compared to assess the improvement achieved over the major.

Revised, 1994-95
Writing samples will be taken from students in ENG 234: Reading and Writing about Literature (the first class specifically for English majors) and in all senior-level courses each semester. The Assessment Committee will form a team to read and evaluate the samples using a rubric constructed by the committee. This assessment will be administered each semester, and the committee will evaluate the samples, report on the findings, and file the results for later comparison studies. To track each student's progress, the department will compile a portfolio, with student names converted to numbers to maintain privacy, that will include the writing samples and assessments from the 234 and senior-level courses, student self-assessments of their writing, and a collection of additional writing samples contributed by the student. The Assessment Committee will evaluate the portfolios upon the completion of each student's degree program.
A new assessment tool measuring students analysis of their own writing and interpretation skills will also be added to the portfolio.

Writing Samples: Random writing samples will be taken from students in ENG 234 Reading and Writing about Literature (the first class specifically for English majors) and in the senior seminar classes during the fall semester. Instructors of the classes will form a team to read and evaluate the anonymous samples using a rubric constructed by the team. These two bodies of writing will be compared to assess the improvement achieved over the major.

The department is deciding if the results will also be kept to be used in a longitudinal study when the 234 students have reached the seminar classes.

1994-95, Capstone courses made a separate category.
Capstone courses: The senior seminars (ENG 443-471) serve as capstone courses for all majors.

The committee used this rubric to assess retrospectively all of the essays received since the inception of the writing assessment process in 1996-97. In conjunction with this retrospective assessment, we received a substantial number of new writing samples from students in English 234 (16) and the largest pool yet from the senior seminars (14). The retrospective assessment coupled with the current assessment allowed us to compare individual and group performances for the last three years. The results reflect improvement by individuals and groups over this period, although it is important to remember that the method of assessment changed significantly over that time. Comparisons will become increasingly effective while the current system remains in place.

These scores produced both synchronic and diachronic assessments. Seniors were compared to sophomores within each year’s assessment group, seniors were compared to sophomores for each of the preceding years’ assessment group, the current seniors were compared to the preceding years’ seniors, and the current sophomores were compared to the preceding years’ sophomores. Finally, the differentials between senior and sophomore groups were compared over time. The results are tabulated below:

Year	Eng. 234		Senior Seminars		Difference
	#	Ave. score	#	Ave. score
1996-97	13	9.09	4	11.875	+2.785
1997-98	16	10.28	1	11.7	+1.42
1998-99	16	10.55	14	13.643	+3.093
		+1.46		+1.768	+ .308

Since the essays assessed each year were collected at the end of each semester, the results seem to indicate that our teaching methods in English 234, or the quality of our students, or both are improving. However, the earlier assessments were based on short essays on a common topic produced for no grade in the last week of each semester. Some students did not submit essays under this system, and those that did may not have done representative work given the nature of the assignment. The most recent assessment was based on an end-of-semester writing project completed for a grade. Nonetheless, the increasing contrast between the quality of work produced by seniors and sophomores seems to indicate that our program between English 234 and the end of the senior seminar (including the seminar itself) is meeting one of our major missions: the improvement of student writing.

The portfolio assessments show mixed results. As yet we have only a few students who provided essays at both the 200 and senior seminar levels. Not all of these students showed progress in their writing; however, others showed substantial improvement:

The differences between the results for individuals and groups indicates that, despite variation in individual performance, student work on the whole improved from the first introductory course for majors to the capstone course for seniors. Although the sample was small, it seems significant that the first three students, whose scores reflect a three-year time span, improved more than the last two, whose scores only cover a two-year span. It may be that students taking a senior seminar in their junior year found it more difficult to meet the essay assignment’s complexity, or students who delayed entry into the major through 234 until their junior year did not perform as well as seniors as the other students did.

Our assessment of portfolios for individual students did meet with some success this year, as for the first time a senior graduated with a writing portfolio in place. However, the portfolio contained only the two writing samples taken from 234 and a senior seminar and the students’ self-evaluations.

It is difficult to assess the accomplishments of the senior seminar writers in comparison with the English 234 classes because we received very few (3) essays based on "The Story of an Hour." No conclusion can be drawn from so small a sampling. It may be worth urging faculty to set aside class time for seniors to complete the sample, since seniors will no doubt be struggling with academic and career requirements as graduation approaches. The portfolio essays submitted by seniors reflect a much more finished type of writing than the writing samples. They handle literary vocabulary much more confidently and use it in the service of their critical needs to a much greater degree. These papers also cite and analyze evidence more effectively. There is, in general, much more familiarity with the expectations of the discipline in the portfolio papers, some of which look as if they had been prepared for student conferences and journals. As such, they reflect the additional goal of entertaining an informed audience.

Because of the assessment results of 1996, which suggested that the English 234 essays were on the whole lacking in argument, in idea development, and in clear and coherent focus that mostly summarized assigned reading, we have revised the 1997-1998 Composition Program curriculum in specific ways. Both the English 101 and 102 courses need to continue the text-based writing/learning program that was instituted in 1995-96; however, we will tighten the focus in the two courses by having composition instructors work from required textbooks. The English 101 list of textbooks will all have a "Critical Thinking" apparatus while the English 102 textbooks will all have a "Writing Across the Curriculum" apparatus. The required book list will provide a way for the Department of English to present a unified and coherent pedagogy in the Composition Program that should directly impact the writing abilities of the students who proceed on to English 234. Both English 101 and 102 will focus and challenge our students to work harder at learning argumentation strategies, become familiar with the academic discourse conventions of various fields within the liberal arts curriculum, and to develop their analytical reading, writing, and thinking skills. In addition, the Assessment Committee began compiling portfolios for each English major, using a numbering system to insure privacy.

Most essays from ENG 234 successfully presented a central idea, but on the whole were lacking in argument, idea development, and a concise focus. Responses were short and several lapsed into book report summary or the presentation of a body of evidence that was not adequately interpreted. Nearly all of the essays showed skill in citing evidence from the text, and many displayed a developing maturity in analysis and interpretation. Students' success with style, diction, and disciplinary conventions varied widely, from writers who demonstrated no command of critical expectations to those who were beginning to develop a sophisticated voice. This suggests that both the ENG 101 and ENG 102 courses need to continue to focus and challenge our students to work harder at learning argumentation strategies and to develop their reading, writing, and thinking skills.

The 400-level essays, in general, exhibited much more skill in presenting a clear, central idea and in citing, analyzing, and interpreting evidence with less tendency to generalize or summarize the narrative. Organization was generally stronger. The most striking area of improvement was a greater sensitivity to language, with attention to connotation, nuances of meaning, ambivalence, and irony--in short, the ability to read and analyze the text closely.

One problem that needs to be addressed in the lower level courses is to increase student argumentation skills (rather than summary) in critical analyses.

As a result, the committee designed another assessment instrument to be administered in the Fall of 1995. Students will respond in essay form to a short piece of fiction. This tool will reflect the student's ability to interpret a literary work, write a limited thesis, and discuss and support it with textual evidence in an organized and coherent as well as stylistically and grammatically correct manner. The committee constructed criteria for evaluation of the essay responses.

On schedule, in Fall 1994 students in 234 have again written to two instruments, one on poetry designed by Dr. Rebecca Umland, one on the essay developed by Dr. Charles Peek.

The plan (Appendix II, p. 79, Self-study for Academic Program Review, also found in the Department Profile) calls for the instrument to be administered to the senior seminars, our exit level class, before the end of the fall semester. The faculty involved (Dr. Kate Benzel, Dr. Becky Umland, Dr. Sam Umland) will have their seminar students write to both instruments. The four faculty involved in 234 and the seminars thus far will then evaluate the writings, comparing the ability to interpret and to express interpretation in writing between students in the entry level and students in the exit level classes. In addition, the Fall 1994 234 class will write another baseline document at the end of the Fall term. These, too, will be used in comparisons both directions: with the initial baseline and with the exit-level student writing.

It is too early for results of the comparison because the other half of the comparison is not yet available. Some preliminary comment can be made on the base line sample, however.

By and large, our entry-level class students can formulate an interpretation and express it clearly. They can do so being aware of other interpretations without being unduly influenced by them, even when they appear "authoritative." Since we are in transition to the new major, it is necessary to distinguish between students in the entry-level class who are themselves entry-level students and those taking the class but already advanced in their study. The sample from 234, Fall 1994, will clearly show an advanced ability of upper class students over lower division students, and lower division students, a higher proficiency among those who have taken courses with us as compared to three transfer students for whom this is their first course with us. So, even within the baseline sample, there is evidence of what we hope will be the case: that taking our courses increases the quality and expression of student interpretation. Additionally, we have discovered that the more focused the intent (in this case to measure quality of interpretation and its expression), the more "common sense" can be the "rubric" by which each writing is measured and the two levels of writing compared.

Programmatic changes will be decided upon when the assessment is complete.