ASSESSMENT INSTRUMENTS AND METHODS AVAILABLE TO ASSESS STUDENT LEARNING
Compilation of material obtained from the following web sites:
http://www.wisc.edu/
http://depts.washington.edu/
Note:
Some of the listed methods are more appropriate for program-level assessment and some for course-level assessment
General Considerations
There are essentially two ways to assess learning outcomes by:
o Asking students, themselves, to assess their own learning. When not included as part of their grades, student self-assessment can be effective at both course and program levels. However, students are sometimes not very good judges of their own abilities. In addition, they may not understand what important aspects of a discipline they may be missing.
o Judging student products by experts. At the course level, development of student leaning outcomes allows a closer alignment between these outcomes and the basis upon which students are graded. Essentially, two elements are needed: student products that allow the inference of specific competencies or lack thereof, and experts who can view these products and assess the competence shown.
Two questions will arise. First, does the current curriculum and course assignments provide the needed student products? In other words, are students getting regular opportunities to demonstrate ability in the outcome areas?
The second question is, who should judge the extent to which outcomes are being attained? Often the instructors of specific classes may be appropriate. In other cases, it might be appropriate to bring a faculty committee together to read a sample of student papers, or perhaps a collection of student portfolios, reflecting work over several courses or the entire major program.
Ways of Assessing
Assessment of student learning can be conducted using a variety of available instruments and methods. Many experts believe that a combination of assessment approaches can be the most effective way to measure student learning.
A. Direct Indicators of Learning
1. Capstone Course Evaluation
Capstone courses integrate knowledge, concepts, and skills associated with an entire sequence of study in a program. Capstone course is the place where students are most likely to exhibit their cumulative understanding and competence in the discipline
2. Course-Embedded Assessment
Assessment practices embedded in academic courses generate information about what and how students are learning within the program and classroom environment. Course-embedded assessment takes advantage of already existing curricular offerings by using standardized data instructors already collect or by introducing new assessment measures into courses. The embedded methods most commonly used involve the development and gathering of student data based on questions placed in course assignments. These questions, intended to assess student outcomes, are incorporated or embedded into final exams, research reports, and term papers in senior-level courses. �..This assessment is a separate process from that used by the course instructor to grade the exam, report, or term paper.
3. Tests and Examinations
In most cases, a test will be one part of a fully developed assessment plan. Tests are commonly used in association with cognitive goals in order to review student achievement with respect to a common body of knowledge associated with a discipline or program.
Using this approach, there are two primary testing alternatives; first, locally developed/ faculty generated tests and examinations, and (2) commercially produced standardized tests and examinations.
Locally developed testing and examinations are probably the most widely used method for evaluating student progress. For assessing the validity of an academic program, examinations designed by the instructors who set the educational goals and teach the courses is often the best approach.
Commercially generated tests and examinations are used to measure student competencies under controlled conditions. Tests are developed and measured nationally to determine the level of learning that students have acquired in specific fields of study. When using commercially generated tests, national standards are used as comparative tools in areas such as rates of acceptance into graduate or professional school, rates of job placement, and overall achievement of students when compared to other institutions. In most cases, standardized testing is useful in demonstrating external validity.
Some of the more commonly used national tests include:
WORK-KEYS, SAT, ACT, LSAT, and MCAT
4. Secondary analyses of course papers
The instructor will read student papers in order to assign a grade. Faculty committees can also read these same papers to assess the attainment of program-level learning outcomes. In most cases, this second reading should be done by other than the instructor or by others along with the instructor, as the purpose for the assessment is different.
5. Secondary analyses of course projects
Products other than papers can also be assessed for attainment of program-level learning outcomes. For example, if students are required to give oral presentations, other faculty and even area professionals can be invited to these presentations and can serve as outside evaluators.
6. Portfolio Evaluation
Portfolios used for assessment purposes are most commonly characterized by collections of student work that exhibit to the faculty and the student the student's progress and achievement in given areas. Included in the portfolio may be research papers and other process reports, multiple choice or essay examinations, self-evaluations, personal essays, journals, computational exercises and problems, case studies, audiotapes, videotapes, and short-answer quizzes. This information may be gathered from in-class or as out-of-class assignments.
Portfolio evaluation is a useful assessment tool because it allows faculty to analyze an entire scope of student work in a timely fashion. Collecting student work over time gives departments a unique opportunity to assess a students' progression in acquiring a variety of learning objectives. Using student portfolios also gives faculty the ability to determine the content and control the quality of the assessed materials.
7. Pre-test/Post-test Evaluation
Pre-test/post test assessment is a method used by academic units where locally developed tests and examinations are administered at the beginning and at the end of courses or academic programs. These test results enable faculty to monitor student progression and learning throughout prescribed periods of time. The results are often useful for determining where skills and knowledge deficiencies exist and most frequently develop.
8. Videotape and Audiotape Evaluation
Videotapes and audiotapes have been used by faculty as a kind of pre-test/post-test assessment of student skills and knowledge. Disciplines, such as theatre, music, art, communication, and student teaching, that have experienced difficulty in using some of the other assessment methods have had significant success in utilizing videotapes and audiotapes as assessment tools.
9. Internships, Field experiences, Clinical Evaluations
Internships, field, or clinical experiences are ideal for assessing many program or major and general education goals. When these occur at the end of the program or major, they often serve as capstone experiences. It is especially useful to have external experts assess the performance of your students.
10. Authentic Assessment
In some courses, opportunities can be found to ask students to engage in a simulation of a real-life problem that they must solve using the knowledge and skills they have gained in the course. A single project can be structured to assess both mastery of course content and attainment of program or major goals as well as certain general education goals such as communication skills, life-long learning skills, critical thinking skills, and social and education values. (Sandy LaVoie had her class invest in the Stock Market)
11. Case Study
Presented with a realistic example of an application in the field, students must respond with an analysis that demonstrates their mastery of course content and their ability to apply the information and skills they have learned. The end product of a case study is a rich, thick description of the phenomenon being studied that illuminates the student's understanding of the phenomenon through the application of the knowledge and skills they have gained.
12. Focus Groups
A focus group is an informal, small-group discussion designed to obtain in-depth qualitative information. Individuals are specifically invited to participate in a discussion focused on a restricted number of topics, usually no more than three to five. The discussion is informal as participants are encouraged to talk with each other about their experiences, preferences, needs, observations, or perceptions. The conversation is led by a moderator whose role is to foster interaction, make sure all participants are encouraged to contribute and that no one individual dominates the conservation. This assessment technique lends itself particularly well to the evaluation of students� attitudes and values.
13. Journals
Journals or learning logs have been used in composition courses for years as a tool for increasing student writing and motivation for writing and for assessing students' writing skills.
14 Writing Samples
Writing assignments can be used as a measure of students' mastery of course content and attainment of program or major goals. Such assignments may also be used as a direct measure of the general education communication skills goal as well as an indirect assessment of critical thinking skills. Examples of writing samples include essays, research or term papers, answers to essay questions on tests, book reports, summaries, lab reports, and the like.
15. Oral Presentations/Oral Exams
Oral presentations can be tailored not only to assess students' mastery of course content but also their attainment of general education goals such as critical thinking, general knowledge and historical consciousness, understanding the impact of science and technology, and educational and social values. Oral presentations based on course content can be used as a direct measure of students� communication skills.
16. Interviews
Interviews are usually one-on-one, private, and involve fewer questions than an oral exam.
Additional Course-level assessment tools:
Reports, projects, demonstrations, lab assignments, homework assignments, intern evaluations, field work reports, self evaluations, critiques, performances, reviews.
B. Indirect Indicators of Learning
1. External Reviewers
Peer review of academic programs is a widely accepted method for assessing curricular sequences, course development and delivery, and the effectiveness of faculty. Using external reviewers is a useful way of analyzing whether student achievement correlates appropriately with departmental goals and objectives.
2. Student Surveying and Exit Interviewing
Student surveying and exit interviews when combined with other assessment instruments produce important curricular and co-curricular information about student learning and educational experiences. During this process, students are asked to reflect on what they have learned as majors in order to generate information for program improvement. Through using this method, universities have reported gaining insight into how students experience courses, what they like and do not like about various instructional approaches, what is important about the classroom environment that facilitates or hinders learning, and the nature of assignments that foster student learning.
3. Alumni Surveying
Surveying of alumni is a useful assessment tool for generating data about student preparation for professional work, program satisfaction, and curriculum relevancy. As an assessment supplement, alumni surveying provides departments with a variety of information that can highlight program areas that need to be expanded or enhanced.
Surveys of alumni can be conducted one, two, and five years after graduation.
4. Employer Surveying
Employer surveys can provide information about the curriculum, programs, and students that other forms of assessment cannot produce. Through surveys, departments traditionally seek employer satisfaction levels with the abilities and skills of recent graduates
5. Curriculum and Syllabus Analysis
In a perfect planning/implementation cycle, once a department has defined its objectives, all phases of the curriculum and each individual course would almost automatically cover all the bases needed to provide each student the opportunity to learn the essential components of those objectives.
In any case, not every course needs to attempt to cover all the objectives for the major. As one technique to keep a focus on the agreed-upon objectives, curriculum analysis provides a means to chart just which courses will cover which objectives.
Syllabus analysis is an especially useful technique when multiple sections of a department course are offered by a variety of instructors. It provides assurance that each section will cover essential points without prescribing the specific teaching methods to be used in helping the students learn those
