External Review of Course Content Name Of Technique: External Review of Course Content Source: David J. Malik, Indiana U.-Purdue U. at Indianapolis, Chemistry, Innovative Higher Education 20(4):277-286, 1996 (Courses in the School of Science, IUPUI) Introduction Faculty members are largely responsible for determining the content of their courses, even in disciplines in which curricula are recommended by external accreditation or professional boards or departmental subcommittees. The expertise and the opinions of the faculty, therefore, are strong determinants in the design of course content, including its presentation and the interrelationship of topics within the course. Disciplinary peers can be invaluable in providing an evaluation of the merits of course content and its assembly. Given the subject of the course, you might want to know if your goals are appropriate. Is the content current, relevant, complete, sequenced well? Do the course materials (handouts, texts, assignments, etc.) contribute to student learning? Do the examinations assess student mastery of the course goals? Self-assessment will be evident in materials assembled and in the reflective memo. Evaluation by reviewer will provide documentation of teaching with regard to content. Description: This technique involves obtaining review of course content in a comprehensive way. External (same discipline, different institution) reviewer evaluates content (defined as materials that adequately reflect the course content: course description/goals, syllabus, reflective memo, text(s), handouts, examinations, assignments, other items, optionally a videotape of a class session) based on open-ended criteria that are focused to some degree in a cover letter from the department describing key issues, the purpose of the process, the potential consequences of the review, an example of an internal rating form, and any standards that have been established by accrediting bodies or professional societies for instruction in the discipline. The reflective memo written by the course instructor should address the overall purpose of the course, how the materials are organized, the style of presentation, the relationship of the sequence of content and the teaching and assessment methods to the course goals, the rationale for topic selection, and how the content might be used for future disciplinary studies. The external reviewer submits an evaluation of the course content in the form of a letter. Purpose/Purported Outcome: To improve course content based on review by objective, respected peers Time Involved: 1) Time for instructor to compile material, write reflective memo, arrange videotaping; 2) time for department to prepare cover letter that focuses review; 3) time for reviewer to evaluate materials, write letter of evaluation; 4) time for instructor/department to consider feedback. Who Is the Reviewer?: Reviewer(s) should be familiar with the instructor without having close professional or personal ties and should be a recognized expert in the instructor's discipline. The faculty member under review should have a substantive role in selecting reviewers. They should be an external, well-respected, unbiased peer (from a different institution) in the same field (instructor has substantial input on selection of reviewers). Who Is Reviewed?: Course instructor(s) (what is specifically reviewed is the course content assembled by the instructor) Format of Instruction: Variable Unit of Observation: No direct observation in classroom; the unit is the total course content. Criteria for Review: Criteria address both the discipline itself (currency, relevance, accuracy, etc.) and mechanics of delivery (sequencing, examinations, etc.). Specific criteria are outlined in the cover letter sent to the external reviewer. Scope: The context of the review is specifically an evaluation of the substance of a course, including all materials and examinations as well as the rationale for their inclusion in the course. Documentation: Instructor assembles course materials and descriptive narratives; reviewer responds with a letter of evaluation that could go to the instructor, department (chair, executive committee), and/or the divisional committee. Pre- or Post-Course Meetings: None Audience/Student Population: Variable Observation of Instructor with Students: Videotaped session, if provided Interaction of Reviewer with Students: None Modifications: This general technique was applied in the following way in the School of Science at IUPUI: Institutional Teaching Review Panel selects 2 external reviewers from a pool (pool assembled with substantial input from instructor); panel sends package of material to reviewers; panel also rates material, considers evaluation letters from external reviewers, and views 2 videotaped class sessions to evaluate how well content is delivered and to assess fairness of student ratings of course and instructor. Instructor receives the final anonymous panel rating. Formative/Summative Use: Summative Specific Directions: Instructor writes a reflective memo about the course describing objectives and how the content is designed so that students can meet those objectives. The reflective should address the overall purpose of the course, how the materials are organized, the style of presentation, the relationship of the sequence of content and the teaching and assessment methods to the course goals, the rationale for topic selection, and how the content might be used for future disciplinary studies. Instructor assembles course materials - syllabus (with course description/goals), course notes, text(s), assignments, examinations, other materials, optionally a videotape of a class session. Instructor identifies or suggests potential external content reviewer(s) (well-respected expert(s) in the same field from another institution). Department chair/executive committee chooses reviewer(s). Chair writes cover letter describing purpose of review, how feedback will be used, any particular issues that need to be addressed, any standards that have been established by accrediting bodies or professional societies for instruction in the discipline, and perhaps an example of an internal rating form. External reviewer submits an evaluation of the course content in the form of a letter to the department. Department (and perhaps divisional committee) considers the feedback. If the review is conducted by a well-respected colleague in the field who has no relationship with the instructor, the evidence should be "objective" and "credible."