I.7. External Evaluation and Assessment Issues

STUDENT LEARNING AND ASSESSMENT

Learning Goals and Student Assessment Plans

Chairs should make certain that the department/program has learning goals relevant for the majors and the minors offered by the department/program. The most recent learning goals are available on the department/program web pages. Any changes that are made in the learning goals should be sent to the Associate Provost for Academic Affairs (APAA) and the web page should be updated. Learning goals are also collected in the Student Assessment Plan links in Faculty Resources.

In addition to the learning goals, each department has a student assessment plan that describes how student learning is assessed. Again, the most recent version is available at the above link in Faculty Resources. The student assessment plan describes how the department/program will evaluate how well the students are progressing with the learning goals. More detailed examples of how data is evaluated can be found in Section F.4.c. of the Faculty Handbook.

Departments/programs are expected to use a minimum of one direct measure and one indirect measure. [A direct measure involves faculty or expert assessment of student learning while an indirect measure involves student perception of their own learning.] The Student Assessment Plan should focus on a short list of measures that are reviewed and used by the department/program on an annual basis.

Each department has a description of how the department/program’s learning goals relate to the Vision for Student Learning. The link above also contains those documents.

Course Syllabi

Chairs should make sure that all the faculty teaching courses in the department or program have course syllabi available for students. Although Hendrix does not have a required format or content, the chair should advise the faculty to include grading policies, attendance policies (which should be compatible with the attendance policy in the Catalog), course learning goals, office hours, academic integrity guidelines for the class, and the statement provided by Academic Support Services on accommodations for disability.

Chairs should maintain a collection of recent course syllabi used in the department or program.

Annual Assessment Planning and Reporting

Chairs should complete an annual assessment report at the end of each academic year. Step-by-step suggestions for how to do this:

• Check the department/program’s Assessment Cycle to determine which goals to focus on for the year.
• Gather the data relevant for assessing that goal as indicated by the direct and indirect measures described in the SAP.
• Create a summary of that data for the department/program to use to guide their conversation during their annual assessment meeting(s). Make evidence-based decisions about future changes to curriculum or courses.
• During the meeting(s), write up a summary of the conversations and any decisions made. Send the summary to the Office of Assessment along with any collected data. Summaries are due at the end of May each academic year.
• The Academic Assessment Committee will review summary reports using a rubric during the following fall semester. The committee may have follow-up questions. Chairs will receive a written response by February 1 of the following academic year.

These completed forms, along with the department or program’s Student Assessment Plan are available online in Faculty Resources web pages showing each department and program’s progress since 2004-2005. The APAA and the Academic Assessment Committee monitors and evaluates these assessment activities and reports back to the department or program chair as well as to the Committee on Faculty, which uses the information in their decision-making concerning faculty lines and departmental budgets.

EXTERNAL PROGRAM REVIEW AND ASSESSMENT PLAN REVIEW

Each department/program conducts a self-evaluation and external review once every 7-10 years. The Committee on Faculty evaluates each external review thoroughly and uses the information in their decision-making concerning faculty lines and departmental budgets. One major emphasis of the cycle of reviews should be the evaluation and revision of the student learning assessment plan to assure that the annual process is useful and manageable. For those programs that have not participated in a previous self-evaluation, the chair should meet with the APAA to work through the process for the first time. Statistical data is available from the Registrar’s Office and the Institutional Research Office.

As described in the Faculty Handbook assessment policy statement, the review process has the following three components:

• The self-evaluation narrative,
• The consultant’s visit and report,
• The department/program response to the consultant’s report.

Previous reports all available from Academic Affairs.

Self-evaluation Narrative

The self-evaluation narrative should speak to the following issues:

• The character of the discipline and the place of its content and methodology in liberal arts study;
• The character of the department and its approach to the presentation of the discipline at Hendrix;
• The learning goals for the general education student and for any degree programs offered by the department/program;
• Available coursework and the typical path or paths of study for the general education student and for any degree programs;
• Typical teaching and learning activities in and out of the classroom, and the connection between departmental goals and these activities;
• The connection between the program and the mission of the College including the relationship of the curriculum to the Vision for Student Learning;
• The outcomes of the degree programs, together with supporting documentation specifying the methods used to reach conclusions about these outcomes;
• A description of and commentary on the human, budgetary, and other resources available to the department;
• Plans for addressing problems or needs that have been identified.

If the external review is not the first conducted by the department/program, then information in the above list that has not changed can be summarized as long as the following issues are included:

• Updated information from the previous self-evaluation narrative highlighting any changes in the goals, character, approach, staffing, funding, and offerings of the department or program;
• Discussion of the resolution or continuance of any issues raised in the previous evaluation;
• Evaluation of and recommended revisions to the student learning assessment plan including plans for integrating assessment into the planning process.

Consultant and Consultant’s Report

The selection of the consultant is a crucial part of the evaluation process and chairs are urged to confer with all members of the department/program. In addition to expertise relative to your discipline, you should consider the following qualifications:

• Experience with annual plans for assessing student learning;
• Extensive experience as a faculty member in a small, undergraduate liberal arts college environment;
• Recent experience with the Higher Learning Commission or Southern Association re-accreditation process;
• Experience with engaged learning in the discipline;
• Previous experience as an external program consultant.

Final selection should be made in consultation with the appropriate Area Chair, the APAA and the Provost. During the semester before the visit is to be scheduled, you should submit one or more names and vitae to the APAA and receive approval before inviting a consultant to participate.

Once you have approval, the chair should invite the consultant to participate. Your invitation conversation should include the following items:

• The procedures and goals for an external consultant.
• Payment details:
  • The consultant completes a W-9
  • The payment amount is a $800 stipend plus reasonable travel expenses (keeping expenses at a $800 total)
  • Expenses are reimbursed as we get them in Academic Affairs
  • The stipend is paid after receipt of the final report
• The schedule details:
  • When the consultant receives the self-study narrative
  • Schedule the on-campus visit (as described in a later section)
  • Consultant report due one month after visit
• Visit meals details:
  • Keep the meals on a small scale with a few faculty members instead of trying to include the entire department
  • Remember that the consultant usually prefers short meals with plenty of evening downtime to work on visit notes.
  • Consider working meals.

In preparation for the visit, the following items should be collected and sent to the consultant:

• Self-evaluation narrative
• Annual plan for assessing student learning and annual assessment reports
• Web addresses for college, department/program, Catalog, Faculty Handbook, and Odyssey Program Guide
• Admission viewbook (or link)
• Curriculum vitae of faculty and staff
• Recent enrollment data
• Course syllabi
• Other data collected for assessing student learning

The consultant should spend 1½ to 2 days on campus. A typical schedule is included below. Please plan as far in the future as you can since the visit includes your department, the Committee on Faculty, the Provost, and, if possible, the President. The appropriate area or building administrative assistant should help develop the schedule and all involved should be sent calendar notifications. The consultant’s vita and the visit schedule should be sent to program faculty members and the Committee on Faculty at least two weeks before the visit. The consultant should be prepared for a concluding meeting that includes department/program faculty along with the Committee on Faculty during which a preliminary summary of the visit should be given. A formal written report to the chair and Associate Provost should be received within one month of the visit.

Department/Program Response

Once the consultant’s report has been received, the department or program should write a formal response to the report. That report should be completed before the end of the semester when the report was received.
Once the department’s response has been written, the original narrative, the consultant’s report, the response, and any changes in the assessment plan should be sent to the Associate Provost. The materials are added to the assessment files in Academic Affairs and will undergo a review by the Committee of Faculty. The Committee on Faculty will send a final closing communication concerning recommendations concerning issues in the review.

Planning for an External Review

Typical Timeline [Additional details for some items are in later sections.]

• Planning Semester (usually Fall):
• Gather the information for the self-study narrative: Besides the department or program’s own records, statistical data is available from the Registrar’s Office, the Institutional Research Office, the Academic Success Office, the Assessment Office, and the Career Services Offices. Talk with those offices to see what they have available or what they can produce for your department or program.
• Prepare a short list of possible consultants. (More on this issue below.)
• Ask the Provost for approval of a consultant.  
• Invite approved consultant and schedule the visit for the following semester: The target date is to have this done by mid-semester. 
• Complete self-study narrative and finalize all additional data attachments.
• Review Semester (usually spring):
• Complete consultant visit schedule and schedule lodging: The target date is 8 weeks before the visit.
• Send materials to consultant: The target date is 6 weeks before the visit.
• Send materials and schedule to department or program faculty and the Committee on Faculty: The target date is two weeks before the visit.
• Make final local travel arrangements with the consultant: The target date is two weeks before the visit.
• Coordinate the visit and get travel receipts from consultant. 
• Thank consultant, remind the consultant about the report, and submit local receipts to Academic Affairs. The target date is one week after the visit. 
• When you receive the report from the consultant, notify Academic Affairs so the stipend can be paid to the consultant. 
• Completion Semester (usually the following fall):
• Prepare the department or program’s response to the consultant report. The target is to have the report submitted no later than mid-semester of the semester following the visit.
• Once the department’s response has been written, the original narrative, the consultant’s report, the response, and any changes in the assessment plan should be sent to the Associate Provost.
• The materials are added to the assessment files in Academic Affairs and will undergo a review by the Committee of Faculty.  The Committee on Faculty will send a final closing communication concerning recommendations concerning issues in the review.The Assessment Committee will also have access to the final report for assessment purposes. 

Consultant Selection  

The selection of the consultant is a crucial part of the evaluation process and chairs are urged to confer with all members of the department or program. In addition to expertise relative to your department or program, you should look for the following qualifications:

• Experience with annual plans for assessing student learning and the evaluation of assessment data.
• Extensive experience as a faculty member in a small, undergraduate liberal arts college environment.
• Recent experience with the Higher Learning Commission or Southern Association re-accreditation process.
• Experience with engaged learning in the discipline(s). 
• Previous experience as an external program consultant. 

You will need to contact those you want to consider. Explain that you cannot commit to them now but want to know if they are willing to serve in this position. You can answer any details about the procedures involved, including payment details.

Ask for a cv but do not make any commitments other than you will contact them again when you have approval from the Provost to proceed. 

During the semester before the planned visit, two or three finalists (and their cvs) should be sent to the Provost. You should avoid consultants who have a personal or strong professional connection with a member of the department or program. Any such connections should be reported to the Provost at the time you submit your list.   

Once you have approval, the chair should invite the consultant to participate.  Your invitation conversation should include the following items:

• The procedures and goals for an external consultant.  
• Payment details:
• The consultant completes a W-9
• The payment amount is a $800 stipend plus reasonable travel expenses (keeping expenses at a $800 total)
• Expenses are reimbursed as we get them in Academic Affairs 
• The stipend is paid after receipt of the final report 
• The schedule details:
• When the consultant receives the self-study narrative 
• Schedule the on-campus visit (as described in a later section) 
• Consultant report due one month after visit 
• Visit meals details:
  • Keep the meals on a small scale with no more than four faculty members instead of trying to include the entire department 
  • Remember that the consultant usually prefers short meals with plenty of evening downtime to work on visit notes. 
  • Consider working meals. 

The consultant should spend 1 ½ to 2 days on campus.  A typical schedule is included below.  Please plan as far in the future as you can since the visit includes your department, the Committee on Faculty, the Provost, and, if possible, the President. The introductory meeting with the Provost and the Associate Provost should be the first on-campus meeting. The concluding meeting with the department or program faculty and the Committee on Faculty should be the final meeting during the visit. The appropriate area or building administrative assistant should help develop the schedule and all involved should be sent calendar notifications.  The consultant’s vitae and the visit schedule should be sent to program faculty members and the Committee on Faculty at least two weeks before the visit.  Remind the consultant that the formal written response needs to be submitted to the chair and Associate Provost within one month of the visit.

Typical Schedule for Consultant’s Visit

Notes: 

1. Make sure someone is assigned to take the consultant from each meeting to the next. 
2. Get receipts for travel expenses from consultant before departure from campus.  
3. Be mindful of the consultant’s time outside normal business hours. Your consultant may need time in the evenings to tend to other demands not related to the review.