Chapter 4

4.5 – Standard V: Educational Effectiveness Assessment

• by

Standard V: Assessment of student learning and achievement demonstrates that the institution’s students have accomplished educational goals consistent with their program of study, degree level, the institution’s mission, and appropriate expectations for institutions of higher education (I.1.001.MSCHE_Standards_RoA).

Summary Declaration

Syracuse University has educational goals in place for academic degree programs as well as for co-curricular and functional units. There is a centrally organized assessment system in place to facilitate the assessment process, and numerous examples exist of how assessment results have been used for positive change. Based on our review and analysis of documentation, we suggest that Syracuse University is in compliance with Standard V.

Overview of Satisfactory/Exemplary Criteria

Criterion 5.1 relates to educational goals. Syracuse University has clearly stated educational goals (student learning outcomes) for all academic plans (stand-alone minors, bachelor, master, certificates of advanced study, and doctoral) that are maintained through the annual assessment and action plan process. Each of Syracuse University’s more than 400 programs, located in school and college departments, are required to submit an assessment and action plan and to report findings and improvement activities annually. The Office of Institutional Effectiveness and Assessment, reporting to the Provost’s Office, coordinates these assessment and reporting processes, which are grounded in the educational goals or learning outcomes identified by each program. Where appropriate, specialized accreditation requirements also inform the process and structure of learning outcomes.

Review across plans reveals goals that are tied to the University’s mission, are interrelated, and that focus on relevant educational experiences. As described in Standard III, Criterion 3.5, both specialized accreditation and institutionally shared academic values such as those expressed in the Liberal Arts Core make significant contributions to the extent to which educational goals are interrelated. The University uses Taskstream-TK20 enterprise assessment and accreditation software to manage, summarize and report on assessment activities from across all areas of the University. Taskstream-TK20 allows individual academic plans to align with overall University goals and general education competencies. Reviewers may access the full set of assessment and action plans for all degree programs in our Tk20 online assessment management system. In the document roadmap, we offer two examples of assessment and action plans—one undergraduate and one graduate—from the Maxwell School (V.1.001.Maxwell_HIS_History_BA_AAPlan, V.1.002.Maxwell_PAIA_InternationalRelations_MA_AAPlan).

The Office of Institutional Effectiveness and Assessment uses a scoring rubric to identify strengths and weaknesses in assessment and action plans (V.1.005.Academic_Program_Assessment_Tracking_Rubric_F2017, V.1.006.Co-Curricular_Functional_Assessment_Tracking_Rubric_F2017). Scores from the rubric serve as feedback to the academic units to help faculty and staff improve the usefulness of the plans.

Meaningful curricular goals with defensible standards for evaluating whether students are achieving the desired outcomes are further promoted through a course development process that involves several stages of approval: All courses and academic programs are approved by departments, schools and colleges, and the University Senate (see Course Proposal Form, University Senate Committee on Curricula) as well as the New York State Education Department (V.1.003.College_University_Evaluation_NYS_Education) and U.S. Department of Education (V.1.004.USDepartment_Education_Accreditation_Details).

The 2015 academic strategic plan includes a goal to “boldly instill in all undergraduate students those core competencies and values, and provide dynamic learning experiences, that drive intellectual excellence, feed creativity, and cultivate the skills needed to excel.” (I.1.002.Trajectory_to_Excellence). Described in our documentation of the shared competencies (III.5.009.Overview_of_Competencies) and above in our discussion of Standard III, Criterion 3.5, this strategic planning goal expresses educational goals at the institution level and applies to both graduate and undergraduate students. As implementation of the academic strategic plan has unfolded, we have taken care to consider what outcomes meet this goal and what each outcome means at the undergraduate and graduate levels.

Criterion 5.2 relates to organized and systematic assessments. With the leadership of the Office of Institutional Effectiveness and Assessment (established in 2014), Syracuse University has developed an organized and systematic assessment system. The office manages outcomes and associated plans in Taskstream-Tk20, a software tool that facilitates standardized reporting structures and document management and supports efficient review of plans. Staff members of the Office of Institutional Effectiveness and Assessment evaluate all documentation submitted to this assessment system using standard scoring rubrics. The office provides feedback to academic, co-curricular, and functional units based on these standard rubrics and also provides coaching for faculty and staff who need assistance with development and execution of their assessment plans.

During the 2014-15 academic year, the assessment working team met with individual faculty, small groups of faculty, and entire departments to communicate Syracuse University’s general process for student learning outcomes assessment. The specific details for each assessment and action plan were left to the faculty to determine, and they were asked to provide Phase 1 components of the assessment and action plans (student learning outcomes, measures, criteria) by the end of the fall 2015 semester.

Throughout the fall and into the spring 2016 semester, the assessment working team reviewed and rated Phase 1 components for 346 academic programs based on specific criteria the team had identified. The ratings revealed that almost half of the programs (47%) had made sufficient progress in plan development to begin implementing during the 2015-16 academic year; even more would be able to implement after minor revisions to the plan components. The table below provides a historical snapshot from March 2016 describing our progress in systematically implementing campus-wide learning outcomes assessment:

Phase 1 Implementation Progress. March 2016
Rating n %
Green | Meets expectations, department can implement 164 47%
Yellow | Minor revisions necessary before implementing 169 49%
Red | Major revisions necessary before implementing 13 4%

As ratings were assigned by the assessment working team, feedback was applied directly to the assessment and action plan and curriculum map documents. About half of the academic programs (52%) executed their assessment and action plans during that first full academic year. Across the schools/colleges, those with specialized accreditation had the highest percentage of programs implementing (Education, 95%; Engineering, 85%; and Falk, 93%). The team developed a rubric to assess the quality of the assessment and action plans and curriculum maps provided by the academic programs. The rubric detailed the University’s expectations and levels of accomplishment for the various dimensions of the assessment and action plan using a four-point scale in which 3=proficient, 2=developing, 1=emerging, and 0=no evidence. The assessment working team provided scores and comments for strengthening each of the plan components. Of the 164 academic programs that documented assessment efforts, rubric scores were distributed as follows for results, interpretation, action, and follow-up:

Rubric Score Results Interpretation Action &

Follow-Up
n % n % n %
3.0 | Proficient 45 27% 47 30% 2 1%
2.5 16 10% 5 3% 0 0%
2.0 | Developing 68 41% 67 42% 112 77%
1.5 6 4% 0 0% 2 1%
1.0 | Emerging 29 18% 39 25% 29 20%

The primary area for improvement in the “Developing” and “Emerging” categories was in the documentation of results, interpretation, and actions. Often, not enough detail was included. Programs were advised to ensure that faculty were giving consideration to and documenting both what students were doing well and where they were struggling—especially those students who did not meet the program’s criteria for success. The two resources returned to faculty (the rubric and strategies for documenting student learning outcomes assessment) provided guidance for how to strengthen documentation.

During the spring 2016 semester and into the summer, the assessment working team met individually with program contacts or in small group settings depending on the preference of the department. The meetings allowed the assessment working team to review the feedback, share best practices for assessment, and offer suggestions for next steps in plan implementation with faculty. Some meetings occurred too late for programs to be able to collect student work, and faculty were advised to provide plan revisions in the upcoming reporting window.

The fall 2016 annual progress reporting period saw an increase in the number of programs providing reports on time, and the assessment working team’s outreach efforts resulted in updates from most of the remaining academic programs throughout fall 2016 and winter 2017. Faculty received rubric scores via campus mail in February and March 2017. Altogether, 314 assessment and action plans were scored, and the overall results for Phase 1 components, as well as curriculum maps, were positive as shown in the table below:

Outcomes Curriculum Map Measures Criteria
Rubric Score n % n % n % n %
3.0 | Proficient 228 73% 168 54% 171 54% 135 43%
2.5 9 3% 1 0% 10 3% 6 2%
2.0 | Developing 68 22% 108 34% 89 28% 138 44%
1.5 0 0% 0 0% 7 2% 13 4%
1.0 | Emerging 9 3% 13 4% 31 10% 9 3%
0.0 | Not Evident 0 0% 24 8% 6 2% 13 4%

Strengths seen across academic programs included well-developed student learning outcomes written with active verbs, emphasis on direct measures of student work, and benchmarks for determining student achievement of the outcome.

Also during this annual progress reporting period, Data Driven Change Reports were provided by 35 teacher prep programs in the School of Education in place of updated assessment and action plan templates. These reports were prepared for CAEP assessment and demonstrated efforts to use relevant student data to inform decision-making. These reports were not scored on the rubric. Phase 1 of the assessment and action plans for teacher prep programs were reviewed in the prior year and consistently received top ratings.

A heat map was created based on the rubric scores to provide a snapshot of the quality of the plan and implementation efforts across the schools and colleges (V.1.005.Academic_Program_Assessment_Tracking_Rubric_F2017, V.1.006.Co-Curricular_Functional_Assessment_Tracking_Rubric_F2017. The assessment working team uses the heat map to determine at a glance where additional support should be offered.

The most recent annual progress reporting window was closed in October 2017. Overall, 359 academic programs (97%) completed an annual progress report and submitted an updated assessment and action plan and curriculum map, if revised. During the upcoming spring 2018 semester, the assessment working team will be reviewing the updated plans and curriculum maps and scoring them on the rubric, returning feedback to faculty.

Most of these academic programs (94%) documented implementation efforts. This is a large increase in the percentage of programs conducting and documenting program-level assessment over the previous year (from 52% to 94%). Implementation rates by school/college increased across the board, with most engagement rates ranging between 93% to 100%. Rates were lowest for the iSchool and College of Law; both have a small number of programs, and one non-implementing program resulted in lower rates.

  Implementing
School/College n %
Architecture 2 100%
Arts & Sciences 109 93%
Education 56 97%
Engineering 27 93%
Falk 34 100%
iSchool 5 83%
Law 4 80%
Whitman 15 100%
Maxwell 42 100%
Newhouse 20 95%
VPA 52 98%
UC 5 100%

Several programs (n=41) are not included in these numbers; the programs did not provide an updated assessment and action plan as data collection was on hold for one of the following reasons: the program was new, under review, closing, or had very low/no enrollment during the 2016-17 academic year.

Faculty responses to questions on the annual progress report provide strong evidence that assessment processes have been helpful and will be sustained by academic programs. Three forced-choice items in particular offer insight into how assessment processes are working within academic programs:

  • 46% of the respondents reported that all program faculty participated in discussions about assessment results; 29% identified a majority of the faculty.
  • 52% strongly agreed and 32% somewhat agreed that collecting evidence of student learning outcomes and discussing results led to informed decision-making for the program.
  • 68% strongly agreed and 21% somewhat agreed that assessment processes are a regular part of their program’s functioning.
  • 91% of respondents reported that results and possible actions have been communicated to faculty, 51% communicated results and possible actions to staff, and 33% communicated results and possible actions to students.

Faculty comments revealed how the assessment process has been helpful and included reflections on how efforts can be improved. Some programs, particularly those with specialized accreditation, noted that assessment has been and will continue to be a critical, core activity. Responses suggest that faculty newer to program-level assessment have found the process to be informative in different ways, including increased faculty engagement; helping faculty better articulate what they want students to learn in the program; exposing faculty to different teaching styles; and highlighting where things are working well, where there are gaps in the program, or where courses are not functioning as intended. Sample comments include:

  • As a program subject to specialty accreditation, we are (and have been) required to complete this type of assessment for many years.
  • Following your assessment guidelines, particularly rubric development, helped us surface some issues with [a course] that we did not realize were issues. We had particularly fruitful discussions about the assessment results with the students who were in that class last year—who are now juniors. Our faculty advisory board is going to contemplate making some program changes on the basis of their suggestions. Having them help us interpret the assessment meta-data led to a discussion that we might not otherwise have had.
  • Formal assessment was already built into our graduate program, so this new assessment model mainly confirms what we already had seen and/or knew about our program.
  • In that our faculty actively take part in the teaching of the [graduate program] curriculum and serve on their thesis committee, we find that learning outcomes directly translate. Further, we have taken time at the end of the thesis defense to reflect on how we can improve the process and support the students to achieve better thesis efforts.
  • None of our programs have ever been assessed, and in some sense we are working from square one. The mandate to assess formally has provoked considerable reconsideration of our program goals, and this is a positive for our department (and our programs).
  • Our first assessment led to changes in the program already. The process is working very well.
  • Since we are an accredited program, curriculum assessment is ongoing for our program. We have an advisory board meeting held annually each fall where many alumni are in attendance. Our curriculum actions will be conveyed to them at that time.
  • The [program] is going through a transition of hiring new faculty. These new faculty have already greatly contributed to the discussion on assessment, providing both an “outsider” and “insider” view at the same time. Level of dialogue has been increased by more frequent faculty meetings and strategy sessions.
  • The assessment provided a good opportunity for faculty from different departments who contribute to this interdisciplinary program to collaborate on assessing its goals. We are now implementing this process as part of the program’s regular functioning.
  • The assessments have helped us think carefully about what we want the students to learn while in the program. We should involve the students more in our discussion of the assessments.
  • We are getting more comfortable with this process. We recently moved responsibility for assessment at the undergraduate level from an ad-hoc departmental committee to the Undergraduate Committee. This will give the process more stability.
  • We’re making progress. There is interest in developing better processes and tools for doing assessment—developing better rubrics, for example.
  • While the department and its programs have not had regular assessment practices in recent years, the faculty as a whole have embraced the task with a considerable level of buy-in and wide participation. Guidance from relevant University, college and department leaders and offices has been instrumental.

Faculty also identified areas to address or challenges they have encountered, including engaging more faculty in the process, assessing programs with low enrollment, lack of administrative support, assessing programs across departments and/or the schools/colleges, making assessment efforts manageable and decision-making timely. Representative comments include:

  • Despite numerous requests for a higher authority to facilitate, there is still no means to convene faculty from across colleges to discuss assessment of jointly offered graduate programs.
  • [Faculty] got involved in the assessment process only during the implementation phase. For the rubric design and data-processing faculty does not get involved with the exception of the Program Coordinator. It will require great amount of effort to continue the assessment process in the future if faculty does not get fully committed to the process.
  • Our department has been doing a form of assessment on student work for a number of years now. This has led to a major overhaul of our curriculum this year. So we believe in the overall concept. Just wish the process wasn’t as complicated as it is now.
  • There are currently only three students enrolled in the [minor]. Those three students are meeting program expectations, but the sample size is too small to reach substantive conclusions about actions to be taken.
  • There is no doubt that much of the information we’ve gathered is helpful. But at this point in time we have been too consumed with the process of gathering and reporting the data to have meaningful discussions about any significant changes we might make to our program. It should also be pointed out that in the [industry] in general, and [area of study] specifically, change is always coming at a fast and furious pace. We are forced to make many decisions on the fly based on feedback from professionals (including former students) and our own observations. These are the types of decisions that are a priority for us to make sure our students are employable, something we do very well and take great pride in.
  • This is the first time our program did the assessment. We do not have any help from staff because the program does not have any budget. No faculty other than the director (me) is responsible for reporting their students’ performance for assessment.
  • We conduct a department “retreat” twice per year where we discuss problems with individual courses or with the curriculum overall, and then take steps to collect more data or to make changes. The meetings help us to maintain quality, although it is a challenge to get all the faculty to attend. We are attempting to build a “culture of assessment.”
  • We regularly discuss our programs, courses, and students based on observations and assessment data. These artifacts help us to make decisions about student progress, program updates, resources uses, etc. We note that our challenges are not related to our program courses, requirements, or assessment data, rather a lack of marketing support in a quickly growing, high-demand area. We are understaffed (administrative and faculty) to do more with our programs. We engage students in small efforts to prepare messaging, gather the more labor-intensive artifacts (e.g., publication records, etc.) from our students, but this type of a process is difficult to maintain as students graduate and move on; it would be helpful to have support to keep our data collection processes working consistently and keep up on data collection.
  • We will only be collecting data on two learning outcomes for the next year. This year, we collected for all (8), which was a bit overwhelming.
  • We’ve always done a kind of assessment each year, but it was informal and not guided by advice from experts in assessment. This first exercise was especially worthwhile. In the future, we probably ought to include a larger circle of faculty.

To summarize the foregoing material, the Office of Institutional Effectiveness and Assessment has primary responsibility for developing, publishing, distributing, and promoting assessment guidelines as shown in these supporting documents:

Likewise, the office provides assessment and action plan templates for all degree and certificate programs:

Finally, the office also provides templates for and promotes the development of curriculum maps for relating student outcomes to specific courses (V.2.004.Academic_APP_Curriculum_Map).

Three phases of activity govern the processes of documenting and reporting assessment and action plans (V.2.005.Academic_Programs_Rubric_AAP_Reviews):

  • In Phase One, the desired learning outcomes are listed. These change from time to time as necessitated by assessment results. Courses in which learning outcomes are either introduced, practiced, or reinforced are mapped to each outcome. Measures of performance indicators are stated, and criteria for classifying levels of outcome achievements are provided.
  • In Phase Two, the results of assessments identified in Phase One are reported and interpreted.
  • In Phase Three, actions for follow-up and to improve outcome achievements in the next assessment cycle are described. This three-phase cycle of assessment and continuous improvement is repeated annually. While all assessment plans are available for reviewers in Tk20, the following documents provide examples of the three-phase process:

On a related note, the Office of Institutional Research and Assessment (OIRA) supports the course evaluation process, which contributes indirect measures to the assessment planning process. In addition to standard questions available for course evaluations, instructors and departments can arrange to include custom questions. Course surveys are administered online in the last week of the semester and also after key stages of the class upon request (note that individual instructors can and do build formative evaluations in Blackboard). The results of course evaluations are provided to the respective department and college as well as to the course instructor, in accordance with academic unit policies and practices (V.2.010.OIRA_Student Ratings) to plan future course improvements and professional development, and to guide instructional staffing considerations.

Consistent with the National Association of Colleges and Employers standards for collection of career outcomes surveys, academic programs collect and provide feedback data to articulate how they prepare students for successful careers, meaningful lives, and, where appropriate, further education (V.2.011.Career_Services_Outcomes_Report). Syracuse University supports and sustains assessment of student achievement both centrally and within schools and colleges, and we communicate the results to stakeholders on program websites and in program overview materials (e.g., V.2.015.Placement_iSchool; V.2.016.Placement_iSchool_MS). These assessment results are also shared with school/college advisory boards, student and alumni groups, and with accreditation entities and government agencies in appropriate formats. Assessment results are reviewed by undergraduate and graduate committees, student and alumni advisory boards, accrediting agencies, and department and college administrations. These groups also periodically review the condition of laboratories and information technology capabilities as well as internship and study abroad program outcomes. Employer surveys, job placements, graduate school admissions, program rankings, and performance of students on standardized tests such as Fundamentals of Engineering (FE), Graduate Record Examinations (GRE), Graduate Management Admission Test (GMAT), and others serve as indirect measures of program performance. These outcomes and assessments inform prospective students and are shared with stakeholders to elicit feedback for continuous improvement.

In the academic realm, these processes help to ensure that the University has defined meaningful curricular goals with defensible standards for evaluating whether students are achieving those goals. In co-curricular and functional units, these processes help to establish and build a culture of continuous improvement.

Criterion 5.3 relates to the use of assessment results. Changes in the assessment infrastructure, curricular improvements, new professional development activities, adjustments to the budget system and other changes provide clear evidence that Syracuse University makes use of assessment results to improve the institution in various ways.

Syracuse University initiated a survey of academic programs in 2015 to clarify how student learning outcomes are assessed and how departments respond to the data generated from these assessments to enhance student learning (V.3.001.SLO_Survey; V.3.002.SLO_Survey_OE_Results). This effort revealed that the majority of programs engage in ongoing assessment activities via student academic performances, course evaluations, and faculty feedback. These results typically inform changes within the curriculum but may also include co-curricular responses (V.3.003.WGS_SLOs_2015-08-27). Findings from the SLO survey were reviewed by the University Assessment and Accreditation Committee (V.3.004.SLO_Survey_Explanation_UAAC_Meeting) and resulted in a concentrated effort to educate departments about a more structured process of delineating outcomes, outcomes assessment, and documentation of assessment-based change. This process is managed by the Office of Institutional Effectiveness and Assessment.

On the basis of the above-mentioned developments, Syracuse University’s academic programs have consistently improved curricula and pedagogy using assessment data and other feedback. The academic program change process requires that both course and curricular changes be documented by the department requesting the change and appropriately justified to the department’s curriculum committee as well as the Curriculum Committee of the University Senate. Documentation of this process (V.3.005.Academic_Programs_Annual Report_F2015_Change_Data) contains phrases such as “AAS 525 … was not enough for master’s-level proficiency… ” or “the curriculum was updated in relation to learning objectives … ,” which demonstrate both the process by which assessment results are considered and that they lead to curricular change. This same document provides evidence that curricular and pedagogical changes have also been implemented in response to student feedback from surveys and from major advisors, faculty, and alumni with knowledge of changing fields and job market pressures, interviews with students and faculty, and work with outside consultants.

A number of changes have been instituted in the planning/budgeting process in response to feedback from academic programs. The parameters of the Responsibility Center Management (RCM) budgetary system have been adjusted to allow more money to be returned to the individual schools and colleges to give them more control over how resources are deployed for curriculum. In the new model, deans are permitted to negotiate cost sharing and revenue generation for programs/classes that are collaborations across academic units. This change enhances interdisciplinary efforts and addresses concerns related to limited ability to execute curricular changes indicated in the assessment processes and in student feedback.

Our RCM budgeting system also helps to ensure that professional development activities have been sponsored directly by the Office of the Provost as well as by other academic and support units in response to faculty requests and needs documented at the University and nationally. For example, based on faculty requests for pedagogical support, Syracuse University has, since 2012, funded a faculty summer “boot camp” for integrating technology into the classroom (V.3.006.SITETL), the Summer Institute for Technology-Enhanced Teaching and Learning (SITETL). SITETL is a weeklong activity that includes faculty with specific projects from all schools and colleges that results in ongoing engagements with instructional technologists. Syracuse University’s ITS group offers orientation opportunities to all new faculty through the Office of Faculty Development. Syracuse University provides classroom and auditorium orientation upon request at the start of every semester. Syracuse University provides classes on making content accessible to all employees. Annually, the University hosts a two-day retreat for department chairs to generate discussion and to assist chairs in leadership initiatives that have been determined to be priorities for the University.

The Office of Institutional Effectiveness and Assessment offers the “Focus on Assessment” workshop series wherein assessment professionals from around the U.S. offer half-day and full-day workshops on campus each semester.

The Graduate School’s Programs Office also has a strong focus on increasing the effectiveness of teaching and learning. Programs such as the Future Professoriate Program, teaching assistant orientations, and annual seminar series for the Certificate in University Teaching (CUT) help current graduate students, fellows, and teaching assistants grow as instructors and members of the University community. The Programs Office performs an in-depth qualitative and quantitative assessment after each year’s orientations for all teaching assistants. Assessment results are used to introduce new orientation approaches and topics. Assessment of international teaching assistants’ classroom language skills is used to make specific recommendations and provide services to those who need further work on their English.

The Office of Institutional Research and Assessment provides retention and graduation rate data as well as analyses examining the impact of Syracuse University experiences on student success (V.3.007.Orange_Survey_Initial_Findings, V.3.008.Decrease_in_GPA_Predicts_Next_Semester_Departure_Report, V.3.009.Ungraduated_Seniors_Report, V.3.010.Transfer_Courses_and_Later_Grades_Report) (IV.6.005.Grad_Rates_SSUI_Participation,VI.8.002.SU_Abroad_AAPlan). Initiatives such as Orange SUccess and Degree Works stem directly from these data and the resulting interest in improving student experiences.

Criterion 5.4 is concerned with review of assessment services designed or delivered by third-party providers. Syracuse University does not outsource assessment. Any third-party partners who provide assessment of some kind are limited to accrediting bodies, licensing exam organizations, or optional certifications such as the Microsoft Excel certification required by the School of Management, with whom we choose to engage precisely because of the external assessment they provide.

Some faculty use TurningPoint Polling technology and Turnitin as tools to facilitate assessment. The University also provides inline grading for instructor assessment of digitally submitted papers through the use of a third-party application (Crocodocs; V.4.001.Inline_Grading) In each case, the tool provides information; faculty decide by what standard the information provided translates to an assessment. For example, Turnitin highlights elements of a paper that appear to arise from another source, but faculty members make the decisions on whether students have used their sources appropriately.

Criterion 5.5 discusses periodic assessment of our assessment processes. Using both internal assessment professionals and outside experts, Syracuse University has engaged in periodic assessment of the effectiveness of assessment processes utilized to improve educational effectiveness. Linda Suskie, a leading assessment expert and former MSCHE vice president, spent a day on campus in November 2016. She conducted a morning workshop for faculty and staff on understanding and using assessment results and met in the afternoon with the Office of Institutional Effectiveness and Assessment staff and a representative from the University’s self-study Team 5 to review and assess the University’s assessment efforts. Prior to her visit, Suskie examined a cross-section of assessment and action plans, as well as resources, including guides and templates.

Based on her review (V.5.001.Suskie_Report), Suskie noted that the University’s assessment efforts were quite good, and she indicated that the guidelines are among the best she has seen. Suskie was impressed with the turnout for the workshop and attendees’ level of engagement. She appreciated the provost’s assessment message to the deans, as well as the University Senate’s requirements that course objectives be identified for new courses, new courses be mapped to program-level student learning outcomes, and an assessment and action plan be developed for proposed programs.

The assessment system that Suskie reviewed was developed in recent years as the result of a determination that the existing process was insufficiently systematic prior to 2014. As described above, in fall 2014, an assessment working team was formed to develop processes and resources that would guide schools and colleges, as well as co-curricular programs/units and functional areas, in creating and implementing assessment and action plans. Shortly thereafter, this working team became the standing steering committee for the Office of Institutional Effectiveness and Assessment. This group thoroughly researched assessment best practices and assessed the University’s past practices, current needs, and regional and specialized accreditation requirements. This group also determined that the University’s earlier assessment documents were comprehensive but complicated to complete; new templates, incorporating some of the previous content as well as new information learned about best practices, were created and piloted before being distributed widely. Requested documents for academic programs now include an assessment and action plan template and curriculum map. Separate templates were created for co-curricular and functional areas.

In spring 2015, Barbara Walvoord, professor emerita at the University of Notre Dame and a leading assessment expert, conducted four workshops on campus and spoke with faculty participants about student learning outcomes assessment. At the conclusion of her visit, she met with the assessment working team to provide feedback on the University’s assessment efforts. Walvoord identified three priority areas but recommended one in particular—supporting faculty in the development of academic program assessment and action plans—as the highest priority because it serves as the foundation. She pointed out the need to specifically provide assistance with determining criteria as she surmised this was an area that might be challenging for academic programs. With that in mind, the assessment working team has met with and supported faculty in a variety of ways as assessment and action plans were developed. Feedback on how to enhance or strengthen assessment efforts was provided to each academic program primarily through meetings with an academic program’s designated contact, a subset of faculty, or the entire faculty through the spring and into the summer.

Areas of Continuous Improvement and Recommendations

Criterion 5.3 is concerned with use of assessment results, and Criterion 5.5 focuses on periodic review of assessment processes. Optimal use of assessment results requires a sufficient number of assessment cycles to be able to identify patterns and assess the effectiveness of changes made in response to previous assessment results. Prior to 2014, our assessment process was uneven, and as a result documentation of the assessment cycle was correspondingly weak. Since 2014, the Office of Institutional Effectiveness and Assessment has proven highly effective correcting this and ensuring we have a robust system in place. Using the Tk20 assessment software for tracking goal alignment and reporting will become particularly useful when we enable all units on campus to review their own assessment records via self-service. The efforts of the Office of Institutional Effectiveness and Assessment, combined with systematic use of Tk20, positions us well organizationally to ensure assessment takes place over time with full participation, and that findings are used for the improvement of student learning, curriculum, student services, budgeting, and professional development. Recommendation 1 in Chapter 5 addresses both the periodic review of assessment processes and the use of assessment results.

Evidence