CAAC Assessment

CAAC Assessment Overview

CAAC is proud to foster a culture of assessment in the undergraduate advising environment. Our assessment processes aim to:

Help us better understand student learning and development that take place in the context of academic advising as well as other programs housed in CAAC
Aid our monitoring of short- and long-term program effectiveness
Support leadership and other stakeholders in evidence- and data-based decision making
Identify programmatic areas requiring attention or development
Promote our commitment to continuous program improvement

Currently, comprehensive assessment plans are in place for the First Year Scholars Program (FYSP) as well as REACH virtual advising. Program evaluation and assessment workshops are also offered through ACE in order to support program evaluation and assessment efforts by colleagues across campus that provide vital student support and services.

Please direct any questions about our assessment processes to:

Tiffany Tsang, PhD
Assessment Specialist & Academic Advisor
ttsang@college.ucla.edu

FYSP Assessment

Background

Since its inception in AY19-20, roughly 50-60 students have completed the FYSP program every year. As of Spring 2022 more than 150 students have completed the FYSP program in its entirety. In addition, yearly program evaluation and assessment cycles have been established to capture student experiences with the program and determine areas for continuous improvement.

In Winter 2022, a comprehensive assessment study was launched in order to investigate the impact of FYSP on students in regard to the program’s student learning outcomes (SLOs), which were developed in accordance with higher education literature on undergraduate engagement and success, as well as areas important to the University, such as academic success and progress toward degree. A retrospective pre-/post-test survey was administered to each FYSP participant group as well as respective “control” groups of students selected from those who were invited but ultimately did not matriculate into the program and whose pre-matriculation characteristics resembled that of the FYSP participant groups.

Impact of FYSP on Student Learning Outcomes (SLOs)

Findings indicated stronger gains in all SLOs for all three FYSP participant groups compared to their respective control groups (Table 1). As such, participant groups’ combined average gains were also higher than the control groups’ combined average gains. For “Knowledge and use of campus resources,” participant groups’ combined average gain from pre- to post-test was 1.65 whereas the control groups’ was 0.41 (on a scale of 1-5). This pattern held true for “Relationship-building with peers, staff, advisors, and faculty” (1.29 vs. 0.28), “Sense of belonging at UCLA” (1.15 vs. 0.25), “Feeling academically supported” (0.80 vs. 0.11), “Identity affirmation and development” (0.79 vs. 0.23), “Academic engagement” (1.03 vs. 0.33) and “Resilience” (1.04 vs. 0.35) (Table 2).

Qualitative data comprised of student survey comments also revealed how impactful the program was in areas such as adjustment to college, academic confidence, and academic engagement. Student comments also indicated that FYSP continued to positively impact students beyond the first year by helping them establish a growth mindset and long-term resilience. In the words of one student from the AY19-20 (Year 1) cohort:

“I believe the start to anything in life makes a difference in how we’re able to embark on future endeavors. Being in FYSP provided me with the resources I needed to excel academically, and I think that pretty much set the tone for the following years.”

Impact of FYSP on Academic Success & Progress Toward Degree

In order to further investigate the impact of the program in areas of particular interest to the University, FYSP participants were also compared to control groups in regard to their UCLA GPA, cumulative UC units, and major declaration status. UCLA GPA was used as a proxy for academic success and cumulative UC units and major declaration were used as proxies for progress toward degree.

Findings indicate that FYSP participants on average had higher UCLA GPAs, cumulative UC units, and rates of timely major declaration (with the exception of Year 2) when compared to the control groups (Table 3). Such findings suggest that FYSP participation has a positive impact on students’ academic success (represented by higher GPAs), and overall timely progress toward degree (represented by higher cumulative UC units and major declaration rates). The long-term positive impact of FYSP is also suggested by participants’ continued higher GPA and UC cumulative unit completion compared to that of their respective control groups both one and two years after program completion.

Conclusion

Assessment findings suggest that FYSP has a highly positive impact on students in regard to SLOs associated with undergraduate engagement and success as well as areas of high importance to the University, such as academic success and timely progress toward degree.

Findings also reveal the ways in which FYSP has contributed positively students’ meaningful engagement and transition to college and perhaps most importantly, the lasting impact of the program on students after program completion.