Tama Hasson

Words like assessment have become part of the necessary lexicon of Program Directors running undergraduate research programs and other student development programs designed to serve underrepresented students. National agencies that fund student programs now mandate assessment in their Requests for Applications. To meet this mandate, mechanisms need to be designed to measure and quantify the goals of a program and to identify which program components are contributing effectively to student success. ASSESSMENT OF SCIENCE RETENTION PROGRAMS: The Leaky Pipeline The nation's competitiveness is intimately linked to innovation in science and technology. However minorities are not being retained in science. While Hispanics constituted 17% of the 18-24 year olds in the US in 2008, only 9% of the bachelor's degree recipients in science, technology and mathematics (STEM) in 2009 were awarded to Hispanics. This percentage decreases as you look at MS and PhD recipients. And significantly only 3% of the STEM workforce in the US for the period 2004-2010 was Hispanic. My research has focused on dissecting this leaky pipeline. Why are minorities not succeeding in science? The first question was how to best evaluate the programs on the UCLA campus that strive to improve retention of minorities in STEM fields. Dr. Linda DeAngelo (Former Assistant Director for Research at the Higher Education Research Institute) and I completed a study as to the best methods to create control groups to assess the impact of minority retention programs. We used as the study subject a program administered by the Undergraduate Research Center-Science called PEERS (Program for Excellence in Education and Research in the Sciences) (DeAngelo & Hasson, 2009). PEERS is a two-year program that supports underrepresented physical and life science majors that have had more than typical life challenge hurdles to overcome to reach UCLA. PEERS' goal is to bring these students into the scientific community by helping them maintain a science major and encouraging them to try undergraduate science research. The PEERS program involves individualized academic and career counseling, "Pathways in Science" seminars tailored to freshman and sophomore needs, collaborative learning workshops that accompany their introductory math, chemistry and physics courses, research seminars by UCLA professors, and the opportunity to both be exposed to and to be deeply involved in undergraduate research. Assessing a program like PEERS is difficult because the program encompasses so many different components. In our study we show that by using post-matching control groups we can assess both short-term outcomes (grades) and intermediate-term outcomes (entry into research). In the last five years, in collaboration with Dr. Marc Levis (Director, Center for Educational Assessment), his graduate student Brit Toven-Lindsey, and Dr. Paul Barber (EEB Department), we have followed up on the above study with a longitudinal controlled study of PEERS (Toven-Lindsey, B., Levi-Fitzgerald, M., Barber, P. and T. Hasson (2015) Increasing persistence of undergraduate science majors: A model for institutional support of underrepresented students. CBE - Life Sci. Educ. 14: 1-12). Focusing on students that entered UCLA in 2009 and 2010, we compared PEERS students to similarly prepared students during their first two years at UCLA. Our study compared the PEERS group to three different control groups, thereby eliminating selection bias and other confounding factors. We found that PEERS students take more science classes in their first two years (so are more likely to graduate on time), earn better grades in those science classes, have a higher overall GPA and are more likely to be in a science major at the beginning of their third year. This benefit is seen whether PEERS students are compared to a propensity matched control group of similarly prepared students, or better prepared students with high incoming Math SAT scores. Engagement in PEERS clearly improves academic success and retention in science, eliminating the achievement gap for URM and other underserved science students. Role on publication: I served as the leader for this project. I directed Ms. Toven-Lindsey, managed creation of the figures and wrote the drafts of the manuscript. I also served as the corresponding author. The Toven-Lindsey et al (2015) publication has had a number of significant impacts. Firstly - UCLA has been approached by a number of institutions interested in reproducing PEERS on their own campus. For example the CSU system contacted me directly to work to implement PEERS on all 23 CSU campuses. In addition this publication has had a large campus impact as described below. In spring 2016 we had over 460 applications for the PEERS program, up from an average of 260 applications. This increase in interest is in many ways due to the change in the demographics of those accepted to UCLA. UCLA accepted our largest entering freshman class in 2016 and many were California natives coming from lower performing high schools. With the knowledge that student base for PEERS had grown substantially, we turned to the Executive Vice Chancellor and asked for support to increase the size of PEERS. The budget for a doubling of PEERS was accepted and as of Fall 2016 we have 180 PEERS freshmen, up from our usual cohort of 80. This is an exciting change for PEERS, and will mean that in future years PEERS will have an even larger impact on the pipeline. FUTURE DIRECTIONS: PEERS has had a significant impact on the academic success of its students. We are now continuing this study following the 2009/2010 cohort of PEERS and matched control students through graduation to investigate the impact of PEERS on entry into research, retention in science, and graduation rate. We have mined senior survey data, as well as data on research enrollments, and participation in URC-Sciences research activity. This information will be paired with outcomes data gleaned from the National Clearinghouse. Our preliminary analysis has revealed that PEERS student do enter into research at a much higher rate, and are more likely to receive research funding. In addition there are a higher percentage of PEERS students retained in a STEM field as compared to control groups. While the analysis is not yet complete, we will be presenting our preliminary findings at the American Society for Cell Biology Annual Meeting special interest subgroup session on "Science with undergraduates". ASSESSMENT OF LEVEL OF MATH ENTRY AND OUTCOMES IN THE LIFE SCIENCES My analysis of PEERS led to an observation that underrepresented students who began at the Math 1 level were less successful than students who began at the Math 3 level when it came to retention in science and grades in introductory chemistry courses. This observation led to the Doris Duke Foundation funded project to analyze the impact of Math entry status on all UCLA life science students. In collaboration with Brit Toven-Lindsey and Erin Sanders, we looked at all students who entered in Fall 2009 or Fall 2010 and took Math 1 or Math 3A as their first math course in the fall. We compared a number of outcomes, but had the significant finding that students in Math 1 received a full one grade lower in their Chemistry 14A course than the students in Math 3A (when they took the Chemistry course simultaneously with their Math course). 50% of Math1 students are URM, whereas only 25% of Math 3A students are URM. So this issue impacts URM students with greater severity. Based on our analysis we suggested that Math 1 students should not take Chemistry 14A concurrently and should wait until Winter to take it. We also showed that students who had done this in the past, had fared better in their Chemistry course. This change was implemented as a "strong suggestion" at orientations for students starting Fall 2014. And as of now, there is now a new policy where students will have Math 3A co-requisite with Chemistry 14A. This will ensure that all life sciences students have at least a basic level of math skill, and should produce students who are more confident and better prepared to take Chemistry. I am particularly proud of this study as it shows how my work with PEERS can be applied to the entire campus for the betterment of all students. These changes have now been applied to Physical Sciences Majors (Math 31A is now co-requisite with Chem 20A) and as part of my membership of the Physical Sciences Diversity Committee, we are now taking a close look at all of the Math and Chemistry series, to determine whether improvements can be made in teaching for these core courses. DISSEMINATION OF BEST PRACTICES The Amgen Scholars Program is a summer research program supported by the Amgen Foundation evaluated by the Center for Evaluation & Education Policy at Indiana University. As part of a dissemination plan for best practices, the Amgen Foundation has supported the creation of a book which highlights what the Amgen Scholars sites have learned as to how to create and maintain a high quality summer program. I have contributed three chapters to this book. The first, written in collaboration with Dr. Patricia Phelps, focuses on how to create an application for a summer program. The second, written in collaboration with UC Berkeley program director, Audrey Knowlton, focuses on how to orient your student to your summer program in their first days on campus. The final chapter focuses on workshops and seminars that have been shown to be useful accompaniments for summer research programs and that also help connect your visiting students to your campus as prospective graduate students. All three chapters highlight what we do at UCLA. The book will be available in a print and on-line to serve as a foundation for creating a new summer research program.