By Jennifer Kishida, junior specialist, Department of Social Work, University of Hawai‘i at Mānoa

Before 2018, it is likely that every student in the PhD in Social Welfare program at the University of Hawaiʻi at Mānoa dreaded the qualifying exam (QE). The QE is the first of four benchmarks the program uses to measure student learning. After passing core courses, students take the QE to demonstrate their ability to accomplish the following program learning objectives:

1. Analyze and apply social welfare theories, research findings and research methodologies to resolve critical social welfare problems;

2. Formulate relevant research questions, and apply appropriate research methods in culturally-appropriate research design; and

3. Understand and analyze social welfare policies and their impact on social work practice within communities and populations in-need. 

The QE consists of four sections – knowledge development, policy analysis, qualitative research, and quantitative research. Concerns about students’ performance in the quantitative research section of the QE had been a repeating theme in faculty meetings and curriculum discussions. Students were also most anxious about this section of the QE. These, however, were all anecdotal.

In the summer of 2018, the program assistant assessed the level of student learning evidenced through the QE during the years 2010-2014 and made the faculty aware of alarming findings. Of the 12 students who took the quantitative research section of the QE during this period, none passed on the first try: half received a conditional pass and the other half failed. This was no longer anecdotal information passed between faculty and students. This was evidence that students were struggling to learn a part of the curriculum and/or demonstrate their learning sufficiently.

Faculty had already been thinking of ways to better support students in quantitative research. The assessment finding urged faculty to take immediate actions to improve student learning through curriculum alignment. The Ph.D. Committee took on the main charge of developing interventions. The committee was composed of six core faculty members and one Ph.D. student representative.
During Ph.D. Committee meetings, members discussed the curriculum and its alignment with the content on the Quantitative Research QE. The faculty recognized that

1. Taking the two quantitative methods courses in the same semester may be more beneficial than taking them in separate semesters.

2. The QE questions and study guides could be aligned better with the content of the QE-related courses. Some students had taken QE-related courses outside of the discipline (most often the Department of Educational Psychology’s variety of quantitative methods courses).

3. Students must finish taking all courses before being allowed to take the QE which could result in two years between the QE-related courses and the QE. The timing of the QE could be improved by interspersing portions of the exam closer to the  completion of related courses.

After identifying the potential roadblocks for students’ successful achievement of the QE skills and completion of the QE, the committee members collaboratively designed the following interventions to better align the QE and the curriculum:

1. The Ph.D. Committee changed the curriculum schedule so that students took both quantitative methods I and II in their first semester, then a multiple regression course in the second semester.

2. Each year, instructors of the core courses reviewed and updated QE study guides and questions. Study guide practice questions were based on syllabi content with the intent to support student preparation. The QE questions were updated as needed. In the case of the QE quantitative research section, QE questions were updated to move away from statistical knowledge testing towards applied methods thinking. Our program connected with instructors who taught equivalent core courses in other fields to ensure that the QE aligned with what they taught.

3. Logistics of QE scheduling and administration were adapted so that portions of the QE are administered each summer after the completion of the related course(s).

The Ph.D. program implemented these interventions in 2018. As a result, the assessment data from 2018-2021 showed that of the 13 students who took the QE quantitative research section, 11 passed on the first try, one student received a conditional pass, and one student failed, a remarkable improvement in comparison with the 2010-2014 results. Recall, none of the 2010-2014 students passed on the first try.

The assessment process proved to be informative to the program about an area where student support could be improved. The program committee members gained a better understanding of the QE procedures. The faculty collaborative discussions of assessment data, problem-solving, and actions to enhance curriculum alignment and sequencing proved to be effective for improved student performance.