Dr. Jose Palma

“How do we maintain a culture of evidence that supports equity?” Dr. Palma posed this question at the start of our interview with him. As an ACES assistant professor at Texas A&M University, Dr. Palma tackles the big questions facing higher education measurement today. His toolbox doesn’t look that different from any assessment professional (i.e. he asks similar questions), but it’s the mindset with which he approaches these questions that informs and centers equity into his conversations with other stakeholders.

Inclusion of diverse stakeholders in conversations along the entire continuum of a project is essential. Diversifying stakeholders allows questions to emerge that may have previously been overlooked. For example when working at University of Texas-Austin as a postdoctoral researcher, he included undergraduate students on his research team in creating a tool measuring global skills development, sense of belonging, and positive self-identity for study abroad programs. Students guided the process by asking questions about study abroad learners’ demographic characteristics (ie. race, gender, first-generation, etc.) impacted study abroad experiences differently than other students. Their curiosity ensured that when questions were developed and piloted, diverse participants were included. Stakeholders also guide the presentation of data and summary reports created to ensure that the presentation of information is laid out in a way that works for the end-user. 

Transparency is another key factor in centering equity. In analyzing data, decisions are constantly made and being transparent in those choices, why they were made, and implications/limitations of these choices is important. For example, small n’s cause many assessment professionals to aggregate participants. This may be aggregating undergraduate upperclassmen from lowerclassmen or other demographics. Dr. Palma provided an example of aggregating student survey responses from various Caribbean countries as there were not enough of any one country to draw conclusions. Aggregating makes sense if the research maintains transparency about this decision and identifies the limitations of doing so. Dr. Palma says it’s about creating balance for the psychometric properties and being inclusive. Dr. Palma clarifies that culturally inclusive assessment may look differently in classroom assessment than in programmatic assessment. He indicates that an item that includes biased language for one group, while not appropriate for use in classroom evaluation, may provide interesting contextual discussion in the aggregate and disaggregated responses of a programmatic assessment tool. Thus, context matters. Engaging with our stakeholders to define and understand these contexts open doors as assessment professionals navigate a space seeking equity centered assessments.