Skip to main content

Using Mixed Methods in Human-Centered Higher Education Research and Design

Lauren Storz is an Academic Experience Analyst on the Academic Technology Design Team, and a PhD student in cultural anthropology at CU Boulder. 

Namita Mehta is a Learning Experience Designer with the Office of Information Technology at CU Boulder.  She has over 15 years of experience in education and holds an Ed.D in Leadership for Educational Equity.  


Wicked problems are those that have interdependent factors to consider. To solve these complex problems in technology-mediated teaching and learning, we emphasize the importance of understanding context and the systems at play to respond to human needs in the first blog post. Developing a clear and focused problem statement is essential to guide a research project, and mixed research methods can be used to understand and continually refine problem statements throughout the research process.  In this post, we want to expand on how we use mixed research methods to achieve a human-centered approach– in other words, an approach which prioritizes the needs, perspectives, and experiences of people at every phase of research. Utilizing a diversity of methods can give us a more realistic understanding of how students, staff, and faculty are experiencing a situation, interaction, or intervention.  Context is not only about understanding the experience of each individual, but it also entails a thorough investigation of the relationship between individuals and their interaction with  sociotechnical systems.  Design-based research must account for how design(s) function in the everyday, lived experiences of those occupying a given space. It must not only document success or failure of a space, but also focus on “the interactions that refine our understanding of the learning issues involved” (DBRC, 2003).  The evolution of such accounts depends on methods that record and connect lived processes with assessment goals (DBRC, 2003).  In other words, a key element of conducting design-based research that centers relationships and embraces complexity is having a mixed-methods approach (Anderson & Shattuck, 2012). 

The Case for Mixed Methods 

Quantitative and qualitative data contribute something different to our understanding of the human experience.  Numerical data is helpful in measuring changes across time.  Quantitative methods can be used to measure if an intervention is effective or even to see the relationship between two facets of an experience.  What qualitative data offers is exploring new realms and gaining greater clarity on what those numbers mean.  These complementary methods can paint a clearer picture of the human experience by measuring  the pulse of a population and empathizing  individual stories at the same time.  

Mixed methods approaches can look a variety of ways.  One option is to use two distinct methods such as a survey and a focus group.  Some people hesitate when hearing this approach as to use two methods for every practice-engaged inquiry could be both time-consuming and exhausting.  This approach does not need to be a burden on professionals who already have so many duties to juggle.  One popular way of integrating these practices is by including both Likert scale and open-ended questions on a survey.   Another way to approach this is by offering places where students can rate certain questions within a focus group.  The main point being that expanding our method allows us to more accurately understand what the experience is and how we can improve it.  To highlight this concept further, we would like to share a recent study conducted by the Academic Technology Design Team at CU Boulder, which illustrates one of the myriad ways a complex research project can be approached by employing numerous sources and mixed methods. 

Learning Spaces Project Example

Titled the Learning Spaces Project, this project was in partnership with the Learning Spaces Technology team that is in charge of maintaining and upgrading academic technology in the classroom. While our partner team was looking for concrete recommendations concerning academic technology upgrades and/or support, we wanted to broaden our scope to better understand how academic technologies are situated within teaching and learning ecologies across campus. We decided to create case studies with courses across disciplines and in a variety of classroom types (e.g. large lecture halls, flexible classrooms, labs, etc.). For each case study we triangulated our data through a student survey, classroom observations, and contextual inquiry with instructors. Our student survey focused on students’ expectations and preferences with academic technology and classroom design. For example, we asked likert scale questions such as: To what extent do the following learning tools impact your learning? Which technology/learning tools do you generally expect to be available in a given classroom? Each question gave students the option to rate the following classroom technologies: screens and projector, blackboards and whiteboards, monitors and TVs, clickers, classroom capture, and other (fill in the blank). We also gave students two open ended questions: How does the classroom technology/learning tools and space impact your learning? And, please draw and/or describe the technology and layout in your ideal classroom and explain your design choices.  Classroom observations were conducted by a member of our team and narrowed in on classroom dynamics (student to student and instructor to student interactions, as well as the ways in which classroom technology and/or the physical classroom design influenced or mediated such interactions) as they played out in real time. Lastly, contextual inquiries with instructors delved into a given instructor’s pedagogical goals, expectations of available classroom technologies, and any difficulties they have experienced. In conjunction with these case studies we drew from pre-existing data from a campus-wide survey and anonymized Faculty Course Questionnaire responses.  

Through this mixed method approach we were able to identify evolving themes that persist across campus, as well as distill concrete recommendations that our partner team could act on right away. For instance, one theme which emerged was that instructors often have to tailor their teaching to fit within the confines of the classroom they have been assigned– which becomes more burdensome when an instructor must teach the same course in two or three different classrooms across campus. This insight has implications for pedagogy, the ways in which instructors are assigned classrooms, and classroom design, but it remains relatively abstract and in need of further investigation. In contrast, a more granular insight came from the student survey. Students from across disciplines consistently highlighted access to classroom capture recordings of weekly lectures as highly beneficial to their learning. However, some are unable to access classroom recordings because the equipment is not physically available in small classrooms for instructors to take advantage of. 

How it Applies to Other Areas of Higher Education

Although the example we used here is related to academic technology, these principles can be applied to other areas of higher education. This approach could be used to improve the residence life experience of students.  A survey with Likert-scale questions could reveal a difference in sense of belonging among different demographics when analyzed.  To further explore what could help, the survey could include open-ended questions, or a focus group could be conducted later with the students.  The combination of methods allows researchers to delve deeper into a problem of practice, and for students to contribute more of their ideas and have the ability to prioritize them.  Two benefits of this approach are that: (1) it limits the researcher bias by creating a participatory approach, and 2) it encourages an iterative approach to designing an experience that works for all students.  For example, using a mixed methods approach would help the emerging needs of students during COVID-19 and allow higher education professionals to improve and re-assess if any new approaches are responsive to these needs. Regardless of the methods used, it is important for researchers to be reflexive about their own positionality and how that will inevitably influence the research. We will be expanding on the salience of reflexivity, along with participatory approaches and the assessment cycle in future posts.

This is the second blog of a four part blog series.
For the first blog click here.


References

Anderson, T., & Shattuck, J. (2012). Design-based research:A decade of progress in education research? Educational Researcher, 41(1), 16-25. doi:10.3102/0013189X11428813
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.



Comments

  1. It used to be that applying for business cash for a smaller business was fairly straightforward. You'd pay a visit to your local friendly banker and talk about your business needs. You'd discuss what you needed and they would help with financing a business loan - yours, to be exact. Then, the financial crisis hit, and banks closed ranks and decided that loans for small business were too risky. Life and Business Coaching

    ReplyDelete

Post a Comment

Popular posts from this blog

Feeling Toward a Pedagogy of Grief

  Z Nicolazzo, Ph.D.,  Associate Professor, Center for the Study of Higher Education, University of Arizona Throughout the past two years, I have found myself periodically returning to a piece Alexander Chee (2020) wrote for The New York Times . In it, Chee discusses the parallels between the COVID-19 pandemic and the AIDS epidemic. When the piece was first published, I remember taking a long walk with a friend and talking about it. We both had vivid memories and personal connections to AIDS and were both trying to make sense of what was happening globally as COVID-19 ripped through communities. In one especially heart-rending moment in the column, Chee recounts a conversation with a friend who, in response to the rising death toll, shared, “‘I’m acknowledging them [the deaths], but I’m not feeling them, just like the old days. …That comes later’” (para. 8). That conversation between Chee and his friend was two months into the pandemic. We’re now over two years in, which would seem

Response to Dr. Z's Article

  Tessa M. Smith, Ph.D. Student, Educational Studies, College of Education and Human Ecology, The Ohio State University I’m sitting with Dr. Nicolazzo’s astute reckoning of grief as unmetabolizable. “Metabolize” is a word that feels so wholly connected to the body. Although our brains are of course parts of our bodies, the cerebral nature of how we, as a society and as educators, acknowledge grief certainly seems insufficient. I can feel a sick pit in my stomach from the incredible losses of the last two years especially. Plagues of gun violence, hatred, white supremacy, and the coronavirus have devastated the country. Meanwhile, many of us are chugging along, trying to find the “new normal” while so many deaths settle into the background of American life, the pits in our stomachs chugging along with us. I’m not sure if grief is something that can be metabolized, changed into something useful for the body, but perhaps that’s worth exploring. If grief were metabolizable, it s

Welcome to the 2022 Division J Blog

Hey, y’all! I’m Alex C. Lange (they/them). I am the new Editor for the AERA Division J blog. I am grateful to Dr. Jenny Lee for extending this opportunity to me, and I am jazzed for the chance to shape some generative scholarly dialogue through this blog. I am also delighted to have Martha Sesteaga (she/her/ella) join me as a Managing Editor for the blog. A bit about me: I am an assistant professor of higher education at Colorado State University-Ft. Collins, which resides on the lands of the Arapaho, Cheyenne, and Ute Nations and peoples. I teach students in our Higher Education Leadership Ph.D. program. My research interests include student learning and development, college teaching, and LGBTQ+ studies in higher education. I often write about these topics, especially teaching and learning, through my blog on Medium ( Medium link ). What do I hope to do with this blog? So glad you asked! The blog will focus on commentaries and reflections from scholars. I hope to curate re