Skip to main content

Using Mixed Methods in Human-Centered Higher Education Research and Design

Lauren Storz is an Academic Experience Analyst on the Academic Technology Design Team, and a PhD student in cultural anthropology at CU Boulder. 

Namita Mehta is a Learning Experience Designer with the Office of Information Technology at CU Boulder.  She has over 15 years of experience in education and holds an Ed.D in Leadership for Educational Equity.  


Wicked problems are those that have interdependent factors to consider. To solve these complex problems in technology-mediated teaching and learning, we emphasize the importance of understanding context and the systems at play to respond to human needs in the first blog post. Developing a clear and focused problem statement is essential to guide a research project, and mixed research methods can be used to understand and continually refine problem statements throughout the research process.  In this post, we want to expand on how we use mixed research methods to achieve a human-centered approach– in other words, an approach which prioritizes the needs, perspectives, and experiences of people at every phase of research. Utilizing a diversity of methods can give us a more realistic understanding of how students, staff, and faculty are experiencing a situation, interaction, or intervention.  Context is not only about understanding the experience of each individual, but it also entails a thorough investigation of the relationship between individuals and their interaction with  sociotechnical systems.  Design-based research must account for how design(s) function in the everyday, lived experiences of those occupying a given space. It must not only document success or failure of a space, but also focus on “the interactions that refine our understanding of the learning issues involved” (DBRC, 2003).  The evolution of such accounts depends on methods that record and connect lived processes with assessment goals (DBRC, 2003).  In other words, a key element of conducting design-based research that centers relationships and embraces complexity is having a mixed-methods approach (Anderson & Shattuck, 2012). 

The Case for Mixed Methods 

Quantitative and qualitative data contribute something different to our understanding of the human experience.  Numerical data is helpful in measuring changes across time.  Quantitative methods can be used to measure if an intervention is effective or even to see the relationship between two facets of an experience.  What qualitative data offers is exploring new realms and gaining greater clarity on what those numbers mean.  These complementary methods can paint a clearer picture of the human experience by measuring  the pulse of a population and empathizing  individual stories at the same time.  

Mixed methods approaches can look a variety of ways.  One option is to use two distinct methods such as a survey and a focus group.  Some people hesitate when hearing this approach as to use two methods for every practice-engaged inquiry could be both time-consuming and exhausting.  This approach does not need to be a burden on professionals who already have so many duties to juggle.  One popular way of integrating these practices is by including both Likert scale and open-ended questions on a survey.   Another way to approach this is by offering places where students can rate certain questions within a focus group.  The main point being that expanding our method allows us to more accurately understand what the experience is and how we can improve it.  To highlight this concept further, we would like to share a recent study conducted by the Academic Technology Design Team at CU Boulder, which illustrates one of the myriad ways a complex research project can be approached by employing numerous sources and mixed methods. 

Learning Spaces Project Example

Titled the Learning Spaces Project, this project was in partnership with the Learning Spaces Technology team that is in charge of maintaining and upgrading academic technology in the classroom. While our partner team was looking for concrete recommendations concerning academic technology upgrades and/or support, we wanted to broaden our scope to better understand how academic technologies are situated within teaching and learning ecologies across campus. We decided to create case studies with courses across disciplines and in a variety of classroom types (e.g. large lecture halls, flexible classrooms, labs, etc.). For each case study we triangulated our data through a student survey, classroom observations, and contextual inquiry with instructors. Our student survey focused on students’ expectations and preferences with academic technology and classroom design. For example, we asked likert scale questions such as: To what extent do the following learning tools impact your learning? Which technology/learning tools do you generally expect to be available in a given classroom? Each question gave students the option to rate the following classroom technologies: screens and projector, blackboards and whiteboards, monitors and TVs, clickers, classroom capture, and other (fill in the blank). We also gave students two open ended questions: How does the classroom technology/learning tools and space impact your learning? And, please draw and/or describe the technology and layout in your ideal classroom and explain your design choices.  Classroom observations were conducted by a member of our team and narrowed in on classroom dynamics (student to student and instructor to student interactions, as well as the ways in which classroom technology and/or the physical classroom design influenced or mediated such interactions) as they played out in real time. Lastly, contextual inquiries with instructors delved into a given instructor’s pedagogical goals, expectations of available classroom technologies, and any difficulties they have experienced. In conjunction with these case studies we drew from pre-existing data from a campus-wide survey and anonymized Faculty Course Questionnaire responses.  

Through this mixed method approach we were able to identify evolving themes that persist across campus, as well as distill concrete recommendations that our partner team could act on right away. For instance, one theme which emerged was that instructors often have to tailor their teaching to fit within the confines of the classroom they have been assigned– which becomes more burdensome when an instructor must teach the same course in two or three different classrooms across campus. This insight has implications for pedagogy, the ways in which instructors are assigned classrooms, and classroom design, but it remains relatively abstract and in need of further investigation. In contrast, a more granular insight came from the student survey. Students from across disciplines consistently highlighted access to classroom capture recordings of weekly lectures as highly beneficial to their learning. However, some are unable to access classroom recordings because the equipment is not physically available in small classrooms for instructors to take advantage of. 

How it Applies to Other Areas of Higher Education

Although the example we used here is related to academic technology, these principles can be applied to other areas of higher education. This approach could be used to improve the residence life experience of students.  A survey with Likert-scale questions could reveal a difference in sense of belonging among different demographics when analyzed.  To further explore what could help, the survey could include open-ended questions, or a focus group could be conducted later with the students.  The combination of methods allows researchers to delve deeper into a problem of practice, and for students to contribute more of their ideas and have the ability to prioritize them.  Two benefits of this approach are that: (1) it limits the researcher bias by creating a participatory approach, and 2) it encourages an iterative approach to designing an experience that works for all students.  For example, using a mixed methods approach would help the emerging needs of students during COVID-19 and allow higher education professionals to improve and re-assess if any new approaches are responsive to these needs. Regardless of the methods used, it is important for researchers to be reflexive about their own positionality and how that will inevitably influence the research. We will be expanding on the salience of reflexivity, along with participatory approaches and the assessment cycle in future posts.

This is the second blog of a four part blog series.
For the first blog click here.


References

Anderson, T., & Shattuck, J. (2012). Design-based research:A decade of progress in education research? Educational Researcher, 41(1), 16-25. doi:10.3102/0013189X11428813
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.



Comments

  1. It used to be that applying for business cash for a smaller business was fairly straightforward. You'd pay a visit to your local friendly banker and talk about your business needs. You'd discuss what you needed and they would help with financing a business loan - yours, to be exact. Then, the financial crisis hit, and banks closed ranks and decided that loans for small business were too risky. Life and Business Coaching

    ReplyDelete

Post a Comment

Popular posts from this blog

A Focus on Rural Student Strengths: Dismantling Deficit Thinking and Language on Rurality in Higher Education

Ty C. McNamee, Assistant Professor, University of Mississippi In recent years, issues that for decades have plagued rural areas have been front-and-center in mainstream news outlets, such as high rates of poverty, declining population, fewer white-collar job opportunities, and comparatively lower levels of educational success. Such issues impact rural students’ higher education attainment, a topic that news media have appeared to notice with headlines such as: “ Colleges Discover the Rural Student ,” “ The Rural Higher-Education Crisis ,” and “ For Colleges, A Rural Reckoning .” Each piece highlighted rural youth holding some of the lowest college attainment rates in the country , compared to other geographic locales.   These pieces put rural students at the forefront of higher education conversations for the first time in decades. Unfortunately, much of the discussion surrounding rural Americans frames their lives and educational experiences as deficient. In turn, my co-researchers

Preparing for the (Non-Academia) Higher Education Job Market

I entered graduate school knowing that my higher-education doctoral degree would be the catalyst for changing my career path.  My original path through student affairs had been fulfilling but not sustainable. The long hours of evening events strained my relationships, and the pay and opportunities for advancement were few. Over time, my values shifted, and I was no longer willing or able to make the tradeoffs in my work-life balance and mental health often required in student affairs positions. I had a vague understanding that I would need further education to embark on a career in research in higher education, but what even was a research career?  I entered my graduate program knowing that a faculty role is the most commonly sought post-graduation career path. Additionally, I felt fortunate that my institution offered a Certificate in Institutional Research and some guidance for students who wanted to follow that path. Beyond those two options, I was unsure what else existed for me.

May Our Memory be for a Blessing

Steven Feldman, Ph.D., Higher Education student , Indiana University Bloomington In August 2017, white supremacists at the Unite the Right rally shouted “Jews will not replace us” as they marched through the University of Virginia’s campus in Charlottesville. On October 27, 2018, Robert Gregory Bowers murdered eleven people attending Shabbat services at the Tree of Life synagogue in Pittsburgh, Pennsylvania.  On October 8, 2022, rapper Kanye West tweeted “I’m a bit sleepy tonight but when I wake up, I’m going death con 3 on JEWISH PEOPLE.”          The headlines continue to pour in and yet only begin to scratch the surface in documenting the rise in antisemitism nationwide. Although it is easy to write these off as the mistaken views of lone terrorists and misguided celebrities, antisemitism nevertheless runs rampant across the country, illustrating an entrenchment in our culture that few are comfortable admitting.     May our memory be for a blessing.   With each passing day, ou

Expansive Blackness: Making Space for the Multiple Identities of Black College Students

Laila McCloud, Ph.D., Assistant Professor, Educational Leadership and Counseling Department, Grand Valley State University.           I didn’t become aware of the term  diversity  until my first year of college. I left the west side of Chicago for a small, private liberal arts college in rural Ohio. It was on that campus that I learned about the many ways to be Black. I learned to love the distinct way that Black folks from Louisville and Baltimore pronounce words with the letter “r” in them. I met Black folks from California and Texas who told me that the tacos I had in Chicago were trash. Some of us graduated from elite boarding schools in the northeast like Choate or the George School. Some of us had white parents and some of us were dating white people. Some of us were the children of immigrants from the Caribbean, west, and east African countries. Some of us were coming into our queerness. We had different religious upbringings that impacted how we understood what to do Fri

Feeling Toward a Pedagogy of Grief

Z Nicolazzo, Ph.D., Associate Professor, Center for the Study of Higher Education, University of Arizona Throughout the past two years, I have found myself periodically returning to a piece Alexander Chee (2020) wrote for The New York Times . In it, Chee discusses the parallels between the COVID-19 pandemic and the AIDS epidemic. When the piece was first published, I remember taking a long walk with a friend and talking about it. We both had vivid memories and personal connections to AIDS and were both trying to make sense of what was happening globally as COVID-19 ripped through communities. In one especially heart-rending moment in the column, Chee recounts a conversation with a friend who, in response to the rising death toll, shared, “‘I’m acknowledging them [the deaths], but I’m not feeling them, just like the old days. …That comes later’” (para. 8). That conversation between Chee and his friend was two months into the pandemic. We’re now over two years in, which woul