top of page
  • Writer's pictureBrett Schilke

How do we measure what matters?

Over the past several years, the world of learning has taken a hard turn into more virtual and remote experiences. Spurred along by the pandemic, the trend was already underway long before, driven by the promise of convenience.


After all, what could be better? We can learn anywhere, any time, in our own style and at our own pace. We can be in the same place and the same moment, while sitting in our own homes or workplaces. It eliminates travel, and the cost of a third place to meet.


And these are, in fact, extremely important things. We champion these changes in the learning landscape because they make learning available to more people in more places and more circumstances.


In short, remote and virtual environments transform access to learning experiences.


But what about the quality?


The virtual learning platforms themselves collect mountains of data, churning out their own evidence that learning in this modality is effective. And why wouldn’t we want to prove that? It’s convenient, and as humans prone to some hefty confirmation bias, we’d love to validate that the convenient path is just as good.


The human dimension


The thing about convenience is that it is often tied to individualism. Convenience of all kinds creates more isolation and self-reliance, and cuts us off from the critical social fabric that actually binds our society. In learning, this couldn’t be more true, and couldn’t be more critical.


It’s a simple truth that educators know and don't often dispute that the “human dimension” is the most important part of learning and teaching.


The latest Global Education Monitoring Report from UNESCO (a behemoth, coming in at over 400 pages that very few people will ever read) serves up some real talk about technology in education. A huge number of statements in the report come with caveats, like:

“[yay technology!]...but requires contextualization and integrated support.”


“[technology is amazing! but]...it needs to be context specific.”


“[let's totally use technology!]...with appropriate pedagogical integration.”


There’s no real debate about this. Technology is a tool, not a solution.


Where that leads us


It’s said that wisdom is the ability to hold two competing truths at the same time. These are ours:


First, virtual learning is here and here to stay, and enables access to learning for untold numbers of people. This is wonderful, and we are here for that.


Second, we know that human interaction is more powerful than anything else in creating transformative experiences that support and strengthen learners.


For the past 15 years, we’ve been acting on these beliefs. We embrace technology as just one part of a complex system of pedagogy and learning, recognizing its potential and its limitations.


We design many of our programs to be facilitated on or offline, and even in our most-virtual programs, we’ve baked in social emotional support through coaching and tutoring, empowered students to form learning communities and meet in real-life alongside their virtual experience, and have used peer-to-peer learning as a powerful tool to create an environment of support and care in their learning.


But there’s a difference between believing something and proving something.


How to measure what matters


What matters in a learning experience is not the analytical data that comes from an LMS. That just tells us whether a thing was done. Learning is about transformation. Learning is about feeling empowered to put new skills into action. Learning is about how you grow with and as part of a complex social system.


Measuring those interactions is hard. It’s not as simple as taking some data out of a platform. It takes real energy, time, and care.


But the most important things in life do take time, energy, and care. So over the last two years, we’ve begun carefully developing a research practice at Eidos, working to put evidence behind the role of human interaction and connection in virtual and remote learning.


Where we’re headed, and how you can help


To start, we spearheaded some pilot research with Google.org and with a PhD researcher from UC-Berkeley and The People Lab at Harvard. These initial experiments have highlighted the challenges of measuring the hard stuff, and we’ll have research notes and reports to share in early 2024.


Right now, we’re focused on building evidence for:

  1. What best enables facilitators to provide effective support and coaching

  2. Best practices for peer-to-peer and community learning

  3. The impact of human interaction and personalized coaching on program completion rates and long-term outcomes

  4. How AI can be used to enhance humane learning and teaching

We’re continuing to build our body of research, and our commitment is to publish our findings openly, creating tools, rubrics, and tips that will help educators and learners everywhere build better learning experiences, together.


We can’t do it alone


So consider this our "help wanted" ad in the newspaper (remember those?):


Global NGO building human-driven learning experiences seeking academics, researchers, data-heads, funders, and partners to turn insights and beliefs into powerful tools for the world.


Ok, maybe it’s more of a personals ad. After all, it’s the human connection that matters.


🫶🏼



Comments


bottom of page