This article originally appeared on LinkedIn Pulse.
>> The story of the remarkable initial success and subsequent collapse of Cambio Education – a team of Minneapolis educators, software engineers, writers, graphic designers and filmmakers who created the Studio Luma program – should serve as both a guide and a warning to any edtech startup serious about taking on the daunting task of changing education.
This second and final part of the series explains how the Studio Luma program ultimately failed, despite achieving one of the holy grails of outcome-based education technology – to engage students and effectively train them in the communication, critical thinking and emotional intelligence skills that are often lacking in the Millennial vocational workforce. Read Part 1 of this series.
How to Flunk Your Own EdTech? Leave Teachers Out.
Education technology will not replace teachers. To the contrary, education technology, done right, has the promise of making teachers even more powerful and more essential in the learning process.
But any new learning tool that neglects the teacher’s role by focusing exclusively on the student experience risks failure. If you don’t pay attention to this, teachers can and will break your wonderful new tools. Not intentionally, not maliciously – it will be more subtle than that.
If a teacher is not yet fluent in the use of your new technology in the classroom, possibly because there wasn’t proper training or enough time to prepare, there will be a natural tendency to fall back on tried-and-true teaching methods. And in doing so, teachers will begin to sow the seeds of uncertainty about your edtech and your new methodology in the minds of the students.
Good Product Design Is Not Enough
Having sound principles about the design of your product isn’t enough. Everyone involved in making that product needs to understand and embrace those principles or the product risks being flawed.
Similarly, once the product is released in the classroom, you need a way to seamlessly traverse the virtual world of the new education technology and the real world of daily practice with teacher interaction. If the principles are not properly understood and supported by teachers, your implementation risks being flawed and the students will suffer.
We saw this happen with Cambio Education’s Filmbook® learning platform and Studio Luma cosmetology program.
- Teachers wondered, “Is that technique being demonstrated in the training film correct?” If some teachers learned a different way and had no proficiency in the way we were teaching, they could often undermine it.
- Teachers wondered, “Is it really necessary to practice this technique again?” If teachers were not able or willing to take on the role of coach, portions of the curriculum related to technical practice became suspect or seemed unnecessary.
- Teachers wondered, “Is this new curriculum complete?” If a teacher couldn’t quickly help a student find a topic within the content because she herself had not yet fully assimilated that content, some teachers assumed that the topic just wasn’t there.
A Tricky Set of Equations
Ultimately, the introduction of any new technology in the classroom that fails for the teacher will also fail for the student. In his essay What Are We Scaling When We Scale Education, edtech innovator and investor Robert J. Hutter makes the case that balancing all the elements in the classroom is a “tricky set of equations, even with the fusillade of modern technology and platforms to lean on. And the best education entrepreneurs are the ones that are the most innovative about this precise combinatoric.”
We had failed to be innovative about the precise combination of elements that would support every teacher in every classroom so that we could get the best outcome for every student.
Cambio Education was clearly innovative in the creation of the Filmbook education technology and the Studio Luma content, as I’ve described in the first article in this series. And in classrooms across the country where teachers could absorb and adapt quickly to this new methodology, we achieved outstanding results for students and the employers that hired them.
But in too many other classrooms, we witnessed firsthand that we were failing to be innovative about the precise combination of elements that would support every teacher in every classroom so that we could get the best outcome for every student. Hutter crystallizes this problem in his assertion that “instructor training is a form of the education scale problem.”
We had to find a solution for the struggling classrooms, and the clock was ticking. We had little room for error because our client schools were also under pressure from an increasingly hostile regulatory environment.
In this final part of the story we’ll examine how the Studio Luma program and the Filmbook technology operated in the classroom – unraveling the mystery of why it worked in some and not in others. I’ll describe what we did to turn things around, and I’ll provide our final takeaways.
Teach Teachers or They’ll Fail to Teach Students
A piano teacher once told me, “If you practice, practice, practice until you get it right, and then stop – you’ve spent 99 percent of your time practicing it wrong.” Knowing how to practice is as essential as knowing what to practice.
Let’s say you’re teaching cosmetology. Cutting and styling hair requires great physical dexterity, and the only way to achieve proficiency is to practice. Cosmetology students first practice on mannequin heads, then on each other, and finally, in most cosmetology schools, on a public clientele that has decided to take the risk of trusting a novice stylist in return for the economic benefit of getting a hair service for about the same price as a double whip latte.
In Karl M. Kapp’s book The Gamification of Learning and Instruction, he describes the notion of distributed practice as “the technique of distributing study or learning efforts over multiple short sessions” to enhance long-term retention. Kapp’s work was influential to us in our early R&D phase at Cambio Education, and we incorporated the idea of distributed practice in the formation of another of our six core principles, repetition.
Distributed Practice Enhances Long-term Retention
The structure of the Studio Luma Program was simple. Each week, students would start a new lesson by watching the films, interacting with the Filmbook app to test their comprehension and dive deeper as needed. After that, much of the classroom time was then devoted to hands-on practice of the skills they just observed and read about.
These distributed practice exercises, outlined in the Filmbook, were carefully constructed to enhance long-term retention by building the student’s pattern recognition ability and muscle memory. They were often repetitive, but not redundant. Subtle and progressive variations were built into each practice series.
During this practice time, the teacher was expected to move around the room and coach each student as she worked on the various techniques, ensuring that she was practicing correctly and getting the desired result.
This came naturally to many teachers. They were experienced cosmetologists with a wealth of knowledge about the intricacies of hairstyling. They could immediately see when a student was getting off track and could step in to lend a helping hand. And they thrived in their new role.
Resistance: Early Signs
But other teachers perceived their new role as something less than what they had before. They resisted in ways that were sometimes subtle, like drifting into inactivity while the students practiced. They approved student practice work with just a cursory review instead of comprehensive coaching.
None of which helped students learn. Without the teachers to guide them, some students were practicing the wrong techniques.
Moreover, visiting would-be students saw teachers sitting around. This was not the active, engaged, technologically-advanced classroom we so successfully promoted in our marketing material.
What did we get wrong? In this article on the reality of transformative edtech, it’s clear that we weren’t alone in this conundrum. Early adopters of innovative edtech in the teaching profession are leading profound changes in our educational model, but many teachers are unable or unwilling to change.
The teachers we worked with, by and large, wanted to be part of our transformation of education. But some of them weren’t quite ready to do that, for reasons that were not yet completely clear to us.
Our notion of introducing what we thought of as a higher order of teaching was starting to unravel.
To many of us, a coach might seem to be lower in the pedagogical hierarchy than a teacher. But in the rapidly-evolving world of technology-facilitated education, the reverse may well become true.
Convince Teachers that Coaching is a Higher Level of Teaching
To many of us, a coach might seem to be lower in the pedagogical hierarchy than a teacher. But in the rapidly-evolving world of technology-facilitated education, the reverse may well become true.
In this New Yorker article, an accomplished surgeon describes his decision to ask another more senior surgeon to become his personal coach in the operating room. As becomes evident in his analysis, coaching at its best requires an extraordinary combination of comprehensive knowledge, acute powers of observation, deep wisdom, and precise judgement. Most important, great coaches must “speak with credibility, make a personal connection, and focus little on themselves.”
Flipping the Classroom? A Flop if Done Wrong
When Cambio Education conceptualized the Filmbook learning platform and the Studio Luma program that ran on it, our intention was to flip the classroom and transform the teacher into a coach.
We even had a tagline for it – “lecture-free learning.” We imagined a teacher no longer tied down by the need to prepare and present daily lectures in front of a bored classroom, a teacher freely roaming a room full of students engaged with our new technology and fluidly intersecting with these students as they moved in and out of our virtual world, adeptly coaching them in a just-in-time fashion. This expert feedback was another of our six core design principles.
And we imagined the students – most of whom were digital natives – adroitly navigating this new digital curriculum with little or no need for training on the technology.
With the students, that is exactly what we did see. Students rarely had a problem adapting to the technology. In one school, a young student told me that she had been in special-needs classes her whole life, and she loved the Filmbook. “I have ADHD. I can learn better this way. It makes me feel smart.”
Lecture-free Learning: From Lecturing Teacher to Active Coach
And with some teachers in some classrooms, we did see the extraordinary transformation we had imagined: from lecturing teacher to active coach. Those classes were buzzing with activity, with teachers constantly in motion. Amidst a dizzying array of student activity, the teachers could focus on each student individually, make a connection, observe what the student was doing, dispense their wisdom with precise judgment, and move on to the next student.
But those teachers were rare. They had a natural ability to multi-task, they became fluent in the curriculum and, most importantly, they wanted to be coaches. But of the over 400 teachers who taught the Studio Luma program from 2014 through 2016, too few could quickly embrace their new roles as coaches.
We thought we were elevating our teachers toward a higher purpose and introducing them to a brave new world. But many teachers thought we were pulling the rug out from under them.
These teachers simply needed more time to adapt, and to adopt.
An Unsettling Trend Emerges
And to make matters worse, a new unsettling trend began to emerge in the data we were collecting on the state licensing board examination results of our recent graduates.
Cambio Education was interested in the real world of the salon and success within it. This ambitious focus became a liability when we unintentionally gave short shrift to the equally real world of licensure.
Licensure: Real Learning vs. Standardized Testing
How do you measure learning? That is a subject of endless debate, and Cambio Education purposely sidestepped that debate by focusing exclusively on two different questions:
- How do you drive learning?
- How do you build test-taking skills?
We figured if we could do those two things, we could leave the measurement part to the state licensing boards.
It’s not that we weren’t interested in measurement. In fact, measurement was another of our six core design principles. But we were interested in measuring different things, like patterns of interactivity in the Filmbook app interface so we could determine how different features and materials were being accessed, how much time students spent with each lesson, and how all that related to student success in the classroom.
We did have all the trappings of traditional assessments, though. Every class had a series of questions that presented themselves to the student throughout the films. And like any good interactive quiz mechanism, the questions came from a question bank and the answers were randomized.
Use Questions to Drive Learning
But our questions had a different purpose than a traditional set of assessment questions, which typically attempt to measure what you have just learned.
Filmbook questions were designed to focus the mind on the most important thing that was happening in the film right then. The questions always paused the film, and often they required the student to explore the underlying written material (the “book” part of the Filmbook that could be accessed by swiping up the corner of the film player) before continuing.
Give Students Freedom to Fail
Our assessments were open book. And there was no penalty for failing. Students could keep re-taking these assessments until they passed or got the score they wanted.
In this way, we solved a fundamental problem we had seen in our schools. Test anxiety was prevalent among our largely non-academic student base. By giving students freedom to fail, we found that overall test anxiety was alleviated and the students would often take these open-book tests multiple times for practice or to get a higher score.
But we made two errors in our question methodology.
- We assumed that by allowing the students to take and re-take open-book assessments with no penalty, we would be building their test-taking skills in preparation for the licensing exam.
- We assumed that by giving them a closed-book option, they would use it as they approached graduation and were preparing for their exams.
We were wrong on both counts. The questions did serve their primary purpose of driving learning – we could see that in the students’ mastery of technical and soft skills.
But students did not have the discipline to force themselves to use the closed-book option. And without a testing mechanism that more closely simulated the actual written licensing exam, we failed to build their test-taking ability and confidence. The state board exam results were starting to show it.
A Third Issue: Preventing the Curriculum from Leaving the Classroom
Then we made another strategic error with our implementation.
To simplify the initial development and rollout, we built Filmbook on one platform first – iOS – and provided each student with an iPad device that we controlled. The devices could not leave the school premises because the filmed content was streaming from a local server in each campus.
With something this new, the more variables you can control the better the user experience you can provide. We wanted no hiccups when the film was playing, and we had not yet built a sophisticated enough algorithm into our player to enable multi-bitrate streaming from the cloud. We also had not perfected a content protection mechanism, so downloading the films to each device was too risky.
From a project management standpoint, all this made sense. From a student standpoint, it was a great dissatisfier. Students felt like they were renting the curriculum – they wanted to own it and take it home with them so they could study whenever they wanted to.
Self-inflicted Wounds and Our Rapid Response
Cambio Education was still standing. But these self-inflicted wounds were starting to have an effect. Once the cracks in the foundation of our education technology experiment began to reveal themselves by the summer of 2015, we set to work on a series of rapid responses.
We began developing a series of teacher training tools and an online community where teachers could share their techniques and communicate directly with the Studio Luma curriculum team for advice. This started to help a growing number of teachers transform themselves into coaches, and their classrooms into highly effective learning environments.
To solve the state board exam problem, we studied several quiz-based apps and created our own version of the licensing exam that had a timer and was closed-book. By setting this up with a game-like interface and social sharing features (“First 100%!”), we found that the students starting playing it addictively even very early on in the program.
It had all the hallmarks of success. Some students reported playing our quiz games non-stop in the weeks before the state boards and then acing the exam.
Filmbook for Student Devices
To solve the problem of 24/7 access to Studio Luma, we furiously began developing the iPhone and Android phone versions of the app and the cloud-based streaming to support them. When we released the apps to the app stores for beta testing, word got out quickly among the student population. By the time we reached our “official” release date, close to 50 percent of our students had already downloaded the app to their phones and were using it to study.
When we visited campuses, many students asked if there was a way to print the Filmbook. We were surprised, of course, because we assumed that with our wonderful new digital curriculum there would be no more need for printed material.
Turns out, we were wrong. As this article on Millennial reading habits explains, print is not even close to being dead yet. We had to figure out a way to provide what students wanted without upending the whole paradigm of a digital curriculum and all its inherent efficiencies.
A simple printed companion to the Filmbook saved the day. It was a kind of workbook that consisted mostly of an outline of the lessons with instructions on best practices for study. The students loved it. It gave them a traditional “anchor” to ease the transition to the new digital world we had built for them.
Transformative Change Takes Time
As Pixar co-founder and president Ed Catmull writes in Creativity, Inc., a culture of innovation requires embracing mistakes as “an inevitable consequence of doing something new.” Or, as these leading thinkers at EdTechTeacher.org put it, failure is mandatory on the road to success and learning.
Cambio needed more time to build solutions to fix the mistakes we made and to perfect how our products operated in our client schools. But as we were building these solutions, the impact of the unfavorable regulatory climate on for-profit education was also building.
By the time we could roll out the new features and start seeing the positive effects in the classroom begin to take hold in the summer of 2016, our timeline for turning things around had all but run out.
Final Takeaway: Make Forward Thinking Backward Compatible
So, you’re transforming education. Being disruptive. Exciting, isn’t it?
As you’re sitting in your edtech laboratory, perfecting your design to create that incredible user experience that you know is going to change everything, remember that at the other end there’s a real teacher sitting in a real classroom working with a real student. They’re trying to get through their day, which is often fraught with real problems you may not be thinking about.
- As you’re creating all these new teacher interfaces, are you thinking about how much time teachers need to prepare for class each day? Get teacher feedback early, so you have a realistic idea of their capacity for the kind of disruptive change you’re envisioning. Teachers will help you get it right if you give them a voice. We did give teachers a voice, but for logistical reasons we were unable to get a wide enough sample of teacher input to properly evaluate every school we worked with. And some schools just needed more time to adapt.
- As you’re adding all that multimedia to your app, are you thinking about the student’s data plan? He or she may not be able to afford streaming your videos every day if your streaming isn’t efficient. We were very careful in optimizing our streaming, because we saw that the number one user-reported problem with many edtech apps was poor performance. You need to consider the entire spectrum of students who will be relying on your app for home study.
Backward compatibility is a fundamental tenet of software development. If you make a new version of something, make sure it doesn’t break things for older systems that need to interact with it.
Backward compatibility must also be applied to any new education technology that interfaces with an established set of standards, teaching or learning protocols.
Cambio Education had an ambitious goal – to transform the very nature of performance-based education by creating a new ecosystem that combined well-crafted storytelling with solid learning science, all in a streamlined mobile interface.
We succeeded in producing graduates who were more prepared to be successful in their careers.
And even though we succeeded in producing graduates who were more prepared to be successful in their careers, as evidenced by rave reviews from salon owners across the country, we also broke things – for students who struggled with the standardized test model of state licensure, for teachers who struggled with their new role of coach, for students who wanted to carry the Filmbook home with them like a textbook, and for teachers or students who felt that printed material could provide a helpful transition to a digital world.
And that is why, despite the incredible efforts of everyone involved at our schools – every teacher, every administrative and operations person – we simply couldn’t move fast enough to implement our full vision for every school in time.
It’s why the Studio Luma films will live out their life as a supplementary add-on to an established curriculum, disassociated from the ingenious Filmbook software that made those films interactive and effectively drove comprehension and learning.
And ultimately, it’s why the extraordinary Cambio Education team failed to fulfill its promise of transforming not only cosmetology education, but all competency-based vocational education.
The Future of Story-based Learning
I believe story-based learning will transform education – it’s just a question of how quickly it will happen.
I’m encouraged by the success that Jo Schneier and her team at Cognotion are having in the field of healthcare training. As far as I know, they are the only ones now doing vocational training with such a high level of narrative storytelling.
I’m encouraged by the efforts of Karl Kapp to educate all of us on the power of using game mechanics – which inherently contain storytelling constructs – in the creation of education technology.
The technological tools we use are new, but these concepts are not. Myths, legends, fables – these are all mechanisms to transmit knowledge and wisdom from generation to generation.
Storytelling in education is ancient. The story of storytelling in education technology is just beginning.
Neal Tillotson is a Minneapolis-based edtech innovator, IT leader and musician. He was the co-founder and Chief Product Officer of Cambio Education.