The eight ingredients of a great MOOC

Massively Open Online Courses (aka MOOCs) have been all the rage in the popular press and edtech publications for the last year, and not all the rage is positive. In an effort to gain a little perspective on the fractious debate (and learn Python in the process), I recently completed two MOOCs offered by Udacity: Introduction to Computer Science (CS101) and Web Development (CS253). While admittedly a small sample size, my experience with these two courses highlighted some of the promises and pitfalls in large-scale online learning.

As I thought about my experience taking the two Udacity MOOCs, and catalogued all of my comments, complaints and kudos, things generally fell into eight big buckets that can help answer the question, What makes for a great MOOC experience?

1. Compelling, knock-your-socks-off content

There can be no learning without content, and it’s probably self-evident to say that bad content makes for bad learning experiences. With this in mind, there are a few obvious questions to ask about the content of a MOOC: How good is the base material? Does it come from a single source, or does it make reference to the broader universe of content available? Is the content static (e.g., images and text), dynamic (e.g., video), interactive? Is the content updated and refined frequently, set in stone, or somewhere in between? The answers to these questions are pretty subjective; learners will each have their favorite flavors. Based on what I’ve read, and my own personal experience, I believe MOOC content should be:

  • Created by a reliable and trustworthy source
  • As dynamic as possible, with interactivity to support engagement
  • Appropriate for the level of the course
  • Couched in a broader context, with plenty of third-party reference material
  • Updated frequently to address errors and oversights, reflect student feedback, and stay up-to-date with current thinking where relevant

2. Rockstar teaching and best-practice pedagogy

As with content, bad teachers make for bad experiences, and a good MOOC relies on great instructors. A teacher doesn’t have to be degreed or professorial to deliver a great learning experience; indeed, famous professors (of the "flown in from another planet" variety) don’t always make for great teachers. In the same vein, a wildly successful entrepreneur or business person may not cut the mustard either. And newbies? Forget about it. Get people with teaching experience.

In terms of pedagogical approach, there are some commonalities among the current crop of MOOCs, as well as significant differences (see The Pedagogy of MOOCs by Paul Stacy for more on this topic). In some cases, the difference in approach is driven by the platform, whereas in others it’s more driven by the instructors and the content. There seems to be universal agreement that the traditional "sage on the stage" approach makes for a boring MOOC, which means good MOOCs will shake things up with presentation appropriate to the delivery vehicle (i.e., the Web). The philosophical approach to pedagogy in MOOCs is a subject of fierce, often abstruse, debate far beyond the scope of this meager blog post (cf. objectivist and behaviorist learning theories vs. constructivism vs. connectivism vs. mastery learning). At the end of the day, I think the approach that should be taken is one that leads to the desired outcomes.

3. Superior user experience that maps to the learning process

An absolutely critical component of large-scale online courses is their user experience design. Good design means things like solid wayfinding and navigation, unambiguous iconography, clear informational text, contextually relevant content, and a visual design that supports all of the above. Bad experience design can spell dropout for all but the most determined students, even if the content is great. Udacity discovered this when they tried to redesign aspects of their site, but were greeted instead with a backlash from students who felt the "improved" design was a step backwards.

4. Avenues for connectivity and human interaction

One consistent criticism that has been leveled at many MOOCs is that they don’t support student-to-student and student-to-teacher (or mentor) interactions. These human interactions can be critical to success for many students; they can help people overcome stumbling blocks, offer ongoing motivation and emotional support, and generally keep people more engaged and accountable for course completion. In an effort to address these very human concerns, MOOCs have adopted different approaches, including things like forums, chat, blogs, Wikis, and even face-to-face meetups. Whatever approach is taken, it needs to scale and support a vibrant, dynamic and diverse community of learners.

5. Connection with real-world outcomes

Whether it’s case studies or down-to-earth examples, MOOCs benefit greatly from content and exercises that are connected with real-world experience in some way. Of course, this depends to a large degree on the course content; the more abstract the material, the more important it becomes to connect those dots for learners. As an example, Udacity’s CS101 course has "learning how to build a search engine" as its explicit goal. By using this as a Rosetta Stone, programming concepts and tools always get tied back to something in people’s experience of using search engines like Google to find useful stuff. This approach helps avoid the all-too-common question in MOOCs: Why am I doing this, again? Without any connection to some real-world outcome or benefit, many students are likely to lose interest and drop out.

6. Geek-Squad-level technical support

The whole purpose of online learning is to lower barriers to entry, primarily by delivering content via the Web to the majority of Internet-enabled devices. First, this means user experience design implementations should be responsive and device-agnostic; course providers should support people when they have problems here (e.g., browser issues with decrepit versions of Internet Explorer). Second, if the course involves the use of technology above and beyond a browser (e.g., programming courses that require software installation or configuration), there should be plenty of information to make sure people can get up and running with minimal frustration. Finally, if there are functional or technical problems with the learning site itself, the provider should offer clear communication about the problems and have the ability to remedy the situation quickly. In short, if the technology in an online course doesn’t work right and there’s no support, it’s dead in the digital water.

7. Facile feedback and genuine response

Communication between learners and MOOC providers should be two-way. It should be easy for people to submit feedback on course elements and content, instructors and the platform itself. MOOC providers should be able to take this feedback and respond to it in a way that lets people know their thoughts aren’t just winding up in the digital dead-letter office. As an example, in the case of Udacity’s CS101, they created a whole new "sub-unit" within the course to address learner feedback about some gaps in the learning material. On the flip side, when I submitted some unit-level feedback on the CS253 course, I got no email response or other indication that my feedback had even been received. I suspect the central challenge MOOCs face is one of scale: their ability to respond to a torrent of feedback in anything other than an automated way is probably limited. The best MOOCs will find a way to scale this interaction.

8. Assessments that don’t suck

Many (if not most) of the people who pursue learning in MOOCs are interested in some form of assessment, whether it’s just to test their knowledge, or to gain some credit for having completed a course successfully (more on this below). For large-scale courses, this can pose a significant challenge for MOOC providers, because anything other than multiple choice tests are difficult to assess automagically. Robot graders pretty much stink; they’re completely limited by the technology that underlies the MOOC platform. To make matters worse, they can’t even be used to grade things like written or project-based work. Some MOOCs have recently turned to peer-based assessments, but those efforts are still early days.

MOOC providers like Udacity offer assessments for simple quizzes and more complicated homework assignments, but the output from these tools can be frustrating. My experience with Udacity’s CS253 provided a case in point. Many of the more involved homework assignments led to binary assessments (i.e., you got it "right" or "wrong") with little or no indication of where you went wrong, the depth of the failure, and how you might go about fixing things. Not helpful. A great MOOC experience will need to do better with assessments, so that people can learn from their mistakes more effectively. I found myself taking the expedient route: skip to the answer when things go wrong, rather than try to get it right. Not really much mastery learning going on with that approach, but I suspect it’s one that many MOOC students will pursue out of frustration.

A few words about certification

It could be argued that I’m missing one ingredient in my great-MOOCs recipe: mechanisms for certification. This question has been one of the thorniest when it comes to MOOCs, specifically when it comes to many people’s desire that they get college-equivalent credits for successfully completing a MOOC. Udacity partnered with San Jose State in an effort to offer this kind of certification, but results have been mixed. Coursera’s Signature Track offers verification for student completion, but it’s unclear how these certifications will be received in the job marketplace. In short, it’s still early days for credit-worthy MOOCs, but certification will likely become an essential ingredient as things evolve.

Next up: Course reviews for Udacity CS101 and CS253

In this post, I’ve laid out what I think makes for a good (or great) MOOC. Coming up next, I’ll provide a more detailed review of the two courses I completed at Udacity, and see how they measured up. Stay tuned!