The messy semantics of educational achievement and value

illusion-big

The Illusion of Agreement (via 37 Signals)

The educational world has its own shibboleth, and so do the worlds of policy and business. As these three worlds come into contact, the terminology and buzzwords used by each collide. People make assumptions that they are talking about the same thing, but in many cases they’re not. Educational establishments have language rooted in tradition (though even the terminology used between US and international educational systems is somewhat different). On the other hand, business people (especially those in startups) often seek terminology for the purposes of marketing, rather than precision, and it’s no different with the rise of edtech startups. One set of terminology that’s riddled with confusion is the language used to describe achievement.

In an effort to cut through the buzzwords and imprecise usage, here’s my objective take on the language of educational achievement.

Achievements represent completed actions

Achievements are things that someone has done, and they come in many shapes and sizes. They can be quantified or fuzzy. They can be big or small. They can be hierarchical (i.e., achievements can be bundled together in order to represent some higher-level achievement). Examples of achievements include:

  • Passing an exam
  • Completing a course (an aggregate of exams)
  • Satisfying the requirements for a college degree (an aggregate of courses)
  • Creating something of value (e.g., a business, a painting, a book)
  • Accomplishing some difficult task (e.g., climbing Mount Everest)

Achievements are the biggest bucket in the language of educational accomplishment, and most of the other terms people use rely on achievements in some way, as we’ll see below.

Assessment measures achievement

An assessment in the traditional (educational) sense is a method for measuring achievement that leads to a quantified score or grade. Assessments can take many forms, depending on what it is one is trying to assess; they can be normative, formative, summative, ipsative, criterion-referenced; there’s a whole zoo of methodologies, and educators argue incessantly about which is best under any given circumstance. At the end of the day, all of these assessments are verifiable and quantifiable, critical properties when others need to use the assessment for some purpose.

These traditional methods aren’t the only way one can think of assessment, however.

How does one assess achievements like the creation of something (e.g., a business or a painting)? These types of assessments seem more subjective, less quantifiable. One possibility is for a market to perform the assessment; whatever the object can be sold for becomes an assessment of its value. Peers or society-at-large can also provide assessments (e.g., "That’s a great painting"), but these types of assessments are often abstract, unverifiable and meaningless when it comes to generating economic value. In some cases, however, there is enough agreement in the assessment that the subjective approaches objectivity (e.g., "Monet was a great expressionist painter").

Portfolios collect work products

A portfolio is a collection of work products that demonstrate achievements. It does not quantify or assess the value of this aggregation of accomplishments, even though each individual achievement may have an associated assessment. Someone looking at a portfolio might try to come up with their own assessment of its total value, but the portfolio is mute on the subject, regardless of who is putting it together. Electronic portfolios of student accomplishment (i.e., e-portfolios) are currently a hot topic in education, but many conflate these portfolios with something more. Portfolios are not qualifications or credentials or certifications, and it is dangerous and misleading to treat them as such, especially for students who think that putting together a random portfolio is going to get them a job.

Resumes summarize achievements

A resume is an unverifiable list of claims about an individual’s achievements. These achievements can include qualifications, certifications, credentials, and skills based on work and life experience. In combination with a portfolio, this can be seen as a proxy for someone’s overall qualifications; the only problem comes with cheating (lying). Resumes as they stand today cannot be verified in any meaningful way.

Qualifications are useful bundles of achievements

Qualifications go where mere portfolios and assessments do not; they aggregate assessed achievements into useful bundles to which value can be assigned. As such, they are generally formally, or officially, recognized. Value is in the eye of the beholder, however, and is often determined by markets or employers. Examples of qualifications include:

  • Degrees from accredited institutions (e.g., a 4-yr college)
  • Completion of a body of work associated with a desirable skill (e.g., a Cisco Certified Network Associate program)

Certification is verifiable qualification

A certificate is a verifiable document, issued by a trusted organization, that asserts a claim about the achievements of the person named on the certificate; certification is the process of issuing the certificate. Certification and qualification are complementary. The piece of paper a 4-yr college gives to a student upon completion of their degree is a certification; it’s just a physical manifestation of that qualification. One can easily imagine verifiable digital forms of certification as well (i.e., it doesn’t have to be physical). It’s something someone can point to that unambiguously and verifiably proves (within a reasonable doubt) they have achieved what they claim.

Credits are the de facto currency of higher education qualifications

The current higher education system in the US, and the qualifications one can receive from it, rely solely on the notion of the credit. If a student gets enough credits in a given curriculum, they earn a qualification. This currency is rigorously controlled by a byzantine accreditation system designed to exclude, and is intimately coupled to financial aid systems and the economics of higher education.

One of the greatest challenges faced by MOOCs (or any non-traditional educators) is that their courses cannot, in most cases, be applied for college credits arbitrarily. Without this accreditation, without credit equivalency, it’s hard for students to assess the value of work done with these non-traditional providers. Their efforts cannot easily be combined into a "degree" (i.e., qualification) that can be neutrally assessed by a potential employer (or society-at-large).

Transcripts aggregate credits

As they stand today, transcripts are just aggregators of credits. They do not represent qualifications (since a transcript only reflects coursework completed, not necessarily the body of work required for a qualification). This is a critical distinction. Startups like Degreed speak convincingly about the need for reform and evolution in the way we think about qualification, but they seem to conflate "jailbreaking the transcript" with "jailbreaking the degree." While these two things are related, they’re not quite the same, and we should be careful about this language. The former is a necessary, but insufficient, condition for the latter.

Credentials are licenses to practice

A credential is a modifier that layers on top of an existing qualification; it acts as a license to practice a given set of skills manifested by a qualification (which is a degree, in most cases). An example of a credential is the passage of the bar exam in any given state; it gives the bearer of that achievement the right to practice law; one can’t practice with the qualification alone. This term represents another place where people tend to play fast and loose with terminology. The catchy phrase "Credentials 2.0" has been used in a few discussions, and by some startups, as a shorthand for talking about a new approach to qualifications. It’s a great marketing buzzword, but it’s imprecise; the discussion needs to remain focused on qualifications and how we can evolve our notions of them.

Bottom line: Get the language right

If you’re an educator or a startup or a policy wonk or just someone interested in education, it’s important to get the language of achievement right. Without clear discussion and terminology, the illusion of agreement will obtain in spades. Everyone wants what’s ultimately best for students, employers and society, but if we’re not careful, we might wind up with low-income housing instead of the beautiful mansions many desire for the new world of education.

  • Tomáš Dostál

    I believe that in this age when everything is virtual, we should be also rewarded for our achievements in real-life by some virtual recognition in our on-line life. There are few solutions for global achievements and badges, one of which is Badge Fever – http://badgefever.com . Definitely worth of checking-out

    • I completely agree Tomas. There needs to be an evolution in the way qualifications are recognized, and it needs to encompass real-world achievements outside the context of educational systems. I’ll definitely check out Badge Fever. Thanks for the reference.

      • Tomáš Dostál

        Exactly! There is so much potential and so much missing in this area. In nature, people are very competitive, once people get the idea of being recognized for their achievements they cannot get enough of it. I could talk for hours about everything I have learned and realized, feel free to ping me on twitter or google +

  • Interesting post, Ryan. I value what you’re trying to do in terms of aiming for ‘objectivity’ when it comes to assessment and credentialing. However, as I mentioned to you over dinner at SXSWedu, I believe strongly that the market is not a good way to determine value. It is fickle.

    Instead of a single, monolithic ecosystem and technical solutions to social problems I believe that putting the learner in control and enabling them to show different facets of their learning and personality is the way forward. Open Badges (http://openbadges.org) is an awesome way of doing that, but the important thing is for there to be open standards in this arena. Until we’ve got that, we’ve got no way forward, I’d suggest.