Ninety percent of a significant education is
good teaching--
the other half is strong assessment

 

Paul Baepler
University of Minnesota, Twin Cities
(baepl001@umn.edu)

 

Hacking is building, and that’s what we want students to do.  Make things.  And now we have an embarrassment of Web 2.0 tools to create and share and reimagine.  So what stops us from taking up this challenge?  Why don’t we tap this well?  With a nod to Yogi Berra, I contend that significant education is ninety percent good teaching and the other half is strong assessment.

Imagine you wanted to assign a student video project in your class. How might you assess it?  The analog of that multimedia project might be a written text:  an essay, a story, or a report.  In that instance, you might engage students in a peer review process, asking them to comment on each other’s work.  At some point in the process, you’d likely weigh in with your own thoughts.  You might intersperse comments throughout the text and add a summation of your general impressions and perhaps a grade at the end.  Sound familiar?  But now your students have submitted videos on YouTube, and you’re faced with commenting upon them.  Written remarks might work fine for your global thoughts on the assignment, but how could you deliver situated feedback within the framework of the student media project?

YouTube has provided one answer by adding commenting and spotlight features to their video sharing services.  These capabilities allow a viewer to type “speech bubbles” directly on top of the video.  (This actually might be an intriguing twist to the assignment whereby students could mashup the video text with written meta-commentary.  It could also result in an unmitigated disaster if appropriate civility guidelines weren’t in place!)  The problem with this solution, however, is that the text is overlayed onto the video screen.  The speech bubble obscures part of the moving image.  Additionally, the process is not particularly efficient; there are several steps that have to be accomplished for each new comment, and this could easily become tedious and effectively reduce the number of comments a reviewer might choose to make. 

A better solution for enabling peer review, delivering feedback, and creating annotated video models is a free application called VideoAnt (http://ant.umn.edu).  Among other formats, this tool accepts the URL of a YouTube video and wraps an annotation interface around it, never affecting or covering the source video.  The interface essentially consists of three pieces:  a viewing area on the left, a series of stacked annotation cells on the right, and a timeline beneath the viewer.  As a reviewer plays the video, she can add an annotation at any point.  A pushpin marker appears along the timeline to indicate that an annotation appears at that exact moment in the video.  When the reviewer is finished, an email is sent to her with URLs for “VideoAnt.”  (Actually, there are four URLs:  1) view only, 2) editable, 3) embed, and 4) rss feed.)  The URL for the VideoAnt can then be sent to the student, and she can watch her video with a column of comments that automatically becomes highlighted at the predesigned moment in the project.  The process is simple, the author never loses control of the video, and the reviewer can choose to share either an editable or view-only version of her comments.  The final product is multimodal, splicing together traditional written feedback with a multimedia text. It's an easy assessment hack that brings additional rigor to new media.

Now, with the ability to provide situated feedback at specific moments within the video, assessment can begin to drive the assignment.  What needs to be added to the technology, though, is a general set of categories that can inform the student’s work and the reviewer’s comments.  Of course, the considerations you choose will depend upon the type of assignment.  If it’s a digital story, you’ll want to emphasize storytelling elements more strongly than if the assignment were, for example, a “public service announcement” that is meant to simply explain a topic.  Below, in Table 1, is an initial list of categories that can help you refine what you want to emphasize in the initial assignment and in the feedback you return to the student.

A more refined set of questions can also guide peer feedback so that students can provide each other with targeted feedback. Additionally, you could combine a subset of criteria and ask students to turn in an annotated version of their project which might reveal their awareness of what they accomplished. Why did they choose a particular image or scene? What did they intend with key decisions they made in editing the draft? The final product--an annotated video--then becomes an amalgam of product and process, and guides students to reflect on detailed aspects of their assignment.

 

Table 1:  Categories and considerations for considering student media projects.


Planning

Will the student turn in a portfolio of materials with the final project?  What might this include?  (Notes, storyboard, script, reflection, peer review forms)  Will your assessment emphasize process as much as the final product?

Research

Will you require the students to perform research?  Will you give them guidance for what constitutes an acceptable level of research?  Is a bibliography required?  Must they conduct interviews?

 

Learning goals

Is there evidence that the student learned the subject matter?  How will you know this? Did the student choose an appropriately challenging topic? Is relevancy important for the assignment?

Assignment parameters

How important is it that students follow directions?  For instance, is the final product the appropriate length?  Did it include the stated number of elements (interviews, images, resources, tags, facts, links, written text, etc.). Would you mind terribly if they jailbroke the assignment?

Reflection

Do you require a final reflection on the project?  What might you consider a rich or impoverished reflection?  Is the reflection to be based on the final product (a self-assessment) or on the process?

Originality

Was the project original?  What might originality look or sound like?  (Is it the subject matter, is it apparent in the editing or the application of sound?)

Resources

Are the sources cited appropriately within and at the end of the project? Are they aware of creative commons and alternative forms of intellectual rights management?

Storytelling

Directness, concision; rhetorical effect; audience awareness?  Do you want students to declare their intentions in written form? Will they do this in a video annotation program or in a separate reflective piece (or both)?

Analysis

Did the student sufficiently analyze the issue?  Was the conclusion relevant and logically drawn from the presented material?  If there was an attempt to persuade the audience to side with the author, was it a credible argument?

Peer Review

Did the student give and accept constructive feedback?

Technical proficiency

 

Are the production values important for the assignment?  Is there a minimum threshold you expect?

Freak Factor

Will you reward an amazing failure, a brilliant tangent, a glittering pastiche that reworks your assignment?

 

Many instructors don't like the word "assessment" because it tends to be associated with rigid standardization, institutional accountability, and overreaching metrics. But assessment, as I mean it in this context, is implicated in the teaching decisions we make every day. We make really good choices not to assign certain projects because they resist efficient or nuanced grading. Solving the assessment issue behind a new technology essentially makes the technology viable in the course, and increasingly, we're going to need to solve assessment issues more quickly and more deftly. For the Academy in crisis, and another nod to Yogi here, "it's getting later earlier out there."

Although we now understand that students are not as technologically literate as we might have originally thought, we also know that more tools and tutorials are available for them to teach themselves how to create new media. What they need from us is guidance to know if what they are creating is effective communication and analysis. What they need from us are opportunities for significant and relevant learning. Smart assessment technologies and sensible commenting strategies provide the framework for this kind of work and can drive course redesign.  And it’s this powerful combination that makes me assert that significant education is ninety percent good teaching and the other half is strong assessment.

 


The views and opinions expressed in this page are strictly those of the page author.
The contents of this page have not been reviewed or approved by the University of Minnesota.