• The presentations on assessment were full.
• Assessment of game-based learning is challenging!
• According to David Shaffer current assessment models only focus on knowledge and skills, but that is only half of an epistemic frame where there are four interconnected aspects: knowledge, identity, skills and value.
• The proposed models for assessment are very resource intensive. In one it requires a teacher who is in almost constant dialog with each learner/player/game designer.
• Learning follows a trajectory, e.g. between point A (knowing nothing) to B (knowing enough). Of course there are other aspects to just the knowledge. Gee: in open-ended games, e.g. Oblivion, there are different paths to get to a point in the game. Over time those paths can be generalised into trajectories. So in the future the game might be able to tell a player, at any time, where they are on that trajectory. The game could even give advice for how to move players towards a desirable place. In theory, then, “testing” could happen based on closeness to trajectory and so a player — a learner — can “pass” as being skilled enough in a particular subject when they are far along enough a trajectory. Assessment then happens in situ, not on a given day in a room different from the classroom where the learning has happened.

GLS 2008 presentation: Increasing Player Engagement Without Breaking Your Budget: Simple Choices That Make a Big Difference.

Questions the authors, who are game designers, asked:

• What makes a game engaging?
• Why do people spend time playing games?
• As a designer, what choices will make my game more attractive?
• As instructor, how can I make my instructional games more effective?

To answer the question the authors created a very simply “game” — as quickly as you can click on a circle when it appears. They demo’d various versions of the game — each with additional features to increase engagement.

Within gameplay:

• Provide simple, ongoing feedback, e.g. half bonus, full bonus if click quickly enough
• Add a narrative
• Allow for some strategy

Within media:

• Add sounds
• Add colours

With pre- and post-tests on the four different versions of the game, the authors concluded:

• Different enhancements did not change level of motivation.
• Women liked the media version the most (they played longer, had higher perceived quality and time compression on this version), men liked the feedback version most. Thus, gender matters.
• Pre-game interest and expectation play a key role: If people were interested in playing the game, they appreciated the extra features. For those who were not interested, extra features did not impress.

This is an interesting pilot project, still a work-in-progress. The way they’re measuring engagement is interesting.

They’re going to be adding another two features in the area of Narrative: a bit of a storyline and characters. They’re also challenging anyone to take the core version’s source code and add features to make it more engaging, which will be tested. Rules: no porn, but violence is allowed.

GLS 2008 presentation: Creating a Culture of Critical Game Designers in Elementary Classrooms and Clubs.

Kylie Peppler, Alicia Diazgranados and Deborah Fields presented their findings from two different learning spaces, formal and informal, in which they used different strategies to establish a culture of critical game design among elementary age students with Scratch. Scratch is a new programming language for kids that makes media-mixing and the creation of designed artifacts easy.

The presenters, along with Yasmin Kafai, have been researching game design and, more generally, creative production as a pathway to critical reflection (Peppler & Kafai, 2007a/b). In other words, kids not just playing games, but actually creating them.

After all, as Katie Salen has pointed out:

“Beyond their value as entertainment media, games and game modification are currently key entry points for many young people into productive literacies, social communities, and digitally rich identities.”

According to the session presenters, there is ample evidence that youth are learning when they engage in computer and design programming, e.g. with Scratch. What is lacking is for youth to speak critically about their work and use the specialist language associated with learning in these subject specific areas. From the prior work in the arts we know that time for critique is imperative to help youth develop their lenses as critical and creative producers (Winner & Hetland, 2007).

The two presentations:

Formal setting: Critical Reflections With Scratch in a Second Grade Math Classroom (Peppler, Diazgranados)

• Prior work in the arts demonstrates that time for critique can help youth question, explain and evaluate (Winner & Hetland, 2007)
• A US math standard that is often overlooked: Making a choice and defending that choice using reasoning that’s convincing. In 2nd grade is the first time that learners get to make decisions about math problems.

Research question 1: What are the local practices to best prep the classroom and class for the exercise?

Background of the 2nd grade classroom:

• 80 learners (20 at a time in a class)
• Math classroom
• High numbers of special education students
• Urban school in LA Unified School District
• > 70% free and reduced price lunch (i.o.w. kids from low SES homes)
• English Language Learners (English not their first language)
• Minority youth: mix of primarily Latino and Korean youth
• Technology and games focus
• Each learner has a laptop in the class (different from SA!)

As part of the process of critical reflection in the classoom it is very important to first create a comfortable, safe space for expression and critique. An exercise:

• Play a game without instruction (become an expert)
• Then teach an adult how to play it (the strategies, characters, etc.)

Another exercise:

• When introduced to Scratch, the kids loved it. The kids had to create artefacts in Scratch, and then critique each others’ work using a given rubric. One group of 20 learners would critique the work of learners in another group. Peer-assessment at its best! On the back of the rubric was also an open-ended feedback space, e.g. What I liked about it, what I’d change about it, etc. And then the learners had to share that evaluation with the actual person who created the artefact.
• The real power behind the creation of the pretty products (outcomes) is the process and the critique.

An exercise:

• As a group they evaluated some of the existing Scratch games (developed by people outside of the class) They found that some games needed more levels, more characters, etc.

Notes:

• Much of what is covered in Scratch is beyond 2nd grade math curriculum, and they don’t have to know those things. What Scratch provides is an opportunity for defending decision making.
• Even kids who can’t read can program Scratch. Initially it is a matter of trial and error as opposed to free creation, but the kids are still learning the tool.
• Kids are natural data collectors. On the whiteboard the teacher showed stats on how the games were evaluated by the learners. This lead into math-specific discussions (addition, subtraction, etc.)
• Explaining variations by the learners.

How to scale this process? Not every teacher is an amazing Alicia?

• The team are working on a process, a replicable model.
• Provide scaffolding for teachers. Give them rubrics. But it must be flexible enough for teachers to take ownership of the process.

Research question 2: How might game design and play scaffold or promote writing and group discussions among predominantly ELL learners?

Kids who don’t like writing (in Language class) are very happy to write up their critiques in the maths class.

Research question 3: How to develop the specialist language needed for critical reflection?

In the critiques the kids begin to use that specialist language, e.g. game mechanics, interactive design, character traits, etc.

Media literacy has moved from the 1970s from a right-wrong answer approach to a more subjective, justifiably opinioned approach. This is in line with a participatory culture where being able to critique and voice a point of view is important.

Overall benefit of the project has been to cultivate a culture of explaining why. Not just “Coz the teacher told me that” but “This is why I think that …”

Informal setting: The Role of a Social Networking Site (ScratchR) in Supporting Reflection on Design in an Afterschool Technology Club (Fields, Kafai)

ScratchR is a platform for sharing programmable media on-line. This platform allows people to publish using the Scratch programming language. ScratchR is the engine behind the Scratch on-line community, a their own animated stories, games and interactive art made in the social network of novice programmers. Unlike other user-generated content communities, ScratchR makes it easy to reuse (we refer to it as creative appropriation) other people’s creations to foster collaborative learning. ScratchR allows members to rate, comment, tag and create galleries.

ScratchR is to programming what YouTube is for video production

The research project, which set out to analyse the role that ScratchR had on stimulating critique and generation of designs in an unstructured setting, took place at an after school tech club:

• 9 x 4th-6th graders
• Met 3 times/week for 2 months (approx 24 meetings)
• Played with Scratch and then ScratchR

Types of participation in ScratchR:

• Browsing and playing projects
• Managing image: creating their online identity, choosing which projects to share, looking for comments on their work, “friending” others
• Critiquing and praising others’ work, e.g. on games: “it’s a glitchy game”
• Design work: downloading, uploading, remixing

From club to wider community

• Weeks 1-2: nothing
• Weeks 3-4: club focused (friending, image management, browsing)
• Weeks 5-7: wider community (from friending and affinity browsing — looking for Halo or Soulja Boy projects as opposed to Most Viewed — to downloading and remixing)

In one exercise the kids had to create geometric art projects that they had to upload to Scratchr for “expert feedback.” Yasmina and Deborah created alias accounts and gave feedback. The learners thought that their pieces were complete upon upload, but the feedback encouraged tweaks to make. Getting kids to revise work is very difficult, usually. But after this style of feedback, the kids extended their pieces to create improved, more complex pieces.

Concluding thoughts:

• Scratchr provided an opportunity for identity construction
• Stages of use — in first weeks they didn’t upload anything because they didn’t feel they had anything to show. From nothing, to local (club) to broader community (whole of Scratchr)
• Provide multiple entries for challenge and design — first exercise is to create your name, then increase challenge for the learners over time.
• Collective projects (remixing, working with others on a project, etc.)

Challenges:

• Bringing social networking into the classroom
• Allowing Google image searching for artefacts to bring into the projects

See also: ScratchR at MIT

ED-MEDIA 2008 paper: Using an online journal (blog) in EFL writing: Learners’ perspectives.

Abstract: This study seeks to research on blogs in education by investigating students’ perspectives. Students from two different junior high schools in central Taiwan were invited to participate in this study for eight weeks. Both qualitative and quantitative approaches were adopted, including interview, works on blog, and questionnaire. This study revealed that students had positive attitudes toward writing online journals with blogs. Other important findings yielded from this study are the following three themes. First, it engages students in critical reading and reflection and enhances their writing knowledge and skills. Second, the features of web links and the unpredictable feedbacks from peers attracted participants greatly. Third, writing diaries on a blog is not only a pure writing practice, but also is an activity of exchanging information and interacting with authentic context outside of classroom.

———————–
Authors: Shih-Ru Lin, Chung Shan Medical University, Taiwan; Shwu-jiuan Huang, Chaoyang University of Technology, Taiwan

I get to attend many conferences and take as many notes as possible to share with others who can’t be there.

2009:

• Web4Dev, New York, USA (February)

2008:

• Handheld Learning, London, UK (October)
• mLearn, Wolverhampton, UK (October)
• Schools ICT conference, Cape Town, SA (October)
• Rural Education Project conference, SA (August)
• ED-MEDIA, Vienna, Austria (July)
• Games+Learning+Society, Madison, USA (July)
• eLearning Africa, Accra, Ghana (May)