Props Can Make the Game

I know …..I know what you are thinking….

“The last thing you need is to spend more money on teaching….. you don’t have the time to jazz up your in-class activities with silly posters or toys….”

But the truth is…

I can’t hear you

over the sound of my new GIANT INFLATABLE 20-sided DICE.

inflatable-20-sided-die

Sometimes if it is more fun for you… it is more fun for them. Cost… between 10-20 dollars shipping included. There are several retailers out there…. email me if you need a list. Happy Saturday.

Does Size Matter? The Perils of Small Classes

JawsSome of my courses this semester have such small enrollments that team-based projects and classroom activities have become problematic.

Balloting in presentation competitions is one minor example that I have apparently fixed. I regularly hold classroom competitions in which teams of students present on a topic and then select the winning team by voting with Monopoly money (see here, here, and here for more details). When voting, students come to the front of the room and place their money on placards with team numbers written on them — the only rule being that students cannot vote for their own team.

In the first two competitions in one course I noticed little variation in the sums of money received by different teams, even though some presentations were clearly superior to others. I asked the class about this unlikely result. I found out that in such a small class where everyone has interacted in some fashion with everyone else, students did not want to risk publicly embarrassing badly-performing peers or be judged as cruel, so they distributed their money evenly across teams regardless of presentation quality. My solution? Create a ballot on Google Forms through which students can anonymously select the best-performing team. After the last presentation of the day, I email students the link and they submit their responses. Students’ votes now better reflect the quality of the presentations. I also find that it’s easier to tally results with a spreadsheet on Google than to count paper money.

I don’t know a similarly-easy solution to other size-related problems. I recently used an ICONS simulation for the first time (more on that experience in a future post) in a class with only eleven students. Six nation-states comprised the simulation’s actors, so five states were represented by two students and one state by just a single student. If the latter student had been absent on either of the days I ran the simulation, or if two students from the same team had been absent, the exercise would have been wrecked.

In the same course, I had organized a significant portion of classroom time around a sequence of eight topics. For each topic, a student team was supposed to give a mini-lecture and then lead the rest of the class through an activity for some active learning-based peer-to-peer instruction. Then in the subsequent class, students take a quiz on the topic to reinforce the previously-presented material. With only eleven students, I could form only four teams, which left me with the task of delivering the content for half of the topics in the syllabus — defeating the main purpose of the exercise for fifty percent of the time.

Given that my students have priorities in life that often conflict with attending class, I now believe that teams should contain at least four or five students. With only two or three students, there is too high a probability that most or all members of the team will be absent on any particular day. This in fact happened yesterday: only a single member of two different teams attended class.

But if I increase the size of teams in a class with ten to twelve students, that means only two or three teams. Such a small number of teams is too few to expose students to multiple perspectives on a particular topic by means of competitive presentations. It is also too few to adequately incorporate peer instruction across an entire semester, unless the teams that exist are responsible for teaching multiple topics, and this increases the risk that students will be exposed several times to a terribly-performing team.

Why not return individually-completed projects and presentations? I prefer to avoid this option, because it gives students the message that collaboration is unimportant. As I’ve said before, group work matters. For example, individual writing assignments mean that I am the only person who ever reads what students write, which diminishes the authenticity of assignments and leads to lower quality work. And while it may be possible to squeeze as many as six individually-delivered presentations into a 50- or 75-minute time block, a class with six or fewer students doesn’t meet the minimum enrollment requirement at my university. It probably doesn’t at yours either.

Structure and Agency in Quality Enhancement

book_pillow_vctz6
Sarah’s plan to learn by osmosis wasn’t going well

Lately, I’ve been thinking about what makes a ‘good’ learning experience for students. There are a number of reasons for this, not least of which is the impending visit from the Quality Assurance Agency this autumn.

The question matters because it forms the bedrock of the educational mission of any institution and if we don’t really reflect upon it, then we risk making mistakes or, at the very least, not making the best of the situation we have.

With this in mind, it seems to me that there are at least four elements at play here.

  1. The students. Learning needs learners, so students have to be at the centre of any learning environment. This means we have to play attention to both the intrinsic and extrinsic aspects of learning: building on their desire to learn and providing them with an appropriate motivational architecture to help move them along through their studies.
  2. The teaching staff. For all that we talk about the use of active- and student-led-learning on this blog, that still requires an engaged and committed teaching staff, to design and facilitate those activities. The step back from the passive model does not mean that staff simply fall to the side.
  3. The learning activities. It’s all well and good to have the bodies, but they need to be doing something that enables them to have the opportunity to access the learning objectives that they set out to achieve.
  4. The learning environment. This is perhaps more neglected than the others, but the space in which learning occurs is also consequently. This should be understood in both the narrow (what’s my classroom like?) and the broad (what’s my institution like and what does it claim to achieve?) senses.

Hopefully, some brief reflection can help us see how these four elements come together in our own cases to shape our practice and our efforts to improve it. Likewise, we can see how difficulties in one area can affect the others and require us to take compensatory action.

To take one example, our students here at Surrey have been changing over recent years: the raising of entry requirements has clearly shifted the type of person who comes through our doors. That has had implications for our teaching, since the prior knowledge, the motivation and the anticipation of those new students has had clear impacts on what and how we create learning environments. To a certain extent, our pre-existing focus on active learning absorbed some of this (since these tend to be more openly constructed), but it has still required much thought about the nature and scope of provision.

The difficulty comes largely in predicting what matters.

The list above is an analytical one, rather than a predictive one – it says we should look at these different elements, which all interact with one another.

While that’s true, it is still helpful to engage in that reflection when reviewing curricula to ensure that as much as possible of ‘what matters’ is noted and accounted for.

When Grade Inflation Isn’t

Norman Mailer Report Card

Courtesy of Charles Gleek at Games Without Frontiers,  I became aware of an interesting discussion of grade distributions at TPRS Q & A:

What grades should kids get? Notes on Evaluation for the Mathematically Challenged.

It’s a bit long but worth reading. The author, Chris Stolz, points out that perceptions about “proper” grade distributions are sometimes based on ignorance of basic statistical principles. Classes frequently are not statistically representative — they contain too few students, and students choose to enroll in them for non-random reasons. Assessment instruments often produce a ceiling effect that masks evidence of improvement in students who come into a course already possessing a high level of proficiency.

The end result can be a class composed of students who are either predisposed to do well in the course (possibly the main reason they enrolled in it to begin with) or who learn enough over the course of a semester to earn high marks on summative assessments. This reduces variation in the grade distribution and skews the curve to the right — instead of a normal distribution from F to A, with most students getting a C, the majority of the class ends up with A’s and B’s. A person who does not understand statistics assumes this happened because of grade inflation.

Several years ago I abandoned grading students against a normal distribution curve for these reasons. I also became much less concerned with testing students’ ability to reproduce factual information on “objective” exams because I knew that the vast majority of what they regurgitated would never move into their long-term memories. I did not (and still don’t) believe that their lives would be fundamentally altered for the worse if they failed to remember that the Turks captured Constantinople in 1453 or that realist IR theory derives in large part from the writings of Thucydides, Hobbes, and Machiavelli. 

I thought that students would benefit more from multiple opportunities to demonstrate how well they could apply concepts in novel ways and effectively communicate their findings. How does this look in reality? Below is my grading system for a course that I’m teaching now. 

Grading Scheme

In this course, final letter grades are based on a 1,000 point scale, in which students need only earn 950 points to obtain an A. Obviously with a total of 1,080 points available, it’s quite possible for a student to earn a high grade if he or she simply keeps plugging away at all the various assignments. But this is exactly what I want — for many students, continuous effort will result in improvement across the semester. Constant practice is also makes it more likely that students retain something after the course ends. And students feel better about themselves and their environment with frequent feedback on their performance.

Since this system of assessment makes it more likely that students will be able to demonstrate proficiency by the end of the semester, my grade distribution shifts to the right. Is this grade inflation? I will argue that it isn’t, because the student’s final grade is not based on a hastily thrown together end-of-semester essay that the instructor simply marks as an A or B.

Back to Basics with CATs

Two Eye Color CatAs promised in a previous post, an example of the usefulness of CATs:

I’m teaching comparative politics this semester, and in this course I divide the content into five geographic regions. and four themes (formerly five themes, but I spun one of them off into a separate course). For each region, students have the following assignment:

Write an essay that uses a single theoretical perspective (rational actor, structural, or cultural) to explain the political events described by the assigned readings.

The first two iterations of this assignment did not meet my expectations — overall the class had done a poor job synthesizing information and presenting coherent written arguments. I looked through CATs for a technique that might work as an in-class writing exercise and found the “one-sentence summary,” which I modified. I gave each student a copy of the following text, created by yours truly but reflective of students’ writing:

I feel as though without a strong and effective ruler being in command a country will either have a revolution or the people will be politically oppressed. A democracy requires economic growth, import and export markets, an education system so people have well-rounded knowledge to participate effectively in elections, and the right culture. The readings discussed South Africa’s modern economy being the product of technology which in turn was able to create democracy. However, in other countries the political leaders control power rather than the people and this results in democracy depending on them by means of their interests. Although the readings we read about sub-Saharan Africa had many different views on the economies of African states all of the readings emphasized the role of political leadership in a rational actor mindset in order to create the democratic systems that they do have.

I gave students ten minutes to edit this passage to make it better address the assignment instructions. I projected the instructions and the passage on the wall screen.

After ten minutes I had each student come to the front of the room and use the classroom computer to make a change to the passage. Other students provided input and I facilitated discussion.

By the time the last student was finished making changes, the passage had been reduced to a single sentence. It wasn’t the ideal thesis statement, but students seemed to understand my point that it’s important to concisely state one’s argument at the beginning of an essay or a presentation.

I then demonstrated how the easiest way to construct a thesis statement for this kind of assignment is to simply reword the question asked in the instructions — in this case, something like:

The rational actor theoretical perspective best explains political events in sub-Saharan Africa.

Once this was accomplished, I had five minutes of class left to do a “wellness check-in” — I went around the room asking each student “How is life treating you?”

Collective Action Game: Failure in Action

So last night I ran a game that utterly failed.

It was in the first meeting of my upper level seminar on Environmental and Energy Security, which meets 1x week for 8 weeks in a 4 hour night class.  One of the key concepts I wanted to review was the Collective Action Problem, and naturally, my thoughts immediately went to how I could turn this into a game.

The game itself is below, but I thought it might be interesting to discuss how I created it and how it went so wrong.  Designing a game is a constant work in progress, and we shouldn’t be afraid to test it out on our students and see what happens.  It helps improve the game for the next time around, and I’ve always found that even when the game doesn’t go exactly as planned (or, you know, dies horribly while 20 undergraduates stare at you, expectantly), there is usually a relevant lesson for the players.  Luckily, that’s what happened this time.

Collection Action, in Action

The core of the collective action problem is trying to understand the ways in which individual self interest can preclude group interests.  If contributing to the group interest poses a cost, individuals who may value that collective goal may still fail to contribute, perhaps hoping to free ride on the efforts of others.  This is particularly a challenge in larger groups, when failures to contribute may be less likely to run social risks than in smaller groups, where contributors can be socially rewarded and shirkers, socially punished.

What I wanted, then, was some kind of decision-making game where students had to choose between a self-interested option and a collective option.  I quickly zeroed in on extra credit as the self-interested option.  The challenge here was figuring out how much extra credit was enough to tempt the students away from the collective option, without being too high.  I went with 20 points– my course is out of 10,000 points, so this seemed reasonable at the time.  As we will see, it was far, far too low to incentivize the students to behave selfishly.

For the collective interest, I considered two options.  One was some kind of group prize, such as baked goods. The other was some higher amount of extra credit.  What I did decide was not to tell the students exactly what their collective interest would be.  This was because my class was full of former students of mine, many of whom have tried to break my games in the past. Limited information was therefore my friend, or so I thought.

For game play, I also wanted to see what role social norms would play.  That meant playing the game anonymously the first time, and then openly the second, to see what kind of impact it would have if they would have to ‘out’ themselves as self-interested players.  This part also did not turn out as planned, by the way.

Here’s what I did:

The Collective Action Game

In round 1, students had to take out a piece of paper, write their names on it, and then indicate whether they chose option A, which was an automatic 20 points of extra credit on their final course grade, or option B, which was a contribution toward the unknown group prize.  75% of the students had to pick option B for the collective good to be provided. Choices were kept completely anonymous, but I announced how many students chose option A and B.

The collective prize was 100 points of extra credit–reduced by 20 points for each student who chose option A (basically, they were taking those points out of the collective pool).  In my class, 4 out of 20 students chose Option A, so that left 20 points for the collective prize.

Round 2 was exactly the same, except now Option A was worth 80 points, and the unknown collective prize was baked goods for our next class. Again, I did not tell them the prize until after the results were announced.

Round 3 had an 80 point option A and a 200 point extra credit prize, but was done openly–self-interested players came to the front of the room, and collective players went to the back of the room.

So how did that go?

The Disaster

Well. Here’s what I expected to have happen: in the first round, most students would take the self-interested route, would be shamed by the collectively oriented players, and then in Round 2, the public round, most students would act in the collective interest.

That’s exactly what DIDN’T happen.  16 of my 20 students chose Option B in round 1.  Although they expressed anger at the 4 unknown students who chose Option A, many (though not all) agreed that they probably would have taken the selfish route if the incentive was higher. The public round 2 was supposed to provide contrast, but suddenly, there was no contrast to be had! So on the fly I made up a new Round 2, a private game with a higher extra credit incentive for the self-interested option.   Maybe 80 points was still not enough, though, as this time 17 students (1 MORE than before) chose Option B!

At this point, I was seriously concerned about the game failing entirely.  I’ve written before about game failure and how it can present learning opportunities, and we’ve discussed it on the blog, but that doesn’t mean it’s a a fun experience when you are facing a new class on the first night and your new, untested game is completely failing to work as planned. I was pretty much resigned to having to admit to the students that the game didn’t work, but decided to try the public round anyway.

In the public round, remember, I had anticipated 100% compliance with the collective option. Imagine my surprise when the social norms I expected to make everyone choose Option B resulted in the exact opposite–8 students came to the front of the room, going for the 80 points, and as soon as the ones in the back realized that this meant the collective good would not be provided, they tried to run up front so they too could benefit.  Instead of promoting cooperative behavior, the ability to see what their peers were doing resulted in higher self interested action!

Total. Game. Failure.

By that I mean that I had created the game trying to engender a particular outcome (a rookie mistake, by the way), and the students behaved precisely the opposite of how I had hoped and planned.  The game, in other words, failed to produce the behaviors I was hoping to discuss.

But I had two things going in my favor. First, and less importantly, I know that failure still provides opportunities to learn, and I was able to make some connections with what did happen that could still illustrate the lessons I wanted.  More importantly, my students had NO IDEA that the game hadn’t worked as planned. I didn’t let on, and therefore they treated it as if I, the all-knowing instructor (hah!), had anticipated their every move.  That meant that during the debrief, they were searching for ways to make the game relevant to the content–and they found them.   They were able to teach themselves and their classmates all kinds of lessons about collective action, showing that even if they hadn’t learned what I’d originally planned, they still learned quite a bit.

Salvage Mission

We had a wide ranging discussion about a number of points. First, the ability to observe and monitor fellow group members matters.  Social norms matter–but if the norm is to be self-interested, then there is no danger in behaving that way.  Likewise, observation allowed students to change their behavior based on what their peers did–which is the essence of free-riding behavior.  In this case, too many students free-rode, preventing the good from being provided.  Another point this showed is that collection action is easier to achieve in a small, highly connected group.  There were only 20 students in my class, most of whom are junior and senior majors that have taken many classes together.  Their cooperation is, therefore, precisely what I should have predicted.  We were able to discuss what the game would have looked like if played in class of 300 or so, and they agreed the outcome might have looked different.  We were also able to discuss how the incentive informed their decision–noting the crucial role that incentives play in motivating their own personal behavior.

The game clearly needs some tinkering and refining, but I don’t regret playing it.  It will give us a common reference point for the rest of the term, and any opportunity to get the students active and moving about during a four hour class is welcome.  Ultimately it teased out several crucial points about collective action that I wanted the students to understand, even if it wasn’t the points I originally intended.  And it also re-taught me the crucial lesson that failure is just a starting point for learning, for both me and my students. I may have failed, but I’ll still call that a win.

Teaching with social media

Facebook_like_thumbReading Casey’s post, I have been reflecting on my own use of social media in the classroom. Here at Surrey, we’ve long been active in this field, because we recognised the value of developing new spaces of communication as a way both of getting more from our teaching and of preparing students for the world outside the classroom.

The value of that has become clearer over time, not least in the facility that our graduates have demonstrated in using those media to make themselves more attractive to employers and to support the community of learning they developed during their time with us. For our part as educators, we have see real benefits from connecting and engaging with a far wider community than might have been possible in the bad old days of the 2000s (sic).

But all of this raises a number of questions, not least of which is the matter of how one gets to a situation where social media can reasonably be brought into the classroom.

In part, this is a transitional issue, since usage of social media becomes more and more pervasive. Certainly, as everyone seems to be using social media, so the pressure to join in becomes ever higher, and the barriers ever lower: the amount of technical expertise needed to use Twitter (for example) is minimal, even if optimal use still requires some work.

However, we have to recognise the limits to this: I still encounter groups of students who have no experience at all with particular media (including my own class, last week). Precisely because there are so many platforms to choose from, there remains a distinct possibility that your students won’t use (or know how to use) the platform you want. Put it like this: none of us use Pinterest.

In such cases, either you have to train people up – as I’ve done for my Twitter negotiation – or you have to use a Bring-Your-Own-Device model, where the substantive content can be accessed via multiple platforms. The latter is more flexible for students, but requires much more technical expertise on your part.

Surrey’s approach has been to create a more general environment in which social media are mainstreaming into different parts of our provision. That means active Twitter and Facebook pages, which get used to connect prospective and current students, programme information and specific content within modules. By trying to link together elements, we raise the overall visibility and introduce the different platforms to users.

A good example of this is our use of hashtags in Twitter, to highlight particular campaigns: currently, our #PoliticsMonth events are bringing in a range of activities to the university, and students can not only see those more easily, but also contribute more easily. When I chaired a hustings for the general election for our Politics Society last month, I could gather questions from the floor using the event hashtag, which meant more efficient gathering and organising of their input to the panel. A similar system could be set up for large classes, with the lecturer getting instantaneous feedback to their device during lectures, which could then be fed more seamlessly into the content.

Likewise, the encouragement of students to use social media to talk with each other and with teaching staff provides a rapid and accessible means of supporting student learning: a tweet or a Facebook post can be read by others who might be in the same situation. In terms of general student support, this has been a great boon, especially for our students out on professional placements, often overseas.

The keystone in all of this is then a degree of engagement by staff. If they don’t use social media, and use it often, then it’s very hard to get students to do the same.

In that, I have been very fortunate to have the group of colleagues I do, since we have had a very high level of buy-in to the social media work. That has covered Twitter, Facebook, blogging and other platforms. Importantly, it’s also required a maintenance of activity over a long time-frame.

Indeed, that time-frame is effectively open-ended: we’re about three years into this and we’ll carry on as long as necessary. That needs someone to keep reminding people to do it, until it becomes almost second-nature: certainly, many of my colleagues use social media very reflexively, which is easy for them and reinforces the message about the value of doing it for others.

Even without the kind of massive investment that some other units have made into this field – LSE is a good example of what can be achieved online – I would hope that our example will show how even a modest application of effort can have significant pay-offs.