14 May

Assessing participation in an online learning community

A couple of days ago, I spoke at a PD evening for high school teachers on using technology, specifically about how I use social technologies for connected learning. There was some interest in how I assess students’ contributions to the learning community and I was asked if I would be willing to share the criteria I use for this.

I think it’s important to provide some context for what I do and why I do it this way, so I want to share a bit of background, too.

How I came to assess students’ contributions in online learning communities

In my first year of teaching, I had a real bee in my bonnet about online students’ engagement – with the learning materials, with me, and with their peers. So when I had the opportunity to design a new unit that year, I built the assessment around engagement, in addition to the learning outcomes. It helped that in this particular unit, students were learning about personal learning networks. This meant I had both pedagogical and content-driven motivation to get them to talk to each other, to me and to professionals in industry.

I had a hunch that I would need to assess student contributions in the online learning community in order to successfully foster engagement. And as it turned out, that year, I learned that by assessing student contributions in an online learning community, I could draw them into participating in a meaningful way. Some colleagues and I did a bit of a project about online peer engagement and interviewed students about their experience in the learning community in this unit. We also interviewed students from an accounting unit and an education unit about their online peer engagement. We looked at three different online peer engagement activities across the three units: in the accounting unit, we looked at the use of a discussion forum; in the education unit, we looked at Collaborate-based online tutorials; and in my unit, we looked at the whole approach to the unit, which was essentially what we might now call connected learning (but we didn’t have a name for it at that point – or at least, I didn’t have a name for it). In a nutshell, I ran a WordPress Multisite installation where the students each had their own blog, and used BuddyPress to turn the site into a social network. This approach was inspired by Michael Stephens’ and Kyle Jones’ work with WordPress and BuddyPress with a course at San Jose State University. I also assessed students’ participation in the learning community. Their participation involved commenting on my and their peers’ blog posts, having discussions on the site using the social networking functionality, and engaging on Twitter through conversation and using the class hashtag to share interesting material related to the unit.

In the interviews, my students reported they started contributing to the learning community because they were required to for assessment, but they kept on contributing because they found it valuable. Assessment drew them in, but they kept contributing because it was beneficial. And that’s what it’s all about, right? Creating experiences that are valuable to students, not just experiences they have to engage in to satisfy the course requirements.

But how do you assess contribution to the learning community?

The answer is: it depends on what you’re looking for.

I was looking (and continue to look) for meaningful engagement with the content and robust discussion and conversation. I wanted students to form a personal learning network, both within the class, and beyond. I didn’t just want them to connect with each other, but also with industry. I wanted them to feel empowered to learn by playing with technology and critically reflecting on their practice. I wanted them to support each other, to demonstrate leadership, to role model playfulness and personal investment in ongoing continuing professional development.

In that first year, I developed assessment criteria for participation in the learning community that were fairly basic. I also negotiated the assessment criteria with students (which I do in most of my units), so they had the opportunity to tell me what they thought participation should look like. The next year, after conversations with a colleague who does something similar, I added in criteria about demonstrating leadership. And the criteria have continued to evolve each year.

So what exactly do I assess?

When I grade students on participation, I’m looking at

  • analysis and critical discussion
  • leadership
  • extent of contribution.

I don’t know about you, but I really dislike developing criteria sheets (aka rubrics). I dislike it because I invariably come to use them and find flaws that make it really difficult to mark against them. I also find it tricky sometimes to find the right words to convey what I’m looking for. So I really labour over these things and at the end of each semester, I make notes about what worked and didn’t work, and I iterate them from semester to semester. I also trawl the web and academic databases to look for rubrics or criteria sheets that might serve as inspiration.

At my institution, we use criterion referenced assessment. My personal approach is to write really detailed criteria sheets so that I am completely transparent about what I’m looking for. I spend time explaining the criteria to students (although I aim to write criteria that don’t need explanation), and then I give them opportunity to think about the criteria and negotiate them with me. (As an aside, I find negotiating criteria a really powerful tool to help students take charge of their learning.)

Criteria sheet

Download my criteria sheet for assessing student contribution in online learning communities [Word]. It is by no means perfect, but you might find some of the words useful.

Over the years, I’ve been inspired by rubrics shared by other teachers – primary, secondary and tertiary teachers. I can’t remember all of the rubrics I’ve drawn from but this year, I found two in particular very helpful, and I’ve borrowed words from both. They are both from iRubric:

The practicalities

While I track the volume of students’ posts in our various online spaces, I am more interested in what they contribute, rather than how much they contribute (although the latter is important too). This means I need to look at students’ contributions, not just statistics, to grade them. This is not easy and it’s not quick, but I believe it’s worth the effort. When I use a WordPress blog network where students have their own blogs, in combination with BuddyPress, it’s relatively easy for me to go back and see what students have contributed. This semester, my students’ blogs are all over the web, so it’s a bit trickier to track their contributions.  I have a multi pronged approach this semester, but the main thing I’ve had to think about is how to capture students’ comments on their peers’ blogs in a way I can work with to grade them. I’m doing this by aggregating the comment feeds from my students’ blogs into a single RSS feed, which I’m scraping into a Google Spreadsheet.

If This Then That recipe for scraping an RSS feed into a Google Spreadsheet.

If This Then That recipe for scraping an RSS feed into a Google Spreadsheet.

A colleague of mine is manually tracking contributions this semester by tallying them in a spreadsheet. She is much more disciplined than me though! I wish I could commit to that level of tracking, but the reality is I’d fall behind and I’d invariably give up. So automating works for me!

I’d love to hear your ideas for tracking student engagement in learning communities, and any thoughts you have on assessing contributions effectively. Please share in the comments!

2 thoughts on “Assessing participation in an online learning community

  1. Hi
    I enjoyed reading your post. I am working as a casual tutor for the preservice teachers at CQ University. We use blog posts in some courses that I tutor to engage all students and assess their learning throughout the course. In one course they get 4 marks for each weekly post over a 5 week period. This is designed to ensure they are all ready for the next stage of the course. Works well and quality is usually high.

  2. Pingback: Using social technologies for connected learning | higheredukate

Leave a Reply

Your email address will not be published. Required fields are marked *