Cathy Elliott: Creativity and feedback in assessment portfolios

Kristina Hoeppner 00:05

Welcome to 'Create. Share. Engage.' This is the podcast about portfolios for learning and more for educators, learning designers, and managers keen on integrating portfolios with their education and professional development practices. 'Create. Share. Engage.' is brought to you by the Mahara team at Catalyst IT. My name is Kristina Hoeppner. Today I'm speaking with Dr Cathy Elliott, Vice Dean Education in the Faculty of Social and Historical Sciences at University College London, whom I'm meeting for the first time.

Aurélie Soulier, with whom you worked at UCL, Cathy, sent me a link to your wonderful short overview video on how you use portfolios in one of your political science classes, which prompted me to ask if you'd like to share your portfolio work with our listeners. Thank you so much for taking time out of your day, Cathy, to talk with me.

Cathy Elliott 00:59

Thanks so much for having me on.

Kristina Hoeppner 01:01

Cathy, can you please tell us a bit more about your role and nature of teaching at UCL?

Cathy Elliott 01:08

I'm an academic. I'm a political scientist. My research and my teaching is by large on the politics of nature, which is the class that you saw the video for, and that's about the politics of nature. I also have quite a focus on education and pedagogy. Jointly with some other colleagues, I run a Centre for the Pedagogy of Politics. As you said, I'm the Vice Dean Education in our faculty. So I'm particularly interested in sorts of innovative pedagogies and trying to think about how we teach politics and how we can do it in an interesting and innovative way.

Kristina Hoeppner 01:38

You are the first political sciences instructor that I'm talking about. It's fascinating to listen to all the different people in which areas, they're using portfolios so that we can show our listeners that portfolios really can be in any field of practice, and also any field of studies.

Cathy Elliott 01:57

Yeah, I think some of my colleagues are starting to get interested in using portfolios as well. So there may be a few more political scientists starting to do this stuff as well.

Kristina Hoeppner 02:05

Wonderful, wonderful. Before we started recording, you had shared with me that you're pretty new to the world of portfolios. 

Cathy Elliott 02:13

Yeah.

Kristina Hoeppner 02:13

How did you get interested in using that teaching method?

Cathy Elliott 02:18

I think as any sort of educator should, I started out from what I wanted the students to do. So I had a fairly clear idea in my mind of what I wanted to happen as part of the assessment, but I didn't really know how I was going to do it. I went and spoke to a variety of learning technologists, and every learning technologist that I spoke to referred me to somebody else. And I had some brilliant conversations about different tools I could use. So we talked about blogging, WordPress, all sorts of things. Eventually, I ended up being referred to Aurélie Soulier, who I know, you know, and who's amazing. She showed me what you can do with portfolios and talked me through it. And it was just really exactly what I wanted to do with the students. So that was how I first got interested in it.

Kristina Hoeppner 02:59

In your video, you mentioned that the students therefore then have a private space of their own, that you also emphasise choice, and that students can reflect on their experience. How have your students taken you up on working with something so very different, I assume, than what they have been using in political science classes until then?

Cathy Elliott 03:23

Yeah. So it's a very interdisciplinary module. I get students from different departments. Most of them are political scientists, but not all. To start with, they were nervous about it. So I got them to fill in a questionnaire at the very start of term asking them - I asked them all sorts of questions at this questionnaire, but one of them is kind of 'What are your worries?' Most of them said something along the lines of the assessment, 'I don't know what this is. It's very unfamiliar. It seems like it might be a lot of work and a lot of time.' And so there was quite a lot of anxiety about it. 

And it's really interesting now because now I'm reading their work because they've just handed it in. There was another questionnaire as a kind of cover sheet when they handed the portfolio in. I asked them 'What was nice about the module? What did you enjoy? How did you feel about the assessment method?' And so far, there hasn't been a negative comment. They've all said they really enjoyed it. It's been great that's encouraged them to work steadily. The thing they've really, really liked is having that creativity and being able to be free to sort of explore what they were interested in as part of the module.

Kristina Hoeppner 04:24

You mentioned the questionnaire that you asked your students to fill in at the start and now that second questionnaire that you're having towards the end of the module. How did you address the concerns of the students then once you've read their first comments?

Cathy Elliott 04:40

Lots and lots of scaffolding. What we did was we made sure the students were in a group on Mahara and we pushed out to them all a template. The template was broken down by weeks. So they could mess with that if they wanted. They could change it, they could do something different, but as a default, they had a space every week where they could put in their work on MyPortfolio. At the top, every week, there was a set of questions and prompts that they could respond to. Nobody sort of sort of sat there with a blank page thinking, 'I don't know where to start.' There were instructions. Everybody had a place to start. There were things that they could think about. 

I strongly encourage that sort of culture of feedback in the class so that they would be able to see each other's, get inspiration from each other's, try and think about what each other were doing well or not doing so well, and I think that addressed a lot of concerns. 

We also worked really closely with Aurélie as well. So she came in the first week, and we had a session together, where she showed them just the technical aspects of like, what this piece of software is, what you can do with it, what are the possibilities. So they weren't scared. They knew what to do. And they also could go to office hours with her on a regular basis. And she said, usually, the questions they had were quite little, like, 'Oh, you know, my pictures are landscape and I want them to be portrait, how do I turn them around? Or how do I put in a caption or I've seen somebody else do something really cool with the background. How do I do that?' Little things. She said, a lot of it was actually of answering those little questions, but also giving them confidence to feel that they didn't know what they were doing, and they could use a piece of software like this and develop in that way.

Kristina Hoeppner 06:12

Sounds like most of the questions were of a technical nature, not so much on the pedagogical side, would that be a fair assessment?

Cathy Elliott 06:20

They were interested in how free they could be. The answer from my point of view was very free. But I think students are very used to being hemmed in. So they're very used to being asked to write a kind of very standard essay, and the instructor has very specific ideas about what they would like the students to do. Then there's quite a lot of criminology among students trying to read the professor's mind. So I had one student in class said to me, you know, 'Usually I just go to office hours with the professor, and I find out what they think, and then I just write that, and then I get a good mark.' Quite cynical [laughs]. I was trying to encourage them not to do that. To do work that they were interested in to then to explore their ideas, to pursue the lines of enquiry through the module that they found exciting, to go wild, to be creative. 

They've really risen to that challenge. I think they were quite excited to know that they could, that they were allowed. So students have done things like they've put their artwork on the portfolio. And some of it's incredible, and it conveys so much. Conveys so much learning. They've put their own photographs, some of them have written poems. Some of them have written stories or dialogue or narrative non-fiction, Others have written it more than the style of a blog. Some people have written sorts of policy memos. There's a whole range of different things that they've decided to do, and no one portfolio is the same, which is super nice when you're marking because [laughs] we read the same essay over and over again, as I'm sure everybody will identify with, it's very, very tedious. This isn't like this at all, you know, every portfolio is different, and they've really indulged their creativity, which is super - it's really very nice to see.

Kristina Hoeppner 07:54

It also sounds like your students are very much taking you up on that invitation to bring in their own personality and also to show really who they are.

Cathy Elliott 08:07

That's right. I think students come to university to learn, and sometimes somewhere along the way, we'll forget about that [laughs]. They both - we assume that they don't necessarily want to learn, they want other things, more instrumental things, they want the high grade this and that, and the students, they take their bit of their cue from us, and they get pulled into that way of thinking as well. And actually just coming back to first principles and saying to students, I assume you're here to learn, I assume there are things you're interested in, why don't you go and explore that? It's just really freeing and enables them to learn and really learn well, and learn the things that they want to learn.

Kristina Hoeppner 08:40

Your students also talk to each other on their portfolios, right? They give each other comments. 

Cathy Elliott 08:47

Yes, they did. 

Kristina Hoeppner 08:48

How have they reacted to that? 

Cathy Elliott 08:50

Well, I get it. That was super interesting, what I did to start with, so I really wanted to culture of feedback. I assumed at the beginning that they would want us feedback from me because I'm the professor rights, I have that secret knowledge. So what I said to them was, 'I will give you feedback on one portfolio post, and then that's your entitlement as somebody in the class. But then if you want more feedback for me, if you give feedback to somebody else, then I will owe you another piece of feedback.' So I thought, this is a nice way of getting them to start giving feedback to each other. I don't know whether I even really needed to do that because very few students took me up on it. So lots of students took the one piece of feedback from me, and lots of students gave each other feedback, but not many people came back to me for the second piece of feedback. And I did sort of stress to them that 'actually the feedback you get from each other might be more valuable because you're quite good at explaining to each other, you know where you're at, you're all at kind of the same level. Whereas I might be explaining things on a like complicated level. I might not be so good at getting into what exactly your problem is. You'll understand each other.' 

And it was true, they did, and they were really very generous at sharing feedback with each other. Friendships developed around giving each other feedback. They developed courage to talk about things that they thought could be improved. They realised that it's a gift if somebody gives you feedback, and so they stopped feeling shy or as if it would be, you know, insulting to say something wasn't perfect on somebody else's work and really engaged with it, which was very nice to see. 

In the past, when I've asked students to do sort of peer marking and that kind of thing on each other's essays, the feedback they've given each other hasn't been of such a high quality. So they might have, you know, focused a bit on things like spelling or grammar or sort of the more surface things that are perhaps less useful. Whereas for this because I think it was a bit more creative and because they were all pursuing what they were passionate about, it was their passion project, they really got into the actual assessment criteria, what really mattered for them in terms of giving feedback, making sure that the portfolio posts were written for an audience, were understandable, were attractive, were drawing on the module readings, were theoretical, and so on. The standard of feedback they've given each other, either identical to what I would have said, or it's better, actually, they've come up with things that I didn't think of. So it's been really impressive. I'm really, really happy [laughs].

Kristina Hoeppner 11:05

Did you give them some criteria for giving feedback to each other? 

Cathy Elliott 11:08

Yeah.

Kristina Hoeppner 11:08

Because it sounds like you've kind of stumbled on the secret sauce of how to make it possible for students to give each other feedback well.

Cathy Elliott 11:16

[Laughs] well, I don't know. Next year, it may not go as well, who knows maybe they were just a freakishly sort of amazing bunch of students. But yeah, did give criteria, and I think that did help. And a lot of them broke the feedback down by criteria. If they wanted feedback from me, they had to tell me which criteria they wanted feedback on. I think that was just a really good way of making sure they were reading those criteria when they were focusing on them. They also knew at the end of term, they would have to fill in a form telling me how they done according to each of the criteria. So they knew they had to focus on those criteria, and it was important to understand the criteria. So a lot of work done around criteria, and they were allowed to invent their own if they wanted, but in the end, not many students did. They were quite happy with the criteria I provided.

Kristina Hoeppner 11:57

So it also sounds like a bit of a constant reflection cycle. So not just reflecting on what they have learned, but also where they think they might want to learn more or dig a little bit deeper in order to understand more where they might be lacking or where they've also done really well.

Cathy Elliott 12:13

Yeah, that's exactly right. That was what I wanted. I wanted this to be a kind of constant process of learning. A kind of revise and resubmit model, You're sending something off, that comes back, you've got some suggestions you're trying to do better. And in fact, one of the criteria I asked them to assess themselves on was evidence of development and learning across the term to show that they were finding their voice that they were developing and learning new things, building on wherever they were to start with. One student actually wrote, so she's quite a quiet student - she wrote in her questionnaire, "I really feel that I've found my voice and found my confidence over the course of the module," which is lovely.

Kristina Hoeppner 12:47

Cathy, you use a lot of formative assessment, as you've just explained, with giving feedback by yourself but also by the peers. But also, you use the portfolio for summative assessment at the end. Can you please tell us how you tie those two things together? What the students then submit for their final assessment?

Cathy Elliott 13:08

Absolutely, yeah. The idea was that they had to write in their portfolio every week, just one or two things, or maybe a picture or whatever it is they wanted to do. There would be this constant process of feedback from me, from each other over the course of the term. They could go back and polish anything up at any point they wanted. So if they'd had feedback, they could change the work that they'd had feedback on, that was all fine. Then they had the winter break, to do some work to do some reflection after the end of term. And after they'd had a couple of weeks to reflect on it, they submitted to me a questionnaire. This questionnaire asked them a little bit about their journey through the module, how much work they'd done, what feedback they've given, how they'd responded to feedback, those sorts of things. Then they had to fill in a table with the assessment criteria and explain how their portfolio had met the assessment criteria in the end, and then they had to give me 2,500 words from their portfolio to have a look at mark. They gave me a secret URL to the bits that they wanted me to read. Yeah, that was the bit that was the summative assessment. And some students said, ', I wish you would read all of it though because although I've given you 2,500 words, I've written so much more than that.' They've they're such interesting pieces of work, I have actually got to sort of read the rest of it. And at least I've been able to give them some feedback on that and talk to them about it, even if I haven't, you know, assessed that everything,

Kristina Hoeppner14:21

How many students did you have in your class? 

Cathy Elliott 14:22

38

Kristina Hoeppner 14:24

Because that I think, is also something really important to know because we know that portfolios can be used extremely well in small groups and also medium groups like yours. And then of course, instructors that deal with 200, 300, 400 students have different requirements there and so are having those restrictions and I think also making them really transparent of what you're looking at and why you're looking at that and that they still get that feedback from their peers is important to share.

Cathy Elliott 14:53

Yeah, absolutely. In the past, I've taught those classes of sorts of 200 students, often with teaching assistants and some other people teaching some of the groups and that kind of thing. And I was thinking about how I might implement portfolio assessment. So I've always tried to do continuous assessments, but of in Word documents, rather than a nice sort of fancy platform like this. I was thinking it would be absolutely possible to have a portfolio, to have students working on it progressively, to have workshop sessions where, you know, you could circulate and see what people were doing. But you would have to really accentuate the peer feedback because as a tutor, you couldn't be reading every single portfolio and giving formative feedback all the time. It's such a big class.

Kristina Hoeppner 15:32

Cathy, for your portfolio work in the end, you ask for a 2,500 word essay. Am I assuming correctly, that that is all written or did some of your students also submit it in a multimedia format?

Cathy Elliott 15:45

Yes, they did. So it's not necessarily a 2,500 word essay, they just have to give me 2,500 words. So some students have got sort of 300 or 400 or 500 word posts, others have written a longer post or two. It could be anything they wanted. All I said was, 'if you've got spoken word in there, if you've got captions in there, then those words count as well, it all counts.' 

So yeah, so lots of students have put photographs. Lots of students have put their own artwork, whether those are sketches or whether those are really, you know, sophisticated, beautiful pieces of art that they've been working on all term. Some students have done poems. I believe I'm still marking and I haven't seen it yet, but I believe there's at least one podcast, and some students have put little YouTube videos on there as well. So all sorts of different sorts of things.

Kristina Hoeppner 16:33

Now that you're getting towards the end of your module, what would you keep the same next time, and what would you change? Have you already made some notes to yourself for that?

Cathy Elliott 16:36

Yeah, I was just talking today about it. And I wouldn't change very much. I think has gone really well. If I could think of a way of getting them to share their work a bit earlier with each other that would definitely improve things. I think some students were saying, 'Oh, it's a shame, I haven't seen many other people's portfolios.' And then you know, so we were asking them, 'Well, have you shared yours [laughs]? Because that might be - you could be the person who takes the first step.' 

Next year, I'll of course, have some examples from this year to show them which will give them a sense of what the possibilities are and also not to be too intimidated, you know, there's different ways of doing it,. I will definitely keep that sort of continuous assessment. My module is a module about nature; nature is changing all the time. And so I don't want students to in the middle of January in London, start writing when they forgotten what it was like in the early days of October when it was sunny [laughs], right? We were all outside. That process of noticing, photographing, documenting, that is something I definitely definitely want to keep that was lovely. Definitely want to keep the ongoing feedback and that culture. I definitely want to keep the scrapbook feel of it and the fact that it's kind of private, and they can work on it privately and do it on their own, keep reflecting on their learning until they're ready to share it.

Some of my students were saying they would have liked to have worked together on a group portfolio. So I was talking with Aurélie again, learning technologist today about like, how we would go about doing that, and we figured out how we would do it, which is nice. So next year, I might say to them, if you want to work together on something and have a sort of collaborative project, that's also fine, so long as you can talk at the end about who did what and reflect on that, that's fine. Yeah. So I don't know come back to me in a year, and I'll tell you what I've changed.

Kristina Hoeppner 18:25

Yes, let's put it in the calendar and have another chat...

Cathy Elliott 18:29

Absolutely [laughs].

Kristina Hoeppner 18:30

... in a year's time. Group portfolios, I think are a wonderful possibility for people to work with each other. SRUC - Scotland's Royal College, Barony Campus, the team around Ali Hastie, has been doing wonderful work over the years. So he might be a good person to talk to, since they use the same technology and have been using group portfolios also for students during project management and working together in a course or in a module. He's definitely a person to have a chat with... 

Cathy Elliott 19:02

I'd love to see that sounds fascinating.

Kristina Hoeppner 19:03

... and find out more how they are managing it. Cathy, you already mentioned that your students would like to be able to do portfolios, but is there anything else that you would love to be able to do with portfolios, but you don't yet know fully how to do that, be that either technical nature or have a pedagogical nature?

Cathy Elliott 19:24

Yeah. So if I tell you, will you magically make it happen for me [laughs]?

Kristina Hoeppner 19:27

We can certainly try [laughs]. 

Cathy Elliott 19:29

Okay. One thing I also do quite a lot of with my students is social annotation. But what I would really love is, if I could on portfolio, just highlight a little bit of text and just comment on that. So I could pull a comment out, rather than getting to the end and then writing a comment at the bottom when you have to sort of write more because you have to explain which bit of the portfolio it refers to and so on. So that would be amazing, and particularly if then people could reply to the comment. That would be super nice. So I would love that.

I heard somebody on a previous edition of the podcast talking about being able to map work on the portfolio to assessment criteria, that sort of thing. That would be super nice. And particularly if you could sort of highlight a bit of text, write a comment on it, and then say how they're linked that to the criteria so that students could see how that comment links to the assessment criteria, but then also see all of those comments reflecting a particular criterion together. Now, that would be really nice.

Kristina Hoeppner 20:24

Well, I do actually have two answers for you. 

Cathy Elliott 20:27

Yes!

Kristina Hoeppner 20:27

Let's start with the last one for mapping the assessment criteria. One possibility there would be to use the SmartEvidence functionality that we do have in Mahara because that allows you to work with competencies or something that you can list in a matrix, and then you'll put the your descriptions of it in there. And then your students can map currently entire portfolio pages toward that particular criterion, and they provide an annotation for it so that they see why they mapped it. Then you can give feedback directly on that annotation, they can also give more feedback, and you can also use it for assessment purposes. So that's one possibility. We are looking into how we can extend that also to artefacts, so to particular a file or journal entry because of course, mapping it against an entire page, especially when you have a lot of evidence can be quite overwhelming.

Coming back to your earlier comment that you'd like to be able to make comments directly the highlight text and then engage with students in a conversation for that, I would like to connect you with the folks at University of Bremen because they do have a prototype for that that they I think probably already even trialling on their instance of Mahara. I'm not so sure about the constant feedback giving then in there. I just know that you are able to highlight text, then a window pops up, and you can put a comment in.

Cathy Elliott 22:04

Oh, amazing, that would be so great. Thank you.

Kristina Hoeppner 22:07

I'll send an email to the two of you, so that you can connect Alex Del Ponte. He's the developer in the department that Dr Karsten Wolff works in education. And they are creating some really, really nice functionalities that we are exploring for inclusion in Mahara. And they showed me one of those prototypes a while ago. So I'm also looking forward to seeing what it looks like right now. 

Cathy Elliott 22:32

Oh, wow, that would be terrific. 

Kristina Hoeppner 22:34

Cathy, we are coming to the end of our session already. So I'd like to ask you three questions in our quick answer round. The first one is which words do you use to describe portfolio work to colleagues or to your students?

Cathy Elliott 22:51

So I've got three alliterative ones: creative, collaborative, and continuous. So ongoing, steady work.

Kristina Hoeppner 22:58

What tip do we have for learning designers or instructors like you who create portfolio activities?

Cathy Elliott 23:05

For instructors: definitely work with your learning technologists. We have such a wealth of experience in universities, we don't always draw on it. And it has been really eye opening. It's been brilliant. Also, for instructors, I would say just make sure that the assessment criteria are really clear, that students can understand them, they're flexible, and they give students a little bit of space to do what they want to pursue their passion and their interests and be a bit flexible and creative.

Kristina Hoeppner 23:29

On the flip side, what is your tip for portfolio authors, for our learners and students?

Cathy Elliott 23:34

So I think work continuously. The best portfolios I'm seeing right now from students that I'm marking are the ones where they haven't left it to the last minute, they've kind of kept working on it every week, and they've taken feedback and they've really developed on that. Be creative, you know, don't be scared. It doesn't have to be perfect, but it's nicer if it's just yours. Give feedback generously and receive it humbly and open mindedly, and remember why you came to university. Relax, enjoy it, learn.

Kristina Hoeppner 24:03

That it is very encouraging and motivating for students. Thank you so much for your time, Cathy. It was wonderful to chat with you. What I love is that you have embraced portfolios so very quickly, have taken advantage of the help available to you through your instructional design team or learning design team, have listened to them, and also have created activities that are extremely transparent to your students so they know why they are doing things. They know what they can expect, and also then oftentimes, maybe not at the start but at least halfway through or towards the end see why it was such a beneficial activity to go through. So I do hope that other instructors in your department and also in the wider university will try something similar so that your students can continue with that work. 

Cathy Elliott 24:58

Yeah, hope so, too. 

Kristina Hoeppner 25:00

Now over to our listeners. What do you want to try in your own portfolio practice? This was 'Create. Share. Engage.' with Cathy Elliott today. Head to our website podcast.mahara.org where you can find resources and the transcript for this episode. This podcast is produced by Catalyst IT, and I'm your host Kristina Hoeppner, Project Lead and Product Manager of the portfolio platform Mahara. Our next episode will air in two weeks. I hope you'll listen again and tell a colleague about our podcast so they can subscribe. Until then, create, share, and engage.