Kristina Hoeppner: 0:05

Welcome to'Create. Share. Engage.' This is the podcast about portfolios for learning and more for educators, learning designers, and managers keen on integrating portfolios with their education and professional development practices. 'Create. Share. Engage.' is brought to you by the Mahara team at Catalyst IT. My name is Kristina Hoeppner. I welcome back Professor Dr Cathy Elliot from University College London, which we'll abbreviate as UCL. It's Cathy's third stint on the podcast, and to my shame, I have to say, that I missed to invite her last year. Time flew by so quickly, and it's already now mid 2026, so that it was high time that I have her back on the podcast. Here we are again, and I look forward to our conversation, which picks up where Cathy had left off last time - how portfolios and ungrading, or what she calls relational marking, go together. Welcome back, Cathy. So good to see you again.

Cathy Elliott: 1:11

Thank you so much for having me.

Kristina Hoeppner: 1:13

For those listeners who have not yet heard your previous two interviews, what is your current role at UCL?

Cathy Elliott: 1:20

I teach in the Political Science department, and I am Vice Dean (Education) for the Faculty of Social and Historical Sciences. So that means I am running education from an academic perspective in our eight departments with 7,000 students and 1,000 staff. But what I'm going to talk about today is more the teaching that I do in the Political Science department, which is on my module, which is a 'Politics in nature' module.

Kristina Hoeppner: 1:43

Yeah. And so you are teaching and also in a leadership position.

Cathy Elliott: 1:47

Exactly, yes.

Kristina Hoeppner: 1:49

In 2024, you told me all about how Geography was implementing portfolios as well. So you can dip your toes also into other social sciences besides Political Science.

Cathy Elliott: 2:02

Yes, it's a huge privilege to be able to talk to other people about their teaching practice and maybe help them do the things that they want to do, but they think they're not allowed. There's a lot of feeling in UK universities that we're not allowed to do this and we're not allowed to do that. And I'm here to tell you that you probably are allowed, as a Vice Dean[laughs].

Kristina Hoeppner: 2:19

That's exactly what we are going to talk about today because in your last interview, you briefly touched on ungrading. You said, and I quote, "The only other thing that I thought I might mention is I've just written a thing about anonymous marking, which has been on my mind." And then you briefly explained what ungrading is and why you were interested in it, and of course we'll get to all of that in a second, and that you were in the process of publishing an article, and I said I'd love to read it. You did send me the article, and it is really, really great to read, also, your style of writing is very engaging for an academic article, I find. And so you wrote about 'Against anonymity - Relational marking and awarding gaps'. That was an article published last year in the Journal of Learning Development in Higher Education. Can you please unpack the terms ungrading, negotiated marking, and also relational marking, which you introduce in your article, before we actually look into well, how does the portfolio fit into all of that?

Cathy Elliott: 3:23

Lovely, yeah. So ungrading is a kind of movement. It's been around a very long time. The idea of ungrading is that the practice of giving a grade or a numerical mark to students to communicate to them about the quality of their work, and also for quality assurance purposes and for award purposes is really counterproductive if what we're interested in is students' ability to learn and improve and grow and develop. What people who are ungrading do is often within systems like ours in the UK where you cannot avoid giving a mark because it's required by all sorts of systems, is we are just trying to cast a kind of a glance at, we're trying to cock a snook, right? We're trying to think in a different way about grading, and just trying to communicate both to students and to colleagues that actually there are lots of things that are wrong with the practice of grading. It is notoriously not very reliable. For example, two different people marking blind will often give totally different marks, even if they're given rubrics. It often encourages exactly the sorts of behavior which are not good for learning. So it encourages things like minimaxing, where students will put the minimum effort possible into just focusing on question spotting, that kind of thing, or they will be very very competitive, which actually is not good for learning, or they will focus so much on the grade they will become obsessed with minute details, and every conversation they have with you will be about how do I get high grade, how do I get high mark, which actually squeezes out all that beautiful curiosity and excitement, which is the thing that really leads to quality work and really good learning. It also makes students really secretive because whether or not they're graded on a curve, they think they're going to be compared with each other, and in fact, they probably are because we, as markers, we all know that we find a level from the work that we're reading, and so students then become very, very secretive about their work. They don't want other people to know what they're doing in case somebody else steals their ideas or does something better with the idea that they've had and that kind of thing. Grading promotes all these sorts of behaviors that we think of as bad. It's also horrible for teachers because then students will all be writing identical work because they are afraid of taking a risk, of being curious, of pushing outside the boundaries, and so then you're sitting reading sort of 60 identical essays that are all all right and that are all exactly what you said they should do, and nobody will ever put forward an argument that you might not agree with as a teacher because they wouldn't dare. Sometimes this goes to the extreme. Sometimes students will not put forward arguments that they even think I might disagree with, even though they don't know what I think, right? And so you get to see some of the assumptions that students are making about what you might or might not think repeated in essays that they're handing in.

Kristina Hoeppner: 6:04

And then also really just writing for the grade. In your article. I think one of the student's mentioned that they were always asking you, "So what does she want? What does she want to hear?"

Cathy Elliott: 6:14

Yes, yes. So they're just writing for you and for the grade. Like, what does she want for the good grade, and how do we do that? So the ungrading movement is just trying to inject a bit of critical thinking and a bit of critical awareness into this grading process, and sort of ask people to consider whether we might not be able to invent a world where we're much more focused on learning and feedback rather than grades and testing. Then we move to negotiated marking. We all exist within these power systems; UCL is no different from anywhere else. So, in order for my students to graduate with a degree, they need to have a mark from my module, and that mark needs to be numerical form, and it will count towards the degree award that they get. There's nothing I can do. Well, there may be things I can do about that, but in the context of the teaching term and the students I've got in front of me, there's nothing I can do about that. I have to give them a mark, or they have to have a mark in the system. Lots of people before me have found ways of injecting a bit of ungrading into such systems. The purpose behind it is mainly to get students thinking about what would a world without grading be like, and would it be better? So it's a way of kind of promoting assessment literacy. Negotiated marking is where students will, with scaffolding and support and using a rubric, so using assessment criteria, come to a judgment themselves about what grade they might be entitled to, and they will give themselves a grade or a mark, and then they will come to the teacher's office. There will be a kind of case conference, and there will be a process of negotiation, and the teacher will negotiate their mark with them. So they make the opening bid, and they are able to discuss why they think they should get the mark that they get. So that's negotiated marking. That's basically what I do. So I am doing a form of negotiated marking. But what I wanted to call relational marking is to try and put the focus on why the mark that we get in the end is the outcome of a relationship that has developed between us from across the term. The reason that I wanted to use this phrase is I really wanted to foreground the trust, the mutual trust, the relationship, the working together, the process, the fact that it's long term as a way of thinking about the way that we negotiate grades. This is specifically in opposition to anonymous marking, which is obviously very popular. In the UK almost all institutions, you do not know whose work you're marking when you mark the work. This is in opposition to that, and it's about marking the student's whole learning journey on the module rather than focusing on the anonymous marking of an output, which might be an essay or something like that at the end.

Kristina Hoeppner: 8:57

That's a lot, and I hope everybody who's listening to the podcast episode already realises where portfolios can fit in super, super easily into that entire concept...

Cathy Elliott: 9:07

Yes, yes.

Kristina Hoeppner: 9:08

... and where also the anonymous marking would not make any sense because we want students to bring themselves in, to bring their personalities in when they are creating the portfolios, and also having multimodal content and images, videos of themselves, audio excerpts, any of those things will, of course, identify the students.

Cathy Elliott: 9:29

Absolutely, the fact that this kind of work cannot be anonymous is one of the things that really worries people. At UCL, we have a rule that marking should be anonymous by default, and there was a very big student campaign by the NUS, the National Union of Students, about 20 years ago now. And at that time, the leader of the student union, he is now a very prominent Labour politician. He's the Defence Secretary. This is a movement really with a history, and this movement was called "Mark my words, not my name". And the idea where we have awarding gaps that's because teachers might be biased against individual students, and if you didn't know whose work you were marking, then the hypothesis was this would get rid of awarding gaps because you wouldn't be able to be racist when you were marking because you wouldn't know whose work you were marking. Awarding gaps are much more complicated than that, and in fact, anonymous marking - and I do have a look at the literature on this in the article - anonymous marking hasn't worked to get rid of awarding gaps for reasons that we might be able to anticipate when we really think about what the causes of awarding gaps are. They're often not that an individual teacher doesn't like an individual student. It's much more complicated than that.

Kristina Hoeppner: 10:34

Mhh and that ethnicities and other demographic factors fall into that, and so, anonymous marking, while in certain circumstances might have been praised in saying yes, more women have been accepted...

Cathy Elliott: 10:48

Yes, you're talking about the blind auditions, aren't you?

Kristina Hoeppner: 10:51

Yeah.

Cathy Elliott: 10:52

There's this one very famous article about blind auditions in orchestras. It seemed to show using statistical data, this is a difficult thing to show because there aren't that many auditions for orchestras [laughs] and not that many records of them. But it seemed to show that if you had blind auditions where the auditioners, the judges, didn't know who was playing because it was behind a screen, you would get more women than you would get otherwise, right, and orchestras became more gender equal. But in fact, people have gone back and looked at the statistical methods that the article used, which is now quite a long time ago, and said, you know, if you'd use the statistical methods that we use now, if you do slightly more up to date methods, you probably wouldn't have found that. Actually, the standard error in this article is very big, and so people don't even think that article is particularly valid. But then musicians have come in later and said, 'Look, what you're doing when you focus on a blind audition is you're really focusing on the output on the day, the sound on the day,' which is exactly what we do when we're marking exams and essays, and all we do is look at the piece of work in front of us. This is particularly true for exams, of course, because that's the more analogous situation to an audition. You've got how did you do on that day, given all the other contextual factors that go on in your life, you know you might have had a bad morning, you might have had a terrible journey into the exam, you might have a headache that day, you might be feeling particularly amazing that day. Were you lucky on that occasion to produce the best sound? In some ways, that focus on the sound a is very subjective in the same way that our response to essay and exam answers are. Very subjective, different people will have a different view about what a great sound is like. Every professional musician that's in a position to audition for this orchestra is already going to be fantastic, and the way that we judge them on the day may be you know much more subjective, and it may be much more question of luck than we think. But also being in an orchestra isn't just about producing an amazing sound. It's about other things. This is where it comes back to the learning journey and the portfolio. It's about being a great colleague. It's about charisma. It's about generosity. It's about teamwork. It's about being able to be creative and take direction and you know and once you start assessing for those things in order to get an orchestra which has a range of different people in it, then you get an orchestra with a range of different people in it, with different skills, with different ethnicities, with different backgrounds, different genders. Once you stop focusing on just the sound and you start thinking about all these different qualities you want to bring in, that's how you tackle the elitism and exclusivity of the orchestra, but it also means you can no longer do the blind audition because in order to assess for those things, you've got to do other stuff. This is why I sort of wrote a bit at length about the blind auditions in the article because I think it's really analogous to all the work we're doing when we're assessing. Do we want to focus just on the final output of a single essay that student has written, or do we want to think about everything that that student has brought to the module, which might be very different from student to student because everybody's different. We all bring different skills and different weaknesses. How have the students showed up with those and then also worked on those as part of the module? Can we assess that? And that's what I'm doing really with the relational marking. Looking at some of the research on reasons for awarding gaps, my hunch is that this is also going to tackle awarding gaps.

Kristina Hoeppner: 14:10

That's where we are now back to the relationships. What does that relational marking look like for you in your classes? Because you had said you're negotiating the grades with the students, it's a much closer relationship between students and lecturers, there's feedback involved. If you walked us through your module, what would it look like for a typical assignment, or maybe actually even groups of assignments because you're not looking at one assignment on its own, but you're looking also at the broader context of the entire module?

Cathy Elliott: 14:43

Students obviously do a portfolio. So I'm asking them to work continuously on developing a portfolio. They're writing every week, I give them a series of structured prompts to write to, and I'm also really clear right from the get go what the assessment criteria are. Those are on Moodle, I talk about them in the very first class, and we discuss together like this is what good work might look like in our discipline. I give them those assessment criteria, and I'm like, 'your job is to think about what you want to get better at, and to use those assessment criteria to get better at that. Or if you want to be assessed on something else, the idea of relational marking is that these things are up for discussion; they're up for negotiation. We can have that conversation if there's something else you want to be assessed on,' and we do do that. Usually, if a student wants to be assessed on something else that isn't in the assessment criteria, that is perfectly fine, but it has to be a conversation. I don't abandon the students to decide their own mark. It's a relationship where we're working together to try and figure out what would it mean to do good work? What would it mean for you, and then what would it mean for me and the discipline? And how can we come together to agree on that in ways that help you with all the other work you're doing in all your other modules? They're aware of those assessment criteria. They can negotiate those if they want. They work throughout the term. One of the assessment criteria, and in some ways, it's the most important one, is the learning and development. You have to show evidence of learning and development across the term. So they are thinking, how can I evidence the fact that I am getting better at something, right, that my work is improving? As they're working through, they are getting feedback from me on the work that they're doing. And whenever I give feedback to a student, and this is honestly my top tip, I say to them, "What assessment criteria are we talking about? Where is that in the marking criteria?" So, if a student comes to me and says, "I want feedback on whether I should be writing in the first person or the third person" I will say, "What does it say in the marking criteria about that?" Sometimes it takes them a moment to figure out what it is that I'm talking about, but in fact, there is an assessment criterion - ability to write appropriately for a particular audience; you get to say who that audience is. So, for some audiences, you might very well write in the third person. That is fine, so long as you're telling me, 'I am writing a kind of objectivist scientific journal article where I believe you would write in the third person, here are some examples of people who've done that.' Fine. Otherwise, which audience are you writing for? You're writing a personal diary? Are you writing a learning log? Are you writing it for me? Are you writing a letter to Kew Gardens, which is an example we had in an earlier podcast? How would you express yourself to that audience? Being able to write for an audience is such a useful skill. So I'm just cluing them in, but I'm not saying, "I want you to write in the first person because that's what I think and I'm the teacher and I'm powerful. I am asking you why you're writing this because you must be writing it for a purpose. You must have a reason for it. Therefore, you will have an audience in mind, and that will help you know the style of writing you want to use, including like which personal pronouns you might want to use." So throughout the term, they're getting feedback. They're getting feedback according to the assessment criteria, which are clearly spelled out, negotiable. And so, some people don't like me talking about clear assessment criteria because they say it's not one of those things that's transparent. It's one of those things you have to engage with analytically, and is really complicated and has to be constantly kind of thought about and negotiated in relation to the particular work you're doing, that's all fine. So we're having those conversations. They're also giving feedback to each other, which, apart from anything else, really helps them understand those assessment criteria because if they give feedback to somebody else and it just says, 'this was very good, but you made a spelling mistake,' that isn't assessing the assessment criteria, and so I don't count that as feedback, and they know that. So they have to think about the assessment criteria in order to give feedback to each other, which is often as good or better than the feedback I could give because they're working at the same level, so they're able to explain things to each other that to me are second nature. And then we get to the end of term, and they have a couple of weeks after the end of term to fill in a questionnaire. Kristina, I wish I could incorporate this questionnaire more easily into Mahara. At the moment, this is now a separate Word document because I can't. But let me know if you have thoughts.

Kristina Hoeppner: 18:46

I do[laughs].

Cathy Elliott: 18:47

Excellent. I thought you might. They have this questionnaire, which is I'm going to say it's brilliant because I stole it. So with due acknowledgement, this is in Susan Blum's book, 'Ungrading', and it's part of an exercise that she uses with her class. She uses negotiated marking, and I have more or less lifted her questionnaire, but it has some tweaks to make it more relevant to my module. And in order to model good practice to the students, of course, I have full acknowledgement and bibliography entry, so that they know that we do borrow each other's work, we do use each other's work, but we must always acknowledge it. And they have to answer a bunch of questions, and they answer questions around the work that they did. How hard did they work across the module? Did they do the reading? Did they turn up to class? Did they help somebody else? Did they receive feedback? Did they ask for feedback? Did they come to office hours? Did they really engage with the Moodle activities? So I have a flipped classrooms. There's lots of videos and things. Did they watch them? Did they watch them at the proper speed and not speed it up [laughs]? I know. Honestly, when I've spent hours and hours making a video, and a student has watched it on double speed, my heart breaks a bit. How hard did they work? That is often around how much effort did they put in? And then some questions around where they found value in the module, what they enjoyed reading, what they would say to the authors, and so on. And then some questions around how good actually, did they think their work was? This is something I'm going to tweak for this year. But for this year, it's on the current assessment criteria. They have to explain for each assessment criterion what they thought they did well and where they still have room to improve. Only after they've been through that full process of filling out that questionnaire can they give themselves a numerical mark. I try and de-dramatise that so different people doing ungrading have different approaches to the numerical mark. My approach has always been, and this is where my module is a bit controversial, I don't really care what numerical mark you give yourself. You can give yourself the Prince symbol if you want, right [laughs]? I don't mind. All I care about is that you've turned up the class, and you've engaged with it, and you've really thought about it, and that you're able to explain why you think you should get the mark that you're getting. By and large, I don't negotiate very much unless it's something egregious. So if a student hasn't tried hard and has given themselves a very high mark, I take a particularly dim view of that because that is broken trust to my mind. But if students have been showing up, working hard, trying really hard, and they think that they deserve a high mark, usually, I will honour that. There are students that don't give themselves a high mark because they're just like,'actually, I took an easy ride this term, or I had other things going on, or I couldn't, and I know that, and I know that Cathy knows me, so she's going to call me out if I'm not honest about that. So I'm going to give myself a slightly lower mark, but I know I'm doing so safe in the knowledge that that will be my mark. You know, I'm not going to fail, and nothing bad's gonna happen. I've been open and honest with myself about my level of engagement with this module, or I've been open and honest with myself about the fact that I've struggled. I don't find this stuff easy, and I'm gonna challenge myself to do better by giving myself a slightly lower mark.' Other people would focus more on numerical mark. I focus much more on the feedback myself.

Kristina Hoeppner: 21:38

It comes down again to the trust in the relationship, and you knowing the students, and the students knowing what your expectations are, having those clear guidelines from the start, scaffolding the activities, and you also giving them the trust that the mark that they give themselves is being honoured and carefully looked at and not just be a token, and then you make your own decision, completely ignoring what they had written.

Cathy Elliott: 22:04

Exactly, yeah.

Kristina Hoeppner: 22:06

Really, which perfectly fits portfolios because there, the portfolio authors are very vulnerable in sharing about their learning, sharing about their failures and also successes, reflecting on what they have done, and then inviting you in as the feedback giver or other students as well to take a look at all of their work. When you'd earlier talked about the grading and the marks and working towards the grade and making sure that you answer the questions the right way, whichever that right way is considered, begs the question for essays in particular, well, if I give it to AI, to a large language model, then I get kind of in a way the ideal answer that is statistically correct because that's what the LLM does, but it does not have that personality included. Whereas I feel like with your approach, you're circumventing all of that. Sure, students might still be using AI for certain things, but have you already had conversations with them about it, or had them talk through why they are using that? I think to remember that we did talk about that. You expecting them to let you know why they've used certain tools, why they are going down a particular route, which then again exemplifies the trust you have in them that they are producing work that they have done, or let you know why they have done something a particular way.

Cathy Elliott: 23:29

Yeah, I have changed my approach to AI a lot, as AI has changed. Yeah, for the first two or three years I was doing this module, my approach was to say to students, "I don't use AI. I don't like it. I'm an environmental researcher. I understand that it's using a lot of electricity and water. I'm not going to use it. I'm not going to teach you how to use it. You can learn about that somewhere else. I suggest you don't use it." For quite a long time, that worked really well. The students didn't use it. They were also a bit scared of it, and it was kind of notoriously very bad. But it's just got so much better. You can say to AI now, please write a reflective post for me. You can give it some examples of your writing and say, please write it in the same style I would write it. Please write it in the style of a third year student. So you can actually outsource a portfolio to AI. I don't think that makes an ungraded module more vulnerable than any other module. You know, if you look at work and it looks like good work, it looks like good work [laughs]. But I was starting to realise that students were doing things like... I have a little questionnaire at the start of term where I just ask them for some information about themselves so that I feel like I know them a bit, and they were using AI to do that. My suspicion is they were using it to translate from a language they felt more comfortable in into English. But certainly, I used to get these questionnaire answers that were a bit messy and a bit chaotic, and they weren't in very good English always, and you know because they were doing it quickly, and they hadn't proofread it, and that was like nice. It felt friendly and it felt good. And then suddenly I was getting these polished answers, which were all seemed also identical and seemed inauthentic. I mean, I don't know they used AI. Well, I do know because I asked them. But when I looked, I didn't know that was AI. I just sort of had that kind of hunch, and so I asked. Then I also had that kind of hunch midway through the term that when they were sending me formative work, they weren't necessarily handing in AI-generated work because they wanted to do the work. They were interested in the work, but what they were doing, which is quite clever, but really took me by surprise, was they had to fill in a form asking for feedback. They asked AI to look at their formative work and think of things they should ask for feedback about, and then they sent me that form. And I was looking at this, thinking this is unlike any feedback form I've ever had from a student before, and I don't know, it's weird. I would give feedback according to what they'd asked, and then they would take my feedback and give that back to AI to help them make their work better. AI is asking me for feedback, and I'm giving feedback to AI [laughs], which is then using that feedback to make the work better. And I'm just like, oh! So we do a class in an art gallery, and it was in the art gallery class that I sort of said to them, "Is this what's happening? I feel like questionnaires and formative feedback requests and things like this. Something AI-ish is happening here. Tell me what's going on." As I've said to you, you know, I have this trusting relationship with my students. It's very friendly. It's quite close. I get to know them. They often say I'm the only tutor that knows their name and a little bit about them, which is a whole other question, which is sad. And so I have this close relationship with them, and then suddenly, even the students that I know very well, that come into my office hours every week, that I have this humorous relationship with, they're all looking at their feet, they're looking at their watchers, they're shuffling, they won't catch my eye. This is so strange, and I said it as a joke, but it's become the title of a paper that I'm writing now,'You can tell me. I'm not the cops', right laughs]? And then we all laugh. Like I'm not a police officer here. I want to know what you're doing so I can help you. I'm your teacher, I'm not the cops. Then they laugh because it's obviously so ridiculous to think I would be a cop [laughs]. I'm letting them decide their own grade, but suddenly now I'm the police. Everybody starts laughing, and then they stop talking to me. They came out to you know all their fears and anxieties. They know how staff talk about them. They know that staff talk about them in this sort of rude, diminishing way,'All students are cheats. They'll try and get away with things. All they care about is the grade. They're not interested in learning. They don't do the reading. They don't turn up. Then they use AI to generate their work, and it's just all total waste of time.' They know people are talking about them in that way. They live in the world. They don't particularly appreciate that, and I don't blame them. Not all teachers talk about their students in this way, by the way, but I think we've all heard it, and we've all also, I think, been tempted by it, right? So I don't want to be self-righteous about this. I was tempted by it when I saw that work coming back, and I thought this is AI. I thought, oh, you little, and then I had sort of, no, hang on, I'm their teacher. I've got a bigger responsibility here. So they know that this is how they're being talked about, and it makes them afraid as well because academic misconduct charges are very serious thing, and they know that. Then also, they're hearing that AI is this life-changing, world-changing technology. It's overturning everything. You won't be able to get a job if you don't know how to use it. You might not be able to get a job if you can't get past it in an interview. It's going to shape their future. So on the one hand, it's going to shape their future, and they need to know how to use it. On the other hand, if they do use it, they're cheating. This is distressing, right [laughs]? And understandably so, right[laughs]? And so it was quite a moving conversation. It moved me to a place of thinking, ah, we have been doing all the surveillance on our students, trying to catch them out when what we actually need to do is having trusting enough relationship with them that we can talk about AI because then we could have the conversation of why did I give you that questionnaire? It was to get to know you, right? So if I can't get to know you through the questionnaire because it's I'm actually getting to know a robot, what's the point of that? And they were like, "Yes, obviously, Cathy, we can see that, but we didn't know you then." But then also, if I'm giving you feedback on your work, like, is it useful for you to run that through AI? Maybe it is. That has made me really think, actually, what I need to do in future is rather than coming with this is not my job. I need to come in at the start saying I will help you use AI. I'm not going to say I'm an environmental researcher and I don't use it. I don't have that luxury, and neither do you. We live in this world. So I'm going to help you use AI. If you want to use AI, we will work together on figuring out how do we use it critically, how do we use it ethically, how do we think about the sources it's been trained on, how do we think about what sort of work AI might produce and what sort of work only a human can produce. If you want to do creative work, is it going to help or stymie your creativity if you start getting AI involved in it? If you don't want to do creative work, is AI actually a really handy shortcut that can help you do something else as well because you've freed up your time? So I'm really moving on AI from let's discourage students from using it; it's horrible to we need to be teaching critical AI literacy where students are able to use it well, including really crucially able to understand its shortcomings and where it is a problem to use it, which doesn't necessarily mean don't use it, but it means use it in a way that's smart.

Kristina Hoeppner: 29:47

Have your then also seen a difference in their writing after you've had that conversation with them?

Cathy Elliott: 29:53

So it's too early to say. This was all happening last term. So I do ask students if they do use AI to write a reflective piece. And increasingly, more students are doing that. Some of those reflective pieces referred back to that conversation and talked a little bit about their hopes and fears and anxieties around AI and what that might mean for their futures. That was really nice to see and really thoughtful. Maybe AI generates this [laughs]. I mean, it is definitely clear to me that a lot of students that used to have English as a second, third, fourth language, and used to be writing in quite grammatically clunky English are now writing in much more polished English. Is it a translation, and is it still your authentic voice if you've translated it and so on? It's useful, but also if I was writing in French, I'd probably get a French person or a robot to help me, right? I can write reasonable French, but I know it will be full of mistakes. So I'm not against them doing it. But I just really want them to have that understanding of why they're doing it, and I'm not there yet in terms of getting them to really write and think about that. It's so difficult to do these things in a module. What I'd really like to do, and what we're talking about, is embedding this across a whole curriculum because actually this is the work of three years, not a 10-week module. You can't ask module tutors to do it all by themselves. But I do think we're trying as a faculty to do things like embed data literacy modules and AI literacy modules and thinking our first year skills modules about what's a good use and a bad use of AI and so on. Hopefully, you know we'll get better at this as we go along. You'll have to ask me back next year to find out how much the tweaks are making a difference[laughs].

Kristina Hoeppner: 31:17

Oh, interview number four. I'll put something into my calendar. When we look at your relational marking, do you remember what a response from students was when they heard that for the first time? Because of course, they would have been used more to the anonymous marking and the conventional ways.

Cathy Elliott: 31:34

Yeah. So the first year that I ran this module, they were flabbergasted. We had some sessions where we talked about grading and what grading has meant for the students' lives. UCL is kind of a posh university. We get students that are real high achievers. They had these stories of their whole lives have been dominated by grades, and it was all they cared about, and it was all anyone around them cared about, and it was all they kind of validated themselves with. And then they come to UCL, and suddenly everybody's got good grades, and they're in competition with all these other clever people. The anxiety around it, and there were many, many stories in class from students who said, "I used to love learning. I remember a time when I was really interested in things, and my academic work stopped being about learning because it was so focused on the grades that it just became this locus of anxiety and misery and fear and competition and secretiveness. It became really poisonous, and I stopped loving learning. And I no longer remember why I came to study politics at university, even though I was interested in it. I know I was because I came to study it." And a lot of students cried when they were talking about this, either in office hours or even in class. For the first couple of years, you would have tears of students of realising how pernicious grades really are and how much we talk and think about them and how much they matter to people. That was really moving and it really cemented me in the idea that this is the right thing to do. We should be doing this. As time has gone on, because my module is a third year module, that has changed a bit because most of the students come to the class. So the class is now massively oversubscribed. Students from other departments can't do it. The only people that can do it are Politics students. That is not only I think because of the ungrading. We also go to Kew Gardens and we do loads and loads of fun stuff. But I wonder whether the ungrading is part of that. And students arrive and they already know because they've talked to their friends in the year above, and so they're so blasé about it. One student this year wrote on her questionnaire: "We used to talk about this module as the hippie module, where you just write a blog and you give yourself a grade" [laughs]. And in my defence, the next thing she wrote was, "However, this has been the most difficult module I've ever taken, and I have really discovered the value of learning" and this and that, and there was something very trusting about the fact that they told me that. Now I go into the classroom that first week and sort of say, "What do you already know about this module? What are your preconceptions? Because I think they think it's going to be an easy ride, but actually, giving up the responsibility of somebody else telling you how good you are and having to yourself take responsibility for actually doing good work, actually taking part, actually showing up, and then actually being able to look me in the eye and say I deserve this grade because I work for it and I think my work is good and I've looked at these assessment criteria. That's actually much much harder. I say to them, you know, I don't go into my performance appraisal every year with my line manager and say, oh, just give me a number. I have to go and I have to advocate for myself, and we have to negotiate whether I did well in that particular year or not. And this is the same. Students now come in thinking it's going to be an easy ride, and hopefully, most of them discover that it really isn't. That learning is one of the hardest things you can do, and that that is what we are putting the emphasis on in this module. I am tweaking my assessment criteria a little bit this year to include something around epistemic humility because I think I am now, because students think it's an easy rise, I'm getting a small minority of students who don't always approach it in the best of faith. They will do work which is good. All UCL students do good work all the time, so they will hand in work that's good, but they will. That it's amazing because they think I might as well. They think I should advocate for myself. That's how you get ahead in this world, right? It's not stupid. It's not a stupid way to behave at all. And I do say to them, please advocate for yourself. Please tell me why your work is good. But I am starting to get students that are giving themselves very, very high marks with perhaps less of insight into where they could do better in their work. So I'm going to try and push the idea of epistemic humility. Because I mean, I don't know about you, Kristina, but I wouldn't myself give myself... so in the UK system, the highest mark we ever give is 85. I wouldn't give myself an 85, and I'm an academic with you know one million years of experience. I've been doing this a long time. I think I would know how to write a good essay. I wouldn't because I know how hard it is, and I have respect for that work. So it's always an ongoing conversation. Things have changed a lot since the start when students were very cautious about what mark they gave themselves, and most students still are. But there's just this little group that I would quite like to, and they're often the best, they're often amazing students, right? But I would just like them to think a little bit less about how amazing their work is and a little bit more about how it could be even better, and so that's what I'm working on right now [laughs].

Kristina Hoeppner: 36:08

Are you then planning on changing the portfolio as well for what they have to put in, so that there's more explanation rather than focusing on that end result, but really showing throughout the term what they have been doing, which then serves as that argument for them, explaining why they think so highly of themselves, or we they draw the line?

Cathy Elliott: 36:32

Probably yes. I need to find a way, and I haven't quite got there yet, of getting them to demonstrate for me a little bit more clearly how they've learned and developed, where the setbacks have been, where they've stumbled, where they've had an aha moment, where they've really learned something. And that is embedded to a very great degree, right, that the whole portfolio is very reflective, and then the questionnaire at the end is asking them to reflect on their learning journey. So I need to think about how do we make some of the work at the top end a little bit less glib about whether or not there is room for improvement? Incidentally, if these are students that probably are getting top grades in all of their work, and that's why they're not necessarily thinking, "Oh, but it could be better." Because if you're a student and you look at your work and it's got in the UK system above 70 as a first class mark, right? If you've got a first class mark, you think that's good enough. I'm happy with that. I'll just continue on. I was much the same when I was a student. They're not necessarily thinking, but how could it be even better? They're thinking, good job done. So how I push those students at the top end to do even better without incidentally stoking the anxiety, right? I want them to do it out of the kind of the sense of curiosity and excitement and what they could do and what they might aspire to if they want. I think there's something there around humility, and I don't know how you embed that yet. But I am giving it some thought. I think it will partly be to do with the pedagogy. Keep their rough drafts in, so that I can see the improvement from the rough draft and the notes to the polished piece of work. To keep diaries in where they've had a moment of a setback, or a moment where they realised they didn't get something and they had to do a bit more work, or moments when they've struggled, something like that. So I can see side by side how they're improving and how they're getting better. I don't know if people have thoughts. Please write in.

Kristina Hoeppner: 38:14

Yeah, normalising that process of sharing those failures. In a very competitive environment, you may not really want to show your weaknesses so much or any of those earlier drafts because you're being judged by those. Or there's the perception then that you're being judged, even if you, in practice, as their tutor, may not really do that.

Cathy Elliott: 38:37

Yeah, the learning moment is in that, 'Oh, if I work completely differently, this changes the way I think about what learning is.' That's the main thing I want students to get out of the module.

Kristina Hoeppner: 38:45

That's how you can also bring innovation about and creativity because then maybe there are some students in your class that go to their other tutors and say,"Hey, in Cathy's class, we worked like this. Wouldn't that also be something that we can try? Or this really worked for us." If that is a possibility, that others are also brought on board with that idea of how you work. I mean, you have brought Fergus on board with how he uses portfolios in his class, and then also Geography has started using portfolio work and embedding that fully into their programme. So it feels like there's both grassroots, so from the ground up, that students can ask and demand or suggest, and also what you as faculty ask of your tutors of how they might want to change or should be changing in order to have an innovative programme, have contemporary ways of learning and engaging the students, and taking that anxiety away, and other things that inhibit their learning. Cathy, you also mentioned that you give feedback to the students and that the students give each other feedback. How does that feedback then influence the students and their further learning? Do you have evidence of how they have changed their approach to something, or how they have incorporated that feedback from their fellow students and you?

Cathy Elliott: 40:07

Yeah, almost every piece of work I see, you can really see the impact of feedback. You can see that students have taken that feedback really seriously, and they've tried to work on it. Sometimes there's so much feedback that they can't do it all, and then they have to be really clear, 'I worked on these things, and I didn't focus on those things because these things seem more important.' Those moments of decision are also really important. Every so often, and it happens, you know, at least once a term, usually more. There will be a student that just has this sort of total breakthrough, and so this year in particular, there were a few, but there was one in particular who... she was clearly really happy with the work she'd done, and I'd been giving her a lot of praise in class because she'd understood some readings that were really difficult. She'd been able to explain them really clearly to other students. She clearly felt she was doing really well in this class. She wrote this lovely personal piece of work about tomatoes in Italy and about the importance of tomatoes in Italy for identity and memory and family ties and so on. She kept referring to tomatoes as a native Italian plant. Now this is a politics of nature module where we think about native plants and botanical colonisation and where different plants came from. And she came to office hours for her feedback, and I just looked at her and I was almost like, I thought it would make her laugh, but it didn't make her laugh."Are tomatoes a native plant in Italy? Is that true? Did you look into that?" She was like,"Yeah, of course, of course they are," and I was like, "Sure?" There were relatively recent imports into Europe. There were a lot of tears. There was a lot of upset as she realised that she'd got something really fundamental wrong, and that meant that the whole argument she was making just totally unraveled, right, totally disentangled. There were some really important points in there around botanical decolonisation that she just hadn't understood, which all rested on this tomato and the fact that she hadn't really looked into it. It really knocked her confidence. I probably wouldn't have tried to have a joke with her about it if I'd known how much it would knock her confidence. But it goes to show how fragile these students' confidence is. Because if somebody says to me, "I've got something wrong, I'm just like, "Yes, well, that's Tuesday" [laughs], you know. It's like [laughs], "nothing new there. I expect I'll get something wrong again in a couple of hours." When you're an academic, people are not shy, are they, about pointing out to you all the many things you've got wrong and all the many ways in which you could improve in every respect. You get used to it. You get a thick skin. You get good at that. But I think these students that are getting first class marks over and over and over and over again, and are investing so much of their personality and their happiness into the fact that they are a grade A student, their confidence is very fragile, and when they do get something wrong, it's like the end of the world. It's like I've pulled out the foundations of her whole world, and she doesn't know what to do any more. She thought she was good at politics, and now she's not. She's made this mistake. And I said to her, at least it didn't happen at the end of term in the summative work, at least it's happened now. This is an opportunity we can get better. The work she put into figuring this out and then thinking about what did it mean for Europe when all these plants came - potatoes and tomatoes and corn and all these - all these plants came from the New World, tobacco. What changed and what does that mean for who we are now was actually an incredibly thoughtful piece of work which she'd really invested a lot into. That was a pleasure to see. But I think also the ability that she showed to have her confidence really knocked, and then to think, right, well, what do I do from here? How do I build back up? How do I become a person who gets things wrong and is able to gracefully say,"Yeah, I got that wrong. Sorry about that." Maybe it's funny, right [laughs]? Maybe getting something wrong doesn't have to be the end of the world. Maybe it can be said. Oh, oops! Now what? How do we move on once we've got something wrong? And I think that is a skill that we all need in life, and in particular, our young people are going to need more and more the way the world is getting as hard as it is.

Kristina Hoeppner: 43:37

Good that she had that experience in your supportive environment.

Cathy Elliott: 43:42

Yeah, thank goodness this happened in the formative piece of work and not in some summative essay where all your mark depends on it. She came back to me a couple of weeks later. She's like, "I've been thinking a lot about you saying that, and it's so true, and it's so great that we have this formative feedback process in this module." So I think she appreciated it in the end[laughs].

Kristina Hoeppner: 43:57

It's good that they have a tutor like you who takes a different approach, so that they do experience something very different to other subjects, and therefore hopefully also reflect on that experience then beyond your class, beyond that last survey that you give them, in order to then see what they want to take from that going forward. Is somebody else then also doing relational marking at your faculty, so that you also have the possibility to reflect on your practice with somebody, be challenged about your work, but also get support from others who are doing something similar, so that you can compare notes?

Cathy Elliott: 44:37

Yeah, so there's actually quite a lot of people doing various forms of relational marking. I don't want to name names because it is a little bit controversial. Some people are really down on it for all sorts of reasons. I've got colleagues that are doing contract grading, which is where if you complete a certain number of pieces of work, then you get a particular mark. So you can contract for, you know, if you work really hard, then regardless of the output, you will get an A, if you don't work so hard, you'll get B, and so on. So I've got people doing that. I've got another couple of colleagues that are doing some form of relational marking. I've got colleagues that are trying really hard to grade process and learning journey rather than output. So yeah, so I think once people can see that there's somebody doing it, it does sort of spread. And there are people sort of around UCL now, we call ourselves the 'Freedom to Learn Movement'. There are people around UCL who are sort of interested in these ideas and are trying them out in various different ways. Some of them are very involved. I've got colleagues in the more STEM disciplines who have these kind of incredibly elaborate flowcharts, which explain, you know, what you would need to do in order to get this mark or that mark, or what you need to be able to do to qualify for negotiating your own mark versus having it marked by the teacher. That wouldn't be my style, but I can see how that is useful for them. So yes, so these different ways of doing the practice are coming up. Hopefully, there are opportunities where you know we present to each other or we go into each other's faculties and give talks and that kind of thing, and around the country as well. It's the one thing that I get invited to come and talk about more than anything else because people find it quite intriguing.

Kristina Hoeppner: 46:08

If somebody wanted to join your movement, what would you tell them how they could get started or what they could do?

Cathy Elliott: 46:17

For me, it was a very long process of getting to the point where I dared do something quite as radical as I've done in my classroom. I would really encourage people, the first thing you do is have a think to yourself every time you're talking about marks, how different would this conversation be if I wasn't focusing on marks? If I didn't think marks mattered? Would I be having a row with a second marker or an external examiner about whether a piece of work is worth 65 or 63, or would I let that go? When students are doing or not doing the things that we would prefer, is my knee-jerk response to say, 'if you do that, you'll get a bad mark, and if you do that, you'll get a good mark.' Is there a little bit of coerciveness involved in that? How would that conversation be different if we didn't have marks? What sort of conversation would you be having instead? Would you prefer that conversation? Going back to my conversation in the art gallery with the students, where I said, "Don't worry, I'm not the cops," right? Now, people have different relationships with the police, and that's absolutely fine. But I think everybody agrees that teachers and cops are two completely different jobs. So I think the first starting point is to consider over and over and over again, how do I choose to be a teacher and not a cop? How do I choose trust and relationship and care and feedback and support over surveillance and punishment and incentives and all of that discipline stuff, police stuff?

Kristina Hoeppner: 47:41

So be reflective yourself, look at your practice, and then maybe even create a portfolio out of that to gather all the evidence.

Cathy Elliott: 47:50

Absolutely, yeah. Once you've gone through the trying to unlearn the grading, then I think trying to unlearn your reliance on the final output, right? So trying to unlearn the idea that you can judge or test a student's performance on the basis of one piece of work that was handed in, perhaps without any feedback, right at the end, or in an exam, maybe months after the learning took place, and learn your approach to that, and instead try and think about how can learning be happening throughout the module? This is where I think portfolios really come into their own because you're asking students to work continuously, week after week after week and get that feedback so that all the learning and all the feedback takes place before the final moment of decision and then the final moment of decision is totally de-dramatised. We're not worried so much about that because we did the learning already and we know where we are and we know if we did well.

Kristina Hoeppner: 48:39

Yeah, and you can also take all of that into consideration when you look at their final work because you've experienced your students and what they are capable of as well.

Cathy Elliott: 48:50

That's right. Yeah.

Kristina Hoeppner: 48:51

So now, before we get into the quick answer round, and I am still keeping the same three questions for you, so that we get maybe even more of your alliterations, but we'll see shortly.

Cathy Elliott: 49:01

I've got some. I've got some. Yeah.

Kristina Hoeppner: 49:03

Oh, perfect! Absolutely fantastic. Now you mentioned that survey at the end and had said that you can't really do that so easily in Mahara. And yes, that is true. At the moment, I would also use a Moodle questionnaire. But what we are looking into as part of also our roadmap work is to incorporate forms so that you can create forms, text, and other elements. So I very much invite you to send me your form or your survey so that we can take that as an artefact and look at all the others that we are getting in order to figure out what should a first phase of forms look like. What do we need in them? And then allowing us also to gather that data in a more not necessarily quantitative way, but in a way that a tutor like you could look at all the responses from the students on one page, for example, and therefore quickly go through it and also aggregate certain data as well. That is a piece of work that we'll be starting very shortly, and by the time that the podcast episode airs, will have already started.

Cathy Elliott: 50:09

Brilliant! Oh, that would be super. Yeah, of course, I will send that questionnaire over.

Kristina Hoeppner: 50:13

Perfect. Thank you. So now to our last three questions, which words or short phrases do you use this year to describe portfolio work?

Cathy Elliott: 50:23

So, on the basis of using it for relational marking, and I've struggled with that. I honestly, I was racking my brains for an alliteration, but I've got one. Definitely relational, radical. So, I think what we're doing is really looking at quite fundamental change in the way we think and riotous. It's really fun[laughs.

Kristina Hoeppner: 50:40

Oh yeah. Speaking of fun, I think we can't deprive everybody of the other two alliterations. So in 2023, you had creative, collaborative, and continuous. And then in 2024, focus on fun, flexible, and feedback. And so the feedback also still stays in this year.

Cathy Elliott: 50:59

Feedback is still there. All of those are still there. I think, yeah.

Kristina Hoeppner: 51:02

Yeah. Cathy, what tip do you have for learning designers and fellow instructors, tutors, lecturers who create portfolio activities?

Cathy Elliott: 51:11

So I don't know if I've said this one before, but my absolute top tip is be really clear what your assessment criteria are because I think that that is the way that we have conversations with our students about what good work is, what constitutes good work? So be really clear what they are. If they're negotiable, be really clear about what that negotiation process would look like. And every time a student asks you, "What should I do? How should I approach this? What's the answer? What are you looking for?" Remind yourself to come back to those assessment criteria and throw the question back at them. What do the assessment criteria say about that? Because I think that's how those assessment criteria become part of your ongoing conversation with the students about how are we making this work as good as it can be, and what do we think good work means.

Kristina Hoeppner: 51:56

Yeah, and bringing it back to why you're also doing all of that.

Cathy Elliott: 52:00

Exactly. Yeah.

Kristina Hoeppner: 52:02

What advice do you have for portfolio authors?

Cathy Elliott: 52:05

I would say don't be afraid of failure. Failure is how we get better. Failure is how we learn. It's all right to get things wrong. It's not the end of the world. It's all right to be upset about getting things wrong, though. It's all right if it does feel like the end of the world. But actually, the lovely thing about a portfolio is you can show, if appropriate for your learning situation, and certainly if my students are listening to this, you can show where you got something wrong, how you picked yourself up, and how you did better. Let's say you generated an answer using AI and it was crappy, but you have really learned how to do a good answer on the basis of like trying to disentangle that. By all means, put the AI answer in. Put in your reflection on that, then put in the better answer that you did. If you made a mistake and you thought a plant was from one place and it was from another place, tell me. Tell me what you learned not only about the substance, the disciplinary knowledge that you're writing about, but what did you learn about yourself, and what did you learn about learning and how to learn through this process of getting something wrong, feeling embarrassed, and then moving forwards, and hopefully doing better.

Kristina Hoeppner: 53:03

Awesome! So much in there in your article on relational marking, and we didn't even talk so much about the awarding gaps that make up the first part of your article. But I just like to encourage everybody to read it because it is very rich and also eye-opening in a lot of things that in the past I've taken very much for granted, not really thinking too much about grading because yep you got to get a grade. And sure, sometimes there's a quiz involved, at other times it's an essay. But at the end, you still need a grade. Now looking at well, what other ways are there that encourage the learning, that make it possible to engage with the learning also differently in today's world, and therefore really get to do the hard work because of course the portfolio, doing the own thinking, doing the reflecting is not necessarily easy. If your students can learn that in your class and be encouraged and also given the time to do that, I think is very good because they will be taking that into their future. So thank you, Cathy.

Cathy Elliott: 54:09

Thank you so much.

Kristina Hoeppner: 54:11

Now over to our listeners. As you think about your own portfolio work, what resonated most with you today? Share your thoughts on LinkedIn, Bluesky or Mastodon, and tag me, or send me an email. This was 'Create. Share. Engage.' with Professor Dr Cathy Elliott. Make sure to check out the resources in the episode notes in your podcast app or at podcast.mahara.org. And if you found this valuable, share it with a colleague who would appreciate it, too. Our next episode will air in two weeks. Until then, create, share, and engage.