Kristina Hoeppner: 0:05

Welcome to'Create. Share. Engage.' This is the podcast about portfolios for learning and more for educators, learning designers, and managers keen on integrating portfolios with their education and professional development practices. 'Create. Share. Engage.' is brought to you by the Mahara team at Catalyst IT. My name is Kristina Hoeppner. Today I'm speaking with Dr Lindsay Richardson from Carleton University in Ottawa in Canada. When I visited the university in May, I learned that she's been using portfolios in very large psychology classes that can also have an artificial intelligence(AI) component. We didn't have the chance to talk more, so I invited her to share her thoughts and experience on the podcast. Thank you for making time, Lindsay.

Lindsay Richardson: 0:51

Thanks for having me, Kristina.

Kristina Hoeppner: 0:55

Lindsay, what do you do at Carleton?

Lindsay Richardson: 0:57

That's a big question [laughs] right out of the game.

Kristina Hoeppner: 1:00

I know you're wearing multiple hats there, don't you [both laugh]?

Lindsay Richardson: 1:03

I have so many hats. What I would say is, right now, I am the Educational Technology Manager in the Teaching and Learning Services Department. So what that means is I work with a group of wonderful people, of which some of them you know, who help to consult on and support the pedagogical uses of educational technology, including cuPortfolio, which is our ePortfolio solution you're familiar with. I also, though, am an Adjunct Professor and Contract Instructor in the Department of Psychology, where I teach mostly large enrolment courses. So this past fall, I had 450 students, and this past winter I had 220. So big classes, mostly intro level. I teach a lot of 'Intro to psych', 'Intro to cognitive psych', although I started out in my teaching journey at Carleton, teaching 'Intro to stats for psychology students', and that, to date, is my favourite course to teach. Most psychology students don't want to take statistics, and they all have to. So they come into my course and they hate me right out of the gate. They don't want to take my classes. But then they leave and they say really nice things, like, 'Thanks for making it not so bad [both laugh]. But yeah, so those are the two roles that I have right now, but So with the 'Intro to psych', students come in with this notion that it's going to be a bird course. So they're really interested right out of the gate, and invariably, I let them down somehow, and then by the end of the semester, I get things like, 'This was needlessly difficult!' So it's I will say that I've been at the institution since 2010. Against not as rewarding. all the advice of our forefathers, I guess I could say[laughs], I did all of my degrees at Carleton. I really like the institution, I come from here, I am from Ottawa, born and bred. So I was happy to stay at Carleton and do my BA, my masters, and my PhD, which I successfully defended in 2023.

Kristina Hoeppner: 3:10

You do have a beautiful campus right on the river.

Lindsay Richardson: 3:13

It's so nice, yeah.

Kristina Hoeppner: 3:15

I did get the chance to walk the river walk with Chloe when I visited. And so it's a really wonderful opportunity for lunchtime walk, getting outside. Well, as long as there are no mosquitoes, I guess in summer.

Lindsay Richardson: 3:28

Yeah, or birds there's lots of geese[both laugh].

Kristina Hoeppner: 3:31

What you actually forgot, Lindsay, is that you're also having a podcast of your own.

Lindsay Richardson: 3:37

Ah, yes, I do [laughs]. I shouldn't forget that. I co-host - it's called'Agile Ed' with my co-host, Dr Ashley Thompson, who is also an adjunct professor and contract instructor at the institution. She teaches out of the Department of Neuroscience. She often teaches smaller courses, which means that between the two of us, we have a pretty big breadth of teaching experience in terms of scalability.

Kristina Hoeppner: 4:05

Yeah, and it's fantastic listening to your podcast because you have a wonderful conversation style between the two, and it really comes across that you two have been working together for a while, that you like working together, and that you're really playing off each other.

Lindsay Richardson: 4:19

Yeah, yeah. Thank you for that. I am quite proud of the conversational tone.

Kristina Hoeppner: 4:24

Lindsay, when did you start using portfolios?

Lindsay Richardson: 4:26

Yeah, I have to tell you that I never used a portfolio as a student ever. I didn't have any instructors who had them in their courses. I will say I was also a first generation student, so I didn't have family to lean on either to know what they might have encountered. Everything that I experienced and thought about in terms of university and what is traditional it all came through just what I was seeing and what maybe I saw in the movies, for example. I thought that portfolios were meant for art class, right, that you would put together pieces of artefacts, and you would show them, or maybe an employer, if you were a graphic designer, an employer would want to see your work. That's where I thought they started and stopped. I was hired in 2020, August of 2020, with what used to be called the Educational Development Centre. So that's Teaching and Learning Services now was the EDC back then. My PhD supervisor used to joke that if I continued to spend my time over there, that they might one day give me a job, which[laughs] isn't exactly how that happened. Yeah. So I mean, you'll remember 2020, summer 2020, we were in the midst of a global pandemic, we needed to shift all of our courses online. So the EDC was doing a pretty big hire, and I was one of their hires to help faculty move their courses online. Incidentally, I was already teaching online. So I was going to need to support cuPortfolio, and I had never used cuPortfolio before. How was I going to do that? How was I going to support instructors' use of this tool that I had never used before? And I thought, well, I may as well just use it. Right, like I have to get in there and I have to I took what was the beginning of my teaching dossier that was a use it. work in progress, and so I was kind of thinking this like feed two birds with one seed, so to speak, I could do something that I'm going to need to do anyway, and I can learn this tool. And that's exactly what I did. So I took what was a Word document, put it into cuPortfolio, and it was so cool. I learned lots about the tool, and I thought, this is pretty cool, and could have lots of uses in psychology. Think about psychology, and most people think about 'Intro to psych', one to two midterms, final exam, and you're lucky if you get some quizzes along the way. Traditionally, that's what an 'Intro to psych' class is, and it makes sense. It's mostly large enrolment courses. There's lots of material, lots of content that you have to memorise, lots of foundational knowledge. And so when I started teaching the 'Intro to psych' sessions, I went, mhh, what if we just had a portfolio? Like just one small component of it? So about two or three years ago, this also came alongside my learning about alternative grading approaches or ungrading, which I've learned is a movement that predates the 80s even. I thought it came out of the 80s, but I've been corrected. I think somebody told me the 60s is when they started talking about ungrading.

Kristina Hoeppner: 7:34

Yeah, some of those teaching methods do have quite a long history. Like portfolios, they are nothing new at all. They've existed for centuries. Of course, not the digital portfolio, but the portfolio itself, in particular, in the iteration that you have mentioned, for the artists where they are showing off their best work. Teaching portfolios have been around, at least in North America, since the 70s as well.

Lindsay Richardson: 7:56

That's incredible. So all these innovative teaching practices that I just would have never even thought about have been around for so long. So I thought, I mean, I wanted to radically change the entire system, and I think I still kind of do, but one step at a time[both laugh]. So I thought that's one thing I can do in my course that is a little bit ungrading or a little bit in line with alternative grading approaches. And I thought, ooh, a portfolio that is self graded. Why not? Right? It's a portfolio. If it's meant to be metacognitive, then it's meant to be self graded as well, I think, it's a perfect candidate. So that's exactly what I did. I started employing them in my'Intro to psych' and 'Intro to cognition' courses where students complete what's called a success portfolio. They first have to define what success means to them, which could be different depending on what students want, right? Some students will flat out say,"Success is an A+ in this course." That's it. I hope that's not what success is, but who am I to judge? Some students will say, "Well, success to me means being able to adequately articulate psychological phenomena related to my everyday behaviours." Okay, great. I pair it with a module on the psychology of success that also has a bundle associated with it, an indigenous knowledge bundle from some of our indigenous knowledge keepers here at Carleton. They talk a little bit about non-western versions of success as well. So students have now the ability to go out and to think through different forms of success from different people and different ways of knowing. In their portfolio, they articulate what success means to them. They look through the learning outcomes in the course of which there are often in my courses, seven. Any educational developer right now is gritting their teeth, seven learning outcomes, not so many [laughs]. Yes, it's too many. So I have students choose between three to five learning outcomes that they want to work on in the course. I don't need to measure all of them in most of the courses, three to five is sufficient. Then they plan for how they're going to achieve those learning outcomes. Very concretely, they say things that they'll do throughout the semester, and then they engage in learning cycles throughout the portfolio to demonstrate with artefacts what they've done, what maybe didn't go the way that they thought, what feedback they got, how they integrated their feedback. And so you're really seeing a journey of their learning throughout the course, where at the end, they reflect on the entire journey, they talk about that, and then they also submit their rubric and their grade for the portfolio.

Kristina Hoeppner: 10:44

And that's in your classes with 200 to 450 students?

Lindsay Richardson: 10:48

Yes.

Kristina Hoeppner: 10:48

Lindsay, you said that your students are self grading their portfolio. What does that look like?

Lindsay Richardson: 10:55

It doesn't look like the Wild Wild West, is what I'll say. It's not that students are just submitting a recipe for guacamole and giving themselves 15 out of 15, for example. I make it very clear in the syllabus and in the assignment instructions that any large discrepancies between the grade that they provide themselves and the quality of work submitted will need to come through me, and they'll need to come and have a conversation with me to ultimately defend their learning. More practically, what that looks like is students get a rubric that they engage with in order to grade themselves. So they have to put themselves on the scales for all the different rubric criteria. And I try to use really plain language to ensure that there's minimal subjectivity in the rubrics. So I avoid using words like'mostly' or 'some of the time', or, you know, things that can have a certain layer of subjectivity that isn't helpful for students. So they really can almost objectively measure their own success. They also have points of practice throughout the term on all of their deliverables I ask them to grade themselves, which I do believe is a great practice anyway. Why wouldn't students grade themselves before submitting their work? So I have them do that, and then they practice their level of calibration throughout the course by submitting their self assessment and then seeing the teaching team's assessment of their work and noticing the discrepancies between that so that, hopefully by the end of the course, they'll be more well calibrated. And then, instead of having to go through and grade all of the portfolios, what we do is we go through all of them as a team, and we look through and make sure that the grading matches on, right. We're just looking for large discrepancies. We're not looking to nickel and dime students. That's not what I'm interested in. Instead of having to grade, what we can do is we can provide authentic reactions to their portfolios. So we're not so much worried about making sure everybody gets the same amount of feedback or the same amount of comments, or like, really looking into the nitty gritty of the grading, we can forget about the grading that's already done, the work's already done. We can really concentrate on feedback.

Kristina Hoeppner: 13:08

And supporting your students through that feedback that they then hopefully also read.

Lindsay Richardson: 13:11

Yes, hopefully, yeah [both laugh], exactly. But I have found, for example, in my winter course, I asked my TAs not to grade the portfolios. I graded each and every one of them myself, and I found that it didn't take a really long time either.

Kristina Hoeppner: 13:26

Now that you've run your course with the portfolio component for two to three years, and you had not used a portfolio in at least one or maybe even two years prior to that, after you had started, have you seen a difference in the students' learning?

Lindsay Richardson: 13:44

I have. So I have some anecdotal evidence from students, via 'Students as Partners Program'. We have a'Students as Partners Program' at Carleton. It's paid work, so they get a grant, and they work alongside a supervisor who is an instructor. They work on projects that help to boost teaching and learning. So I had a student, this partner, work with me on improving the portfolio. So they improved the instructions. We actually took the instructions and put them right in cuPortfolio. So now students have to engage with the software even to read the instructions. So they did that. They tweaked the rubric a little bit and conducted a small scale survey on the students from last cohort to ask questions about their experience. What was surprising because it wasn't a question we were asking students, is students started divulging that they were less likely to use AI to help them with their work. What we found come out of this, and I'm really looking forward to doing some actual research in this area, so I'll have some data hopefully soon. But just like what we've noticed with the alt grading movement that has come from the 60s, right, is that when students are motivated by grades, they're robbed from their love of learning, essentially. They're motivated by their grades. That's extrinsic motivation, and that's one way that we call an AI hazard. That's one way for students to want to offload their work because they're scared that if they don't, they're not going to get the grades that they need to get to the next level and do the next thing. So with this type of assignment, students were saying, like, 'I'm not going to use AI because I'm going to grade myself. Why would I use AI when I'm just going to grade myself?' Or 'why would I use AI when this is about my journey?' It's made it deeply personal for them. I have nothing to compare it to because for as long, as you know, ChatGPT has been around, I've been using these portfolios. My non-use of portfolios predates November 2022 unfortunately[both laugh]. What I can say, though, is it brings it to the surface, right? I'm no longer relying on our teaching evaluations or student feedback surveys to see the learning that's come out at the end of the term, right. I get to read beautiful reflections about their learning and have students clearly articulate things to me that they're taking out of my course that maybe I didn't even intend to. It's quite beautiful to see what students are taking out of the course.

Kristina Hoeppner: 16:17

Mhh because that's when you realise that they've actually learned and taken on what they have learned and applied it to their own contexts.

Lindsay Richardson: 16:27

Absolutely, yeah.

Kristina Hoeppner: 16:28

So your students don't necessarily use AI in their portfolios to write their reflections, but you do allow the use of AI. What does that look like?

Lindsay Richardson: 16:37

I mean, I think next semester, it'll look different from last semester, which looks different than the semester before [laughs]. I'll say that. It's definitely an iterative process. I will say, though, that the graphic design component of the portfolio is not a learning outcome of mine. I don't need to measure evidence that students can use the software or that they have graphic design capabilities. So one example is students can engage with AI or chat bots to help them make their portfolios more visually appealing or help reduce the cognitive load of their audience. AI can offer advice on where to put artefacts or elements on a page, for example. And you know, it's a software which means that there's going to be a little bit of a learning curve. So if students want to engage with AI for some troubleshooting, that helps me, honestly. It helps to reduce my workload as an instructor, where students can engage with AI to help them learn the tool, which, again, is not a learning outcome of my course. What I do is I just ensure that as with all deliverables in my course, students submit an AI disclaimer that says how they used AI and exactly what they used. They have to name the model, when they used it in line with APA formatting, and talk through how it helped their learning. What I'm finding more and more is that having this information in the syllabus is not enough. Now I have exactly how they can use AI in my course in my syllabus, and then on each assessment, it needs to appear in the assessment instructions. And I also have a rubric item now. One of the criteria is that you have to have submitted an AI disclaimer and that it is in line with the template and in line with the policies, our course policies.

Kristina Hoeppner: 18:30

The students use AI in your courses, currently, primarily for structural elements so to generate images that make their portfolio more visual, plain English, so I assume something that helps them rewrite and be more concise in their writing, and then also how they should structure their reflections.

Lindsay Richardson: 18:51

I would also urge students to avoid the use of AI to polish their language. We talk a little bit about the different types of language for different types of submissions. And while AI can be really helpful in structuring work that is meant to be templated, like a manuscript for a psychology paper, for example, that is pretty templated language, there is right and wrong tones. I mean, there's a little bit of an art to it, but for the most part, AI can be very helpful in helping students be more streamlined and concise in their writing. But for a portfolio that's meant to be a little bit less of a professional tone because I'm using this as like a metacognitive journey, rather than something that's meant to be super polished and ready for an employer, I would really urge them to use their own voice and avoid the temptation to have aI clean it up because what they might think is cleaning it up is making it real boring. Everybody will just say how the things have been woven through the tapestry of the landscape. And like [Kristina laughs], I don't want to read that. That's too boring to me, too flowery.

Kristina Hoeppner: 19:57

What was the use of AI from your students that you've seen in the rubric since you have implemented that and also the disclaimer, is there anything that they use predominantly or anything that nobody uses at all?

Lindsay Richardson: 20:10

Interesting question. I don't have them all in front of me right now, so I'm going based off of memory. One thing that I can say, though, is I was surprised at the number of students who were telling me that they used it for translation. So I had a number of students who were French as a first language, and who had said that they completed their entire assignment in French and then used AI to translate it into English, which I think kind of cool. People get paid a lot of money to translate work, and so if we're asking students to do this work, and then a subset of our students are having to do the work that we're asking them to do and also translate it, that is completely inequitable, in my opinion. They shouldn't be having to translate. That's not the learning outcome for the course. Maybe if they're going into jobs where they'll need to work on their English, they might need to do that. My course isn't necessarily the place that they need to do that. So if they can outsource some of that work, great, if that allows them to truly dig in and engage with the material authentically.

Kristina Hoeppner: 21:21

Is there anything else that had stood out for you, in general, in the use of AI by your students?

Lindsay Richardson: 21:27

Yeah, I mean, so there's a spectrum out there of students, students work, and there's work that I can see that is, I mean, if they're using AI, they're doing a really good job hiding it, and they must be fixing it up on their own. And so there are students that seem to be avoiding it and engaging authentically. And then there are students who all the way on the other side of the spectrum, who seem like they might just be completely disengaged, but it feels blatantly obvious. With a paper it's harder to tell, right, because if a student writes 'delve' 12 times, most instructors would say 'delve' is a telltale sign that you're using AI. Well, that might not be true, and especially not in five years, when students start engaging with synthetic material more and more, they're just gonna start writing like AI writing. So it's hard to say it's like a chicken and the egg problem, like which came first? We don't know, and so it's going to be hard to pinpoint and say,'you definitely wrote your paper with AI.' Even with those horizontal lines that we start to see now, the HTML lines that chatgpt uses to separate the sections. I'm starting to see more and more of that in students' writing now. I'm like, where did these horizontal lines come from? Nobody's ever submitted horizontal lines before [both laugh]. Now, papers are riddled with them, annotated bibliographies, especially. But with a portfolio, it's the whole assignment. If I open a portfolio, for example, and the template that I give students is seven pages. They're meant to take these seven pages and do something with them. So if I see seven pages, each of which has one block of text or even it has a few blocks of text, but there's bolding, and then it's kind of like a block of text with a few bolded items and then bulleted items with bolded first parts, and then another block of text, to sum it all up, that looks templated by AI. Whereas if you open up a portfolio and you're seeing artefacts and you're seeing images and you're seeing videos and quotes pulled out and it looks very intentional. It's easy to tell whether students have short circuited their learning AI or not.

Kristina Hoeppner: 23:30

Do you use AI yourself in your teaching with portfolios?

Lindsay Richardson: 23:37

Yeah, I do engage with AI alongside my teaching. I created a custom GPT using the paid version of ChatGPT, which I'd love to stop using because it's in US dollars[laughs], which is an astronomical amount of money for me as a Canadian right now, among other reasons. I've created the custom GPT that I've called 'Psychology course designer'. This GPT is trained on my teaching dossier, my teaching philosophy statement, the history of all the syllabi that I've ever created. It has certain articles that I've pulled that I think are relevant for instructional design in my area and a few other materials. So now I have this AI bot'Psychology course designer', and I'm not offloading it to get it to design my course for me, but what I'm doing is I'm leaving it open on the side, and if for nothing else, it helps to keep me on task because I no longer feel like it's an astronomical ask to design a course. I can sit down and I can plan myself, and I can offload my cognition to this AI bot and say, like, I'm filled with ideas. And you can probably tell by the way that I speak that my ideas kind of just shoot out of my head, and I chase them, and I have a hard time putting them all together, but AI can help with that. I'm sat down, and I start all my course. Okay, I'm working through my syllabus. Here we go, and then any cognitive offloadings I can offshoot to AI, I can ask it questions, and I can engage with it that way. I write the assignment instructions myself, mostly because they already exist, and what I started to do was get it to tell me what it thinks it should do. Notoriously, AI right now is pretty bad at giving constructive feedback, like it's really good at telling you how great you are at things[laughs], but not so good at outlining ways for you to improve. So if I give it my assignment instructions and say,'Is this clear', for example, which is a leading question. AI is not so great with leading questions? Is this clear? The bot will come and it will say,'Yes, it's so clear, you're the master of the universe and here's all the reasons why'[laughs], right? But if I spit it on its head for a minute and I say, here are the instructions for my assignment, do the assignment? Okay? Well now I can see what it does, and that helps to outline all of the ways in which I was not clear.

Kristina Hoeppner: 26:01

That can be one tip for educators to still write your own assignments, but then use a bot to figure out whether it could be understood and whether you'll come across with what you want to do. Do you have any other AI and portfolio related tips for other lecturers or learning designers?

Lindsay Richardson: 26:21

Yeah, one is to think carefully and critically about your level or degree of integration. It's necessary at this point to think about how you want students to engage with AI in your course, and how you want AI to be infused in your course, and it might be none at all. That's okay, but it needs to be made clear to students, and the reasoning and rationale behind it needs to be made clear of students. One of the reasons why is confirmation bias. So if I am an instructor, and I say students in my course can use AI however they want? And you just kind of like, tell me how, and that's great. And then Kristina, you have a course, and you tell your students you're not going to use AI under any circumstances. If that's all students have to go on, and there's a student in both of our classes, they're going to go, 'Kristina, you have no idea what you're talking about [laughs]. I like Lindsay's version better, so I'm going to apply her course policies to your class because I'm just taking her word for it', not understanding that there might be nuances behind why we've made those choices and why it might be reasonable that students should avoid AI use in your class and not necessarily mine. So not only thinking through the degree, but being sure that we're critical about that and that we're making it clear to students. The flip side of that is to not require students to use it unless the course is about generative AI. If that is not a learning outcome in your course, though, we need to make sure that we're giving students the ability to opt out of Gen AI use and not exposing them to synthetic content without their consent because there are very real ethical reasons why students might want to avoid using these systems, and they need to be at this point in time, given the opportunity to disengage from AI. Maybe they don't want to leave such a large carbon footprint. There are very big climate impacts and energy demands of these systems, and we might have some environmentalists in our courses, or some people from different cultures that might feel like AI undervalues their culture and that their ways of knowing are not necessarily represented in the training data.

Kristina Hoeppner: 28:39

Yeah. It's been great to hear about your use of AI, how you think about it, also the transparency level with your students, that you're making it very clear to them what the consequences are or what they might want to think about, and then also using the alternative grading methods for them to critically engage with their learning, and again, also reflect on that learning because they need to put themselves into a different position when they are putting their hat on for judging their own work. So they are also engaging in a reflective process for themselves there. So the last three questions, then for you: which words or short phrases do you use to describe portfolio work?

Lindsay Richardson: 29:20

I would say transformative, intrinsically motivating, and student-centred.

Kristina Hoeppner: 29:27

What tip do you have for learning designers, instructors, educators, who create portfolio activities?

Lindsay Richardson: 29:34

Pair them with very clear rubrics.

Kristina Hoeppner: 29:40

You design your own rubric, right? Or do you sometimes also use a pre existing one, like one of the VALUE rubrics from AAC&U?

Lindsay Richardson: 29:48

I create all my rubrics myself.

Kristina Hoeppner: 29:51

What advice do you have for portfolio authors, be they students or also staff, who might be creating their teaching dossier?

Lindsay Richardson: 29:59

Mhh, I would say, find a way to stand out and reduce your audience's cognitive load.

Kristina Hoeppner: 30:06

What are some examples for standing out?

Lindsay Richardson: 30:09

Ooh, I would just say, look different. Be different. Do what feels authentically right to you, and don't worry so much about traditional standards that a portfolio ought to look like this or should look like that. I often say to my students, don't let anybody 'should' all over you like if somebody's coming at you with 'you should this, that' forget about it for this, especially. It's a portfolio. it's meant to be about you. Put something that makes it you. That's how you can really stand out.

Kristina Hoeppner: 30:42

So nice personalisation component there that also matches that students can choose their own learning outcomes. So really personalise the course to them. Thank you so much for today's conversation.

Lindsay Richardson: 30:53

Great. It's been awesome talking to you. Thanks so much.

Kristina Hoeppner: 30:59

Now over to our listeners. What do you want to try in your own portfolio practice? This was 'Create. Share. Engage.' with Dr Lindsay Richardson. Head to our website, podcast.mahara.org, where you can find resources and the transcript for this episode. This podcast is produced by Catalyst IT. Our next episode will air in two weeks. I hope you will listen again and tell a colleague about our podcast so they can subscribe. Until then, create, share, and engage.