Kristina Hoeppner 00:05

Welcome to 'Create. Share. Engage.' This is the podcast about portfolios for learning and more for educators, learning designers, and managers keen on integrating portfolios into their education and professional development practices. 'Create. Share. Engage.' is brought to you by the Mahara team at Catalyst IT. My name is Kristina Hoeppner, and I look forward to speaking with Dr Orna Farrell today.

Orna is Assistant Professor at Dublin City University in Ireland and specialises in online and blended education. She's published extensively on ePortfolios in her native Ireland and internationally, and she's a founding member of Eportfolio Ireland, the professional learning network for ePortfolio practitioners and researchers in the country. Thank you, Orna, for your time today to catch up. It's good to be talking with you.

Orna Farrell 00:54

Thanks, Kristina. Great to be here. 

Kristina Hoeppner 00:57

Can you please tell us a little bit about yourself so that we can situate you a bit more?

Orna Farrell 01:02

Sure. And thanks for that lovely introduction. So as you say, I'm a professor in DCU. But my interests in portfolio date long back in my career. I was trying to remember the year, I think it's 2008, that I began experimenting and working with portfolios. At the time, I was working in a private university in Dublin, called Dublin Business School, and I had a been given a module to teach that I knew very little about, about personal development. And I was talking to a colleague saying, "Oh, my God, what will I do here? I'm a bit stuck." And he said, "Why don't you do a portfolio?" I thought that's interesting. So I went and did a little bit of reading about portfolios, and I decided because I always enjoyed even back then things that were digital, I said, "Sure, I'll try an ePortfolio." 

That was the first time I dipped my toe into the ePortfolio world. It was a bit messy. It wasn't, I'd say the most beautifully designed assessment or learning experience for students, but we actually enjoyed the process. We were using a very user friendly tool to do the portfolio to begin with. We were using this little plugin in Moodle, I think it was called Diary or something very, very clunky. And I suppose I knew very little about ePortfolio pedagogy at the same time, but myself and the students worked on this together for a semester, and we really enjoyed the experience, and at the end, I got them to give a little presentation about their portfolios, and what they've learned and what they thought worked well, and what didn't work well. And I remember thinking after that is like, I'm really onto something here. This is really interesting. I've learned more about my students in that period than I'd ever learned before. I could see how they were thinking about their learning, and it really stuck in my mind, like this is something special, I need to know more about this.

So then I started working on a proposal to do a doctorate of ePortfolios, and I applied to Trinity College, to do a doctorate in education on the topic of portfolios. That took a really long time [laughter] because I did it as I worked, and I moved to DCU, and I had two babies. So it was a very long term project. It took eight years, but at the end, I really learned a lot about portfolios.

I did a beautiful study with online students and their experiences of learning with portfolios, and I interviewed them with their portfolios. I used the portfolio as a stimulus. I also analysed the data from their portfolios, the written, visual, the audio visual, and I suppose what emerged was some really interesting stuff about what it's like to learn with the portfolio.

So that's my kind of background. I suppose to hook in Mahara, in 2009, I persuaded the college I was working in to install Mahara and to combine it with Moodle, and create a Mahoodle. I was an early adopter, and I did a lot of early experimentation, and I trained staff and students on Mahara. And all of this was just kind of literally like an add on job. My role was to be a lecturer. I enjoyed the experimentation with Mahara, and we quickly formed a little community, I suppose around Mahara in the college as well. 

So when I moved to DCU, they were also in a very early stages of portfolio. My colleague from Eportfolio Ireland, Lisa, Lisa Donaldson, around that time became our ePortfolio champion, and I agreed to be a pilot in the project. So there was about 20 of us, I think, dotted around the university, and we introduced Mahara, but they rebranded it Loop Reflect and did a bit of customising and making it a bit fancy. That pilot proved hugely successful. And now, whatever, about eight years later, we have huge numbers of students and staff using portfolio. I think it's about 12,000 students at the moment, and I'm unclear about how many staff. And this peaked in COVID, too, which is interesting, Kristina. Obviously, because of all the changes people had to make to the assessment, but a lot of it was kind of hastily designed and implemented which is, I suppose, the kind of the story of COVID. So the student experience, I think of portfolios was a bit mixed. The feedback wasn't necessarily the best feedback you would hope for. But I mean, you know, careful design of these kinds of complex assessments is needed. I think that's the kind of the lesson out of that experience and careful integration.

Kristina Hoeppner 05:22

You also use it quite a bit for assessment then instead of going down the route of online exams like a lot of other universities have done, right?

Orna Farrell 05:32

Well, in Ireland, actually, online exams are not that common. They chose to use some, but most chose to use continuous assessment instead of exams. That has been a trend anyway. We were moving already away from the terminal exam, more from actually a kind of a philosophical and pedagogical approach because you know, the research on exams, you know, they're not a good supporter of or measure of student learning. So that trend had been underway. So during COVID, yeah, many, many lecturers changed to continuous assessment instead of changing to an online exam. Some didn't because in some, some disciplines, exams seem to kind of be part of their pedagogy nearly, particularly hard science and maths and stuff like that. But also, the ethics of proctoring would be not considered very favourably in Ireland or Europe. I know, there's varying opinions on that, but people are quite against proctoring, for obvious reasons. But maybe I shouldn't say no more about that.

Kristina Hoeppner 06:29

Let's leave it at that. You mentioned at the start that you have been using portfolios since 2008, with your students and that you learned much more about your students through their use of the portfolio, and you got to know your students. Have you seen that as a trend to continue also in the portfolio work since then because that's now already 14 years that you've been in the portfolio community?

Orna Farrell 06:55

I've used them in many different ways and contexts since my initial efforts, and I've supported many, many staff as well with their own practice. Certainly, for me, yeah, that trend has continued. I mean, the group, gosh, the group I followed for my PhD study, I knew them inside and out [laughter], which is amazing. But yes, since then, I think that trend continued. It personalises the learning experience, both for the students themselves, but also for the lecturers. And I think it builds a connection that helps engagement, and that's just one aspect. Obviously, it's the learning itself that's key. And I suppose it depends on the type of portfolio, too. I often think that people need to think more carefully about the purpose of a portfolio, you know, is it an assessment portfolio? Is it a learning portfolio? Because I think students can find that very confusing. They don't actually know what's needed. I always very clearly with students, 'This is why we're doing it. This is how we're doing it.' I think a lot about the design. I use very particular prompts. I try not to be too specific because I want to allow for creativity. 

More recently, I've become interested in accessibility and universal design for learning. So now when I'm writing instructions, I ask students to give a written, audio, or video reflection. You know, giving them choice, which is important to make the learning accessible tool. And I always try and prompt them to include evidence as well. So that if they're making a reflection of some part of their learning that it's connected to evidence, and the evidence is clearly explained. Because students will, you know, you'll get some really random stuff like pictures of their dog or something, which is fine. But you know, why is the dog here? A woman who had a picture of a dog, and I thought, "Why has she got a picture of a dog?" And then I read the reflection and it said, you know, when she's walking her dog, she thinks about her learning. And I was like, "That's perfect. You know, that makes absolute sense."

Kristina Hoeppner 08:41

Yeah, but you do need that explanation going with that image, right? Because otherwise, you might completely interpret it differently.

Orna Farrell 08:50

Completely. And I think you do need to give students some framework or some boundaries because they can get really enthusiastic and give you like, 1,000s of words or minutes to listen to or watch. So I think you have to give them good boundaries, you know, I want kind of this length, this amount of evidence because otherwise it makes the marking and the feedback very arduous. And that I think is possibly one of the challenges of portfolios is they really are an excellent form of assessment, but you have to be mindful of the workload both of the students onto the lecturer. And if you've got a well designed rubric - that was one of the projects we worked on in Eportfolio Ireland is to design a rubric for portfolio because that was something that people kept asking us about. And we did it as a nice kind of crowd sourced project.

Kristina Hoeppner 09:38

Yeah, the Holy Grail of Rubrics, wasn't it?

Orna Farrell 09:40

The Holy Grail of Rubrics, yeah. We did that at our first unconference in 2018 I think. That resource is still really used. I mean, I'm about to use it myself because I have a new batch of students in the new module about educational theory that are going to do a portfolio. So I was looking at the Holy Grail and adapting it to that context.

Kristina Hoeppner 10:01

I love how you scaffold the portfolio work for the students, and in particular also that you make it very explicit and also very transparent of what a reflection or what the evidence can look like. Because so often, even though we have been having ePortfolios, for a very long time, it is often that written reflection that we see rather than an audio or a video or in imagery representation. So making clear to the students that they can use any of those media I think is really fantastic.

Orna Farrell 10:35

But clear assessment instruction is really important. A colleague of mine has just completed a PhD on that exact topic, and how students often have no idea of what is expected, and that the instructions they receive and the clarity and the language used, it really matters. So I try and make things explicit. I also use exemplars. So we have a lovely bank of exemplars built up in DCU of student portfolios. People find that portfolios a bit abstract until they see examples. That's one of the things that I always do with both students and staff when I'm talking about portfolios. Just let's look at some portfolios. These are the kinds of things people have done, this is the context, this is the what they had to do. You find that the understanding, you know, it really builds understanding much more than just talking about portfolios.

Kristina Hoeppner 11:25

You also have the Annual Showcase and ePortfolio Awards, which are a fantastic way to make that work of the students public and also shared within the University and beyond.

Orna Farrell 11:37

Absolutely. And that's a lovely idea that Lisa brought in, I think, four or five years ago, but it's also one of the vehicles that we get fresh, new examples because each of the Student Award winners, we ask them permission, can we share your portfolio. So this is the reason we have a bank of really good examples. We also have an excellent bank of resources for both the students and the staff, that familiarising yourself with the pedagogy, but also the technology of portfolios, you know, it has to be part of this process of bringing the portfolios in. It is a form of change, of organisational change. So the support needs to happen early on in the process.

Kristina Hoeppner 12:15

It is not just a pedagogical tool, but it is also a technological one, and therefore, you kind of do become a bit of a tech whiz, when you work with portfolios as a teacher, don't you?

Orna Farrell 12:26

Absolutely. Well, you have to know what you're like with any tool you're going to introduce in your classroom or in your online classroom or whatever the setting. You need to become nearly an expert to teach other people. So it's a great way of learning new skills. In order to be able to teach someone else you have to teach yourself first. I always just go with the usual 'I press all the buttons and try everything out.' And then I teach other people the same way.

Kristina Hoeppner 12:51

Best method of doing that, and usually you also know you can't really break anything. So it's a safe space to experiment. 

Orna Farrell 12:58

That's the thing. Yeah, I mean, Mahara has become more user friendly as time has gone on. I mean, initially, in the early days, there was nearly too much functionality. But now it's become definitely a much more straightforward tool to use. And I find that students adapt quite quickly.

Kristina Hoeppner 13:14

Orna, you had already mentioned Eportfolio Ireland a little bit and earlier, your own research interests and your work in that area. So definitely, I can't get around also mentioning the special issue that you were involved in with a bunch of others during the pandemic, putting a two volume issue together really. And with a lot of different ePortfolio stories. Do you want to tell us a little bit about that? I'll definitely make sure to link to that as well as any other of the public examples that you can share so that people can follow up with them.

Orna Farrell 13:52

So yeah, I suppose maybe I'll just talk a little bit about Eportfolio Ireland. The origins of Epportfolio Ireland lay in that same pilot of ePortfolio in DCU. So that initial group of pilots, we then formed a little user group. It was a Mahara User Group at the time. And then as we began talking to people outside our own university at conferences and stuff, they were like, 'Oh, we're quite interested in portfolio,' and we made the decision to form a community of practice or a network, I'm not sure which we call ourselves a network called Eportfolio Ireland and to make a platform agnostic because people were using a variety of platforms. So you know, when we had no particular attachment to one platform over another, although we do like Mahara. 

So then it came about and we started organising little community events. They were very small initially. We organised an unconference. We got a little bit of funding from a national agency here called the National Forum to do a few more events. I started getting invited to do a lot of professional developments outside our own university. And we still do do that, and we still continue to get invited to do quite a lot. 

And we began collaborating on little projects. We found this was a good way of stimulating the community. One of our early projects was the Holy Grail of Rubric. Then we created an ebook. And that was to give examples of assessments because again, people need examples. But also they need examples in their context. A lot of the literature and practice in ePortfolio was from outside of Europe where the educational context really is quite different, and the philosophy is different also. So you know, we continue to work on different resources. We had a kind of an open policy as well, although I don't know if we ever actually explicitly said it because all of our resources are Creative Commons licensed and published open access and are available to anyone on our website. So that philosophy was there, too, but we never really made it explicit, but it is. Maybe we should make it more explicit.

Two of us in the steering committee, myself and Tom Farrelly, would be both very interested in open education as well. And that's where also just to connect it to the journal, so myself and Tom are although I - since the special issue, I've become an editor of the IJTEL, so the Irish Journal of Technology Enhances Learning.

Kristina Hoeppner 16:06

Congratulations. 

Orna Farrell 16:07

But Tom was brief - was an editor - well, thank you. He was an editor there, and we had an idea, 'Why don't we do a special issue' because one thing we realised from our research in the space is that there was really in Europe very little, a little bit of UK literature, a lot of American literature, a bit of New Zealand, Australia literature, but really hardly anything about Ireland, and the two or three articles that were published in Ireland were a while ago, and we knew the people who publish them [laughter]. There's a huge gap here. Let's see what can we can do. 

So we had the idea for the special issue. And at the same time, we had the idea of doing a national survey of portfolio practice in Ireland. So we had those two projects, and we decided, 'Why don't we put them together? We'll write up our research about the national study and that can form part of the special issue.' The call went out before the pandemic, I think it was 2019. Loads of interest. And our plan was to be quite - to do a quite of a supportive model is a little bit like how the AePR journal operates. So that we were trying to encourage members who may not have published before or recently or were dipping the toe in the water of this space. So we wanted to be very supportive. So we had a couple of support sessions. We had a pomodoro session. We had a session about writing an abstract, and people were really enthusiastic. We kind of hit a tone.

Kristina Hoeppner 17:34

You had a number of first time authors and a lot of women authors in there, right? 

Orna Farrell 17:40

We did, yeah. We had a large number of women authors. I could find the stats. I did a bit of analysis of it. And also, I suppose that struck a chord because when when the actual special issue process got underway, it was deep COVID. And a lot of the issues around female participation and female work life balance and publishing that was kind of a hot topic, inspired us a little as well. So the femedtech group published this very interesting open letter about women scholars and how they were faring in the pandemic. So we kind of wove that in. That turned out to be a great kind of angle as well, you know. We presented on it at the OER 2022 conference in London there in the spring, and it was very well received. And the whole, I suppose feminist angle was one we didn't expect in a way, but it was a happy coincidence.

Kristina Hoeppner 18:32

The editorial says 83% women authors. That is a very significant amount.

Orna Farrell 18:38

Yeah, it was. It was huge. But I suppose three out of the four editors were women as well. I was the lead editor. Yeah. And Tom and Lisa. So the journal is published in the Irish Journal of Technology Enhanced Learning. And that is the journal of ILTA. So ILTA is the Irish Learning Technology Association. All of us are involved in ILTA, as well. It is the Irish equivalent of ALT in the UK, ASCILITE would kind of be that space in your part of the world. So we were all members of that, and that is the journal associated with that. And it's a good fit for the journal as well. 

Kristina Hoeppner 19:16

In New Zealand's the somewhat equivalent would probably be the Flexible Learning Association of New Zealand.

Orna Farrell 19:22

Yes.

Kristina Hoeppner 19:23

It is very much focused on flexible and distance learning.

Orna Farrell 19:27

Yeah, I've read their journal as well. Yeah, it's good. And AJET would be the kind of equivalent one in Australia.

Kristina Hoeppner 19:34

Those were the two big projects for Eportfolio Ireland over the last couple of years, and they were massive projects. I mean, I was really looking forward to reading the survey and then seeing the results and how far portfolios had spread in a relatively short amount of years, we must say, right? But what are your plans for the coming few months or a year? What would you'd like to be able to do in that network?

Orna Farrell 20:02

Yeah, so you're right, the special issue was a big milestone for us, and particularly, the study about portfolio in Ireland. So, you know, we have plans to develop further phases of that project, to run the survey again, to collect data for 2022, and compare it to previous years. And also, we have a related qualitative study planned as well. So interviewing particularly staff because there's more research on students and staff about their experiences of portfolio. And one trend that we noticed with the special issue particularly was the growth in the use of portfolios for professional learning. That seems to be a growing area. So we're kind of curious about exploring that further. So that's the kind of future plans on the more research side.

On the professional learning side, we have a great collaborative session coming up in October with AAEEBL, about digital ethics and accessibility in ePortfolio design coming up on the 25th of October. So we're really looking forward to that collaboration. And that's something we're doing more over the years is collaborating with different organisations. AAEEBL is one. We also have collaborated with ePortfolios Australia, and we're very open to collaborating with anyone else really who's interested. 

Kristina Hoeppner 21:20

So call for papers and call for collaboration is going out hereby, and people are invited to get in touch with the Irish community. 

Kristina Hoeppner 21:28

So Orna, what would you like to be able to do with portfolios that you can't just yet do?

Orna Farrell 21:38

That is a very good question. I'd like to design a kind of a radical portfolio module, which might break many of our systems. Something perhaps for postgraduate students that I think might capture the postgraduate learning experience. We're very focused on a modular system, even at postgraduate level. So something that goes outside of modules. You know, for example, to take my own experience, I was studying for eight years for my PhD. You can imagine what a portfolio would have looked like over eight years. I think there's an interesting concept there, that by capturing those huge learning to capture there, but again, it breaks all the systems. So that's one idea.

Kristina Hoeppner 22:22

Well, we can dream, we can dream, and then somebody can build it. It sounds a bit like you have this massive portfolio and then kind of narrowing it down and drilling down more and more so that you can extract the main use in a way.

Orna Farrell 22:37

I think would really help the research process as well, that reflective cycle. Like people do it naturally anyway. You spend a lot of time reading and thinking what you're doing for their study. If you could somehow capture that and then return to it at different points to see 'That's how I thought about that then that's really interesting. I've completely changed my mind.'

The other thing I think would be really cool, and actually, again, could be a great collaborative project. Should we have a MOOC about portfolios and portfolio learning, you know? I think it could fill a nice gap. If it was on one of the larger MOOC platforms, and it was open to all.

Kristina Hoeppner 23:10

And to collaboration between the three organisations that you had already mentioned, Eportfolio Ireland, ePortfolios Australia...

Orna Farrell 23:16

Yeah.

Kristina Hoeppner 23:16

... and AAEEBL as a start and then have other communities come in too.

Orna Farrell 23:21

Like a 10-hour short course or - that people could self study, but that was obviously written by experienced practitioners, and maybe, you know, could be situated in a variety of contexts. I think that could be a really nice resource for the community, but open, not closed.

Kristina Hoeppner 23:37

I see another an unconference in your future where the plan might be developed for such a MOOC.

Orna Farrell 23:45

I think it could be great.

Kristina Hoeppner 23:46

We are coming to the end of today's session already really, really quickly. And so I'd like to ask you the same three questions that I'm asking everybody. So the first question is, which words do you use to describe portfolio work?

Orna Farrell 24:01

I typically go for 'learning portfolio' these days, and I don't talk about ePortfolio. I've moved away from that kind of conception because I think the technology is secondary. What's more important is the process and the learning. So that's why I think to go with portfolio or learning portfolio.

Kristina Hoeppner 24:17

Onto a tip for learning designers and instructors who create portfolio activities. What can you advise them?

Orna Farrell 24:24

Creating like a checklist or a grid or like a design sheet for academics is a very useful approach. You know, like, 'Okay, have you given prompts? Have you given exemplars? Do you have a rubric?' and trying to nudge people towards good design. Then also, good help resources for students and staff.

Kristina Hoeppner 24:47

Lastly, what advice do you have for portfolio authors? So the students or also the professionals creating portfolios.

Orna Farrell 24:56

Learn about reflective writing. There is a particular way to write reflectively that many of us have not done before. So there's a different voice, a different style. Anchor your writing in some kind of reflective framework.

Kristina Hoeppner 25:10

Do you have a preferred reflective framework?

Orna Farrell 25:12

Actually, I was just trying to remember which ones I said. One second, I will refer to my assessment brief. I like the 'What? So what? Now what?' and I'll give you the reference because I just put it in something. Yeah, Borton's, 1970. That's 'What? So what? Now what?' Gibbs is an old reliable as well. You need something to structure it around. Like you can go outside the structure, but I find unless you have that in place, you do not reflect. And also for students I would always give them examples of reflective writing because they are so trained into academic writing. So for example, there's like an academic writing in our part of the world, you typically never write, 'I'. So you write in indirect speech a lot. So in reflection, you are allowed to use 'I'. There's academic conventions that are different. Teach people how to do that. 

I suppose the thing is, reflection is uncomfortable, but it's the cornerstone of this type of learning. So one thing I try to encourage people to do is, you know, 'You've written a reflection. Leave it for a while, a couple of weeks, and then look back.' And sometimes that's when new learning will happen. You will look, 'Oh wow, I've changed. I've learned this, this and this since then.' So it's that cycle of looking back and looking forward, I think is really important.

Kristina Hoeppner 26:21

Thank you so much for these wonderful tips. I'll definitely include them in our growing list of tips that we can then impart to other community members and of course, make them openly available. So thank you also for this conversation, Orna. I really loved hearing about the work you do the research, what influences you, in particular also how you scaffold the work for the students to make it transparent to them, therefore living the principles of universal design in learning.

Now over to our listeners, what do you want to try in your own portfolio practice? This was 'Create. Share. Engage.' with Dr Orna Farrell today. Head to our website podcast.mahara.org where you can find links to resources that she shared and the transcript for this episode. This podcast is produced by Catalyst IT, and I'm your host Kristina Hoeppner, project lead and product manager of the ePortfolio platform Mahara. Our next episode will air in two weeks. I hope you'll listen again and tell a colleague about old podcast so they can subscribe. Until then, create, share, and engage.