Kristina Hoeppner: 0:05

Welcome to'Create. Share. Engage.' This is the podcast about portfolios for learning and more for educators, learning designers, and managers keen on integrating portfolios with their education and professional development practices. 'Create. Share. Engage.' is brought to you by the Mahara team at Catalyst IT. My name is Kristina Hoeppner. Today's guests are Yvonne Moore and James Pearce from the University of Bath in the UK. Their university has been using Mahara for quite a few years now, and in more recent years, in particular, with our competency framework functionality SmartEvidence. So I wanted to have a chat with both of them about their practice and learn more about it. Thank you so much, Yvonne and James, for sharing some of your work with us today.

Yvonne Moore: 0:57

It's good to be here.

Kristina Hoeppner: 0:58

Yvonne, can you please tell us what your role is at the university?

Yvonne Moore: 1:04

My title is a bit of a mouthful, so it's Digital Education Developments Lead. Essentially, my job is to look at how we might use technology to improve teaching and learning. So that can include the technology that we already have. We have Moodle, we use Mentimeter, so looking at those and seeing what features might we use more effectively for teaching and learning. But it's also looking at tools outside the university, so across the sector, that are new or gaining an interest from across the university sector, and I look at those, and I'll evaluate them, test them, try them out and see if there's any benefit using them at our university. Before I got this role, so I've been in this job about three years, before that, I was a learning technologist within the work through for the timeline. In the past, what we would same team. James and I both work in a Centre for Learning & Teaching at Bath, which has a Technology Enhanced Learning typically have, is somebody would turn up, well, 'I want to team. And I was a learning technologist. Essentially, I was the very keen one on Mahara. I was the one asking all the questions and trying to get people interested and doing presentations and all that kind of stuff. So as time's gone on, I now support some of my learning technologist colleagues in the team to use Mahara, and I get involved with James for things like updates and looking at, again, how we might develop Mahara in the ways that our academic staff might need. So my role involves lots of different things, but essentially trying to improve teaching and learning through the use of technology. do an ePortfolio. I'm starting in two weeks.' And we would go[laughs], 'Well, okay, can you pull you back a bit? And we - let's look at the timeline of actually what you need to do in order to be successful in your implementation.' Now that we have that set of guidelines, we now feel confident we can talk to staff effectively about how we could introduce this most effectively for them.

Kristina Hoeppner: 9:37

James, is your team also involved in that process? Or do you come in when Yvonne or colleagues of her want to have some changes made in the technologies that you're using?

James Pearce: 9:49

I'm there to support any technical issues there might be. Every piece of software sometimes has a little bit of difficulty to work with. Hopefully I'm going to talk about the competency framework, and there's a couple of pieces of work there that we've done to improve the workflow for staff there to actually interact with it.

Kristina Hoeppner: 10:07

Yvonne, I'd like your approach of having team guidelines for your team so that you can streamline the support and also that everybody on your team, all the learning designers and learning technologists, are knowledgeable about onboarding and what is needed, which reminds me quite a bit of the interview I've had with Shari Bowker when she worked at The University of Queensland, where she's also introduced a nice flow of activities that she's taking everybody through, and then also the DiKuLe project at Bamberg, which just finished at the end of last year, who developed a planning and decision making guide for the use of Mahara, which is quite comprehensive, and they did say that it can be pared down and localised to own needs.

Yvonne Moore: 10:56

I don't have the name of the university, but one of my colleagues, Rachel Applegate, who's a learning technologist, she has been talking to another university because they've also got a similar framework, and I think theirs was quite graphical. It might be one of the ones you've mentioned, but I'm not sure. But again, the idea would be to take that and combine it with what we've written to produce a nice graphical view of how you might approach an ePortfolio implementation.

Kristina Hoeppner: 11:18

Yeah, which takes us to what James had already alluded to, and Yvonne, you as well, the competency frameworks because that is also one of the reasons why I wanted to talk with you because you make extensive use of SmartEvidence, which is essentially our way in Mahara to work with competency frameworks, provide them directly on the site administration level so that students can plug them into their portfolios readily and have all the competencies or skills that they need to complete available, all the criteria that they need to fulfil, instead of needing to add that laboriously to their own portfolio or check against a PDF. How do you then work with competency frameworks in the student portfolios? Is that primarily in the work placements or also in the more course-based portfolios?

Yvonne Moore: 12:10

It's actually one course team. So our electrical engineers, they decided that this looked like the most effective way for them because they've got 250 plus students, and they thought this was a good way of trying to map all of the skills that the students were undertaking in their portfolios. Because we did quite a long run in with them in terms of preparation, we were able to create a basic SmartEvidence framework. So we have a basic template and a basic SmartEvidence framework that the students can take, they can copy the template, they can add the framework, and they can see how it works, so they get a very clear idea in a induction session of what the standards are, how they claim them, using the annotation block. They then move on to the full one. Now, the full one is huge [laughs].

Kristina Hoeppner: 12:49

What is huge for you? It's that 120 competencies or something like that?

Yvonne Moore: 12:53

I don't think there's that many. There probably is something like 25-30 standards. As well as the number, there's also the complexity that we have in that each assessor will probably take one or two standards, and so it becomes quite a big task in order for multiple people trying to assess the students. James can explain the things that we've done to try and work with that workflow, but what we found is, is that I think we're on our third year of the engineers using the framework is that they can continue to use it. So they found that there are benefits to doing this because the students, particularly are able to take control of claiming. They can claim the skills that they have, and then the staff member can go and assess whether they've achieved it, and I think that way around, the students aren't just handing something in and then getting some feedback. The students are actively having to claim that they've achieved a skill and then have it assessed, I think has worked really well. What we found is the complexity of it, the number of standards, the number of pages in the portfolio and the number of assessors, has led the academics to try and find workarounds to make the process a bit more user friendly, I guess. But then James has done his magic to try and improve the way that that can work for staff.

Kristina Hoeppner: 13:59

What, James, have you improved on core Mahara, which I find always fantastic to hear because Mahara is an open source project, therefore we do encourage customisations, localisations, and then in your case, also, you're contributing some things back.

James Pearce: 14:15

Yes, it is truly one of the benefits of open source software is that as users use it in a slightly different way than the core team imagined, we can make those changes and give them back to the core team, which is our plan for what we've changed. As Yvonne said, we've got some competency frameworks. I've got quite a lot of standards, and there's quite a lot of assessors there. So they were getting into a quite complicated situation of only certain assessors needed to see certain standards. And they came from a quite technical background, so they worked out this solution, which was capturing portfolio URLs by accessing the portfolios and picking out the pages that needed to be assessed by different assessors to match the standards, and although this was working for them, it was, what I would say is a little bit fragile in that if something wasn't quite in the place where they were expecting it, then it was all falling apart. So we spent some time understanding what they were needing. So we talked through that process with them, and we have made some adjustments to the standard matrix screen so that it's improved filtering, so you can filter down to just the standards that the individual assessor needs to assess. Yeah, it's just improving that interface. We also have done a small piece of work to allow the actual framework standards to be edited after they've been submitted because we were finding that although they had produced the framework and they had signed off that the standards were all perfectly fine, then they would find small little tweaks that needed to be made, and when it was already in process, it was a little bit complicated that it would then need to be just tweaked. I've added a piece of functionality to allow them to do that. It's just ease of use features that we are trying to improve, and as I said, the aim is to get them all approved by Mahara core and push back and give them back to you guys, and so everybody can hopefully benefit from them down the line.

Kristina Hoeppner: 16:00

Yeah, and those two enhancements are really, really nice because often we implement a feature in Mahara in a first phase approach. So we either have a client project or we have a community conversation and then decide, okay, yep, this is how we think it can go. And then once we release it to the rest of the community, people find different ways of using the functionality, or even if they use it in the way that we've initially envisaged it, through the actual use by students and staff on a regular basis, can we find out where there are tweaks that should be made or sometimes larger enhancements in order for it to work better. Because yes, currently in core Mahara, you could make your changes to the competency frameworks in the database, but you don't really always want to go into the database and have a developer needing to do that, but it's much more comfortable doing that on the admin interface. So your use at the university is fantastic to uncover such improvements that can be made to

James Pearce: 17:04

I would say it's one of the again, the benefits the platform. of Mahara is that it's quite flexible in the way it could be used, and so, as you say, certain users will kind of push it in a way that you weren't really expecting, but Mahara is able to cope with it, and with a few little tweaks, we've made that process a little bit easier for them.

Kristina Hoeppner: 17:20

Yeah, and for those on the podcast that haven't worked with SmartEvidence in the past, what James, Yvonne, and I have just been talking about is the visual interface, the matrix screen, which forms the first page of the portfolio, where students can and also the assessors can see in one glance which portfolio pages are ready for assessment, and then also which portfolio pages have already been looked at, where an assessment status has been given. Just to make that visual because especially if you're in a very large portfolio with 10-20 or more pages, it can be quite teachers to always go through and try to figure out whether you've already seen the portfolio or not, especially when you also have 150 plus students. And so that visual overview page is an awesome one to just see that pretty much in one glance. That was a project that we had brainstormed together with the University of Canberra in particular, and also some community members at our first Mahara Hui ever that we had held in Wellington in 2014. So it's fantastic to see how that work has then continued and is also being used around the globe. Now, Yvonne coming back to the portfolios, besides SmartEvidence, what do the portfolios look like of your students? Are they quite multimedia heavy? Or does it really also depend on the study programme?

Yvonne Moore: 18:50

I think it does, but because the ones that we've got now are skills based, a lot of them are multimedia. They're taking video of them doing something in a lab or they're taking code that they're writing and they're embedding that in the screen. So there's lots of different ways that they're describing what they're doing. The video has, I think, become very popular. Particularly, I think, the electrical engineers are using a lot of video to demonstrate what they've been doing. If they're doing an exercise, they're recording it and then trying to embed it in their portfolios. It's probably the one area where we try to get our head around is what's the best or the most effective way of doing this. We have a tool called Panopto at the university where we can record videos and we can store videos, but obviously that's an internal, institutional tool that doesn't always allow access, you know, depending on the settings, to people outside of the university, and for students, they want to kind of impress a prospective employer or placement's employer. Sometimes we have to think,'Well, where are they going to put this video in order for it to display in their portfolio?' We're also concerned when they leave the university and they want to take their portfolio with them that they've got that video recorded somewhere where they can access it. So that's an area that has increased in the last two years, where we've had to really think about our systems of what the settings can do inside our tools in order to enable the students to have access to their video recordings. The electrical engineers, they do a lot of circuitry type diagrams, so quite large images that they want to display. We've had to work very hard with the students to understand file sizes and formatting so that they can keep the file sizes down. But, but, yeah, there's a range of media being displayed in the portfolio. One of the things that we are keen to do, and we haven't yet managed, is to find some examples of portfolios that we can share publicly so students can see what good portfolios look like. One of the drawbacks we have because a lot of our portfolios are of accredited units and are used for assessment, is that we have to be careful not to display assessment material to other students. But we're trying to find examples of where we can show the types of materials that are producing good portfolios so that our students can get a good sense of what a good portfolio looks like.

Kristina Hoeppner: 20:50

Yeah, maybe through the copying faction of the portfolio and then anonymising it by putting it into another account can that be achieved. Besides making SmartEvidence available to your students, which is kind of a scaffold for they're skills based portfolios, you also briefly mentioned that you're using templates. How does that work for you?

Yvonne Moore: 21:10

Each course team creates a template for the activity that they're going to do, and we work with them to develop that. Often it's their multi page template. The portfolio has headings that guide the students through the activity. We also include instructional blocks so that the students don't have to keep going back to Moodle to find out what was I supposed to be doing in this portfolio. They can see in the page the kinds of content that they're expected to be writing about. Generally, at the beginning of an academic year, the course team will review their template, they'll make tweaks, and then we make them available at site level. We have a group inside Mahara that's just for our team, and we keep a copy of the templates in there so we can do version control. So we can see how they've changed over time, and what things that the program teams are thinking of that didn't work or this did work. We've got a little timeline of that by keeping the versions in there. But we make them available at site level, so the students, any student can see these templates. So even if it's not your course team, you could still have a look and see what templates are created elsewhere. So people can get a good idea of what a portfolio might look like.

Kristina Hoeppner: 22:09

How do your students then take on the portfolio activities through all the scaffolding and the support that your team provides, both from the learning design side as well as the technical side?

Yvonne Moore: 22:22

Generally, that's done by the teaching team. We will discuss with the teaching team, you know, what can work and what can be helpful, but the teaching team itself will induct the students into the portfolio creation. Generally, they'll put formative points in through the use of the portfolio to check in with the students and answer questions and give them feedback and so on as they go through. Then at the end of the unit, the students get to give feedback about how that worked. The course team will then go and tweak the templates for the following year, taking on the student feedback.

Kristina Hoeppner: 22:49

Is the teaching team then also the people that are providing the ongoing support to the students throughout the semester? Or do any of those come back to you?

Yvonne Moore: 22:58

They do, but what we do as a team because we're a staff facing team, not a student facing team. So what we have done is provide induction material for our teachers, but training material is there for them. They just have to tweak it based on how they're using it in their course. Even though we're not student facing, we have created that material to support our staff to do that.

Kristina Hoeppner: 23:17

Where do you then see the long term benefits of creating a portfolio for your students and also your staff. Do you also offer the option for staff to create their own portfolios?

Yvonne Moore: 23:28

Mahara is available to all staff and students. So anybody can create a portfolio. One of the things that I've been trying to encourage - we do a what's called a 'Pathways' programme for our probationary staff. So anybody who's just teaching for the first time, they'll do a fellowship with the Advanced HE which is generally what all of the staff in the UK sector will do. What I've been trying to do is to get people to keep a portfolio of their teaching practice as a means for then writing their fellowship application. We've made it available two years now. I don't have any stats on whether it's being used, or anybody has used it to bolster their application, but it's there as a template for them to take and they can build their portfolio, reflecting on their teaching practice.

Kristina Hoeppner: 24:08

And the benefits for the students?

Yvonne Moore: 24:10

For the students long term, what we hope is that the university as well. They've got that then for when they go they will then see the benefit of reflecting on their learning and reflecting on the feedback that they get about what they're learning. We found that, I think, it's the electrical engineering team that their plan was to do a two-year portfolio. So they do it in year one and year two of the degree. Recently, I answered a query where some students are doing it in year three. That's the hope is that they continue to do it and see the benefit of continually reflecting on their practice and on their learning, so that they can see what they're good at. So, you know, these are the things that I do well, but they can also see the things that they need to improve on, and they can practice that through their portfolio. into work, and they can then if they want to carry on, they can carry it on and to take stock of what they're learning in their employment as well. That's the hope.

Kristina Hoeppner: 25:06

Do you then also offer your portfolios to alumni? Or do students lose access once they finish?

Yvonne Moore: 25:13

Yeah, they lose access once their formal degree is completed. We expect them to take it with them, so we do lots of guidance about how they can export their portfolio and how can they can use it elsewhere.

Kristina Hoeppner: 25:23

James, now, from your more technical perspective, where do you see any benefits for the students creating portfolios?

James Pearce: 25:32

I don't think learning stops at the end of your degree. I think you learn for your entire career. So having a portfolio and getting used to having that as a habit of logging what you've done, what worked well, what didn't work well, that kind of reflection that portfolios provide, I think, that needs to carry on throughout your career. It's almost like teaching the students to get into that habit of keeping a track of what they're learning.

Kristina Hoeppner: 25:58

Is that then also what you wish everybody knew about portfolios that they are not just for the coursework or to complete a degree, but really for that lifelong and life wide learning?

James Pearce: 26:10

Yes, yeah. I mean, I would say that, yeah. I mean, I think so that you have to learn in today's world, in industry, you have to keep learning. If you don't, you just stand still, and eventually your skills become irrelevant. It's a lifelong process as far as I'm concerned.

Kristina Hoeppner: 26:21

Yvonne, do you want to add anything?

Yvonne Moore: 26:21

I was just going to say, I think it's that thing of you can just start now. It doesn't need to be part of an accredited unit or a degree. You can just start. It is that thing of habit, building, having a reflection of what did I learn today? How am I going to use this again? What could I do differently next time? It's that whole reflective process. If you can build it into your daily life, is actually long term is helpful.

Kristina Hoeppner: 26:48

Do your teaching teams have some guidelines in terms of how frequently they want students to reflect?

Yvonne Moore: 26:56

The ones I know about, they tend to have a weekly tutorial where the students will be reminded to work on their portfolios. It's a bit of a tricky one, where, when it's used as also an assessment activity. It could be a semester long or a year long portfolio, but students will be expected to be working on it consistently. But in terms of reminders about reflecting, I think rather than just putting something into your portfolio, but actually take some time to reflect on it, I think that's where the tutorial system helps them remember to do that aspect of it, and that tends to be weekly, I think.

Kristina Hoeppner: 27:24

And then at the end of their year, the students can look back at their portfolio that contains the SmartEvidence framework and see really all the skills that they have gained throughout the year and where they have been making progress. That now already takes us into our quickie answer round, where I'd like both of you to answer each question. And for the first one, I'll start with James. Which words or short phrases do you use to describe portfolio work?

James Pearce: 27:55

I've got a couple of phrases. A portfolio shows your progression. Been saying this time and time again that it shows your progression of your development, of your work, and your skills, and it also helps you understand the point at which you are in your education, to understand what your next steps are.

Kristina Hoeppner: 28:10

Thank you, Yvonne?

Yvonne Moore: 28:13

I'm going for just three punchy words, right? I'm going for 'reflective'. So I always think portfolios are reflective. It's that chance to think back. They're a'showcase'. It's your opportunity to show off and show what you know, what you've learned, and it's your unique opportunity to describe that. And then the last one is'journey'. It's that, how far have I come in whatever it is that you're learning about? That's it for me, reflective, showcase, journey, are the three for me.

Kristina Hoeppner: 28:36

Those are really nice bunchy words, and also are demonstrated in what you have told us about the work of the students, that they are reflecting on their work in regular intervals, they look at their skills, and then they also use it for showcasing in order to get jobs. Now Yvonne, what tip do you have for fellow learning designers or lecturers, tutors who create portfolio activities?

Yvonne Moore: 29:01

It's a really boring one. It's about putting the time in to learn the software. Because I think too many times I have conversations where, 'Oh, Mahara can't do that.' And I'm like, 'Yes, it can. It's just you haven't spent enough time with it yet.' And I think that's where the idea of our guidelines for building in a timeline for implementation that involves that time for training, I think, is really important. Sometimes, I think, with any new tool in this day and age, people think, 'Oh, I'll just be able to use it.' It's like, no, there is a learning curve. It's not a steep one with Mahara, I don't think, but there is one, and so you have to have that time to learn how to use the software before you can then think effectively about using it.

Kristina Hoeppner: 29:36

Yeah, James, what tip do you have for somebody in your role who is supporting the technology on your own server infrastructure?

James Pearce: 29:47

I would say, embrace the fact that Mahara is open source, and really look at the code and try and understand the code and understand what it can do and identify things that you know you could probably help with and change for your institution, for your academics. Those features offer them back to Mahara core because everybody can then benefit from them.

Kristina Hoeppner: 30:12

What advice do you have for portfolio authors, be that your students or your staff? Yvonne, let's start with you.

Yvonne Moore: 30:20

Yeah, for me, I think just be bold. EPortfolio is a really nice opportunity to showcase how you learn and how you want to present yourself. And I think that idea that you can personalise your portfolio to represent you, I think that's a lovely idea.

Kristina Hoeppner: 30:35

And James?

James Pearce: 30:36

It's a similar idea. But I think experiment. Consider how you can present your work. Not everything should be text. Can you provide an image that will do just as well, or image gallery or video, or even audio, anything like that. The portfolio should be able to handle all of those. So, you know, experiment, go a bit crazy.

Kristina Hoeppner: 30:55

Thank you so much, and that is a tip that has come up time and time again. It is either be bold or be brave, be yourself, experiment with it. And so it's really good to hear that reiterated, so that it does really show that this is not just one person's thinking of saying that to the students, but that it really is something universal, where we can help students personalize their portfolio, bring in their voice, and therefore really also make it their own, so that they see it not just as a tick box activity that they need in order to finish their degree, but that it is something useful also for their further studies. Thank you so much, Yvonne and James, for sharing how you at the University of Bath currently work with portfolios and work in particular with competency frameworks and templates making it easy for students to get started and also to maintain and continue working on their portfolio. So thank you so much for your time.

Yvonne Moore: 31:58

It's been lovely to chat to you.

James Pearce: 32:00

Thank you.

Kristina Hoeppner: 32:03

Now over to our listeners. What will you experiment with in your portfolio practice after today's conversation? This was 'Create. Share. Engage.' with Yvonne Moore and James Pearce. Find everything you need at podcast.mahara.org, including resources and related episodes. If you think what James and Yvonne talked about could be valuable to someone else, help a colleague discover it and send them the link to this episode. I'll be back with more in two weeks. Until then, create, share, and engage.