Kristina Hoeppner 00:05

Welcome to 'Create. Share. Engage.' This is the podcast about portfolios for learning and more for educators, learning designers, and managers keen on integrating portfolios with their education and professional development practices. 'Create. Share. Engage.' is brought to you by the Mahara team at Catalyst IT. My name is Kristina Hoeppner. 

I'm delighted to be talking with Dr. John Ittelson today, Professor Emeritus at California State University, Monterey Bay. John has been in the portfolio world for many, many years. Welcome to Wellington. You are sitting across from me about 12 years after you visited here when I met you for the first time, John, It's wonderful to see you again in person.

John Ittelson 00:53

It's great to be here. It's sort of fun to be talking about ePortfolios. You mentioned, I'm a professor emeritus, which technically means I'm retired, but my wife claims I'm failing retirement. So I'm not as active in the portfolio community, but I'm definitely active in keeping track of it and seeing the trends.

You know, I've been involved in this space since I got connected with the project at EDUCAUSE. It was then called NILL. The EDUCAUSE set up a group to look at how technology could help improve higher education. And they did a lot of different initiatives, communities of practice. They were essentially looking at, you know, how can the technologies that used to be just primarily like bookkeeping, and grades at institutions, and really get closer to the students.

For that fellowship, I took a part time leave from the university to work with a group of people. And my first research project with them was really looking at student identity and what does it mean, your digital identity. That really opened up the whole ePortfolio world to me, and that really became an area of research and interest for, you know, the last part of my career at the university.

Kristina Hoeppner 02:00

So from information technology to student identity and then to ePortfolios. Looking at your LinkedIn, there's also TV included, broadcasting, and I also know that you like to dabble in technology a lot and have all the latest gadgets and possibilities of what one could use.

John Ittelson 02:19

Yeah, the funny thing about gadgets and portfolios, Helen Barrett, who's also very active in the ePortfolio community, when we would meet, we would spend a part of our time talking about ePortfolio and part of the time talking about the new gadgets and technology that we had. Yes, I had an interesting career in the sense that I started my advanced degree work - when I got out of the Navy, I did a degree in instructional television, thinking I got a job in public broadcasting, but that ended up sort of moving and I ended up getting a degree in Industrial Engineering, but my focus was the use of technology in education and training.

So in many ways, the world's caught up with me. One of my first projects as a PhD candidate was working on video conferencing. So I did a video conference in 1977, between the Northwestern Medical School and a ship at sea between O'ahu with slow scan video via satellite. So the Zoom connections we have now are like lightning compared to then. But I was always interested in media and teaching, and that's why I first got interested in public broadcasting.

But what really - I was thinking about this when you said, you know, you'd like to interview me, you know, I know I got started in portfolios when I had the fellowship, and I was also working with a teacher credentialing programme in the state of California. That teacher credentialing programme had a paper based portfolio. So it was a state wide, delivered electronically, site based teacher credentialing programme, and so we were trying to move that paper portfolio to a digital portfolio. We were using a lot of technology in terms of providing our instruction. 

We were one of the original users of the course management system, WebCT, and that all sort of merged together, and I was trying to think, well, how did I end up you know, really being this interested in the use of electronic portfolios? And it dawned on me that it really started much earlier than that. I don't think I've ever - this is a first. You're getting up close and personal. In the sixth grade, I was tested - the school test every sixth grader in the States, and I was tested and determined I had a learning disability. So I was a slow learner, and they put me into the slow learning classes. I do have a form of dyslexia. I don't call it a disability because I think people who spell the word the same way every time are not creative, but my dyslexia really did impact me academically and primarily because the standardised test and the normal ways of fill in the bubble forms and short essays I couldn't excel at.

So I realised that during my elementary school, middle school, and high school, I was constantly looking for alternative ways to demonstrate my skills. And I think that's why I got interested in broadcasting because broadcasting and media played to my strengths and not my weaknesses.

I fast forward when I think about the work that I did in portfolios, but when I say 'I' should say 'we' because it really was a tribe. There was a whole community. We had the EPAC community, we had the NILL fellows. But we were trying to figure out how to help students document their learning and giving the broadest ability of students to show their unique capabilities, help teachers come up with better ways of being able to not so much document the student work, title of the book that Helen and Tracy and I wrote - 'Documenting learning' - but really to help students take control of their learning.

Kristina Hoeppner 05:42

You mentioned that the project you were involved in was an early form of the electronic portfolio. What did that actually look like at the time because the internet would have still been in its infancy?

John Ittelson 05:55

Yeah, this was very early on. In fact, we had some of our faculty who were very traditional, the students would send them electronic copies of their paper, and they would still print them out to correct them, and then they would mail them back to them with the corrections. And the videos that we had, we had CD-ROMs that we sent out because the internet didn't have enough bandwidth for video. So in the course management system, we would provide an advance organiser give an assignment, and then they would have to plug in a CD to be able to watch the video that we wanted to look to. Yeah, it's sort of hard to imagine how quickly that changed because we went from, you know, almost no video to not that many years later, we were using Zoom back in 2012, but for remote observation, and students being able to record videos of their teaching and also be able to record reflections.

So a lot of the teacher credentialing programmes had a formal portfolio process. That was the final assessment. And these ended up being three inch binders, you know, with paper protectors, and this was a elementary credentialing. So it had, you know, crayon drawings [laughs], worksheets that the kids had. The process, not only was probably not very effective, but clearly wasn't going to work as we built out the programme.

So as we're developing this site based online programme, we decided to review a certain percent of the portfolios, and the portfolios were from students all across the state. So we did 10% of them, and they fit in the back of, you know, a couple banker boxes of the three inch binders. We had a faculty team analyse them and learned a lot because we were norming the assessment, and we would find things that some of our students didn't do well on. And if we found out so many of them didn't do well on it, we realised it wasn't a student's problem, it was our instructional problem. This was such a great idea of this grading collectively that the thought was 'Oh well, we should have the teachers who are working with one set of students grade the portfolios of the others.' And so the idea was there would be this system wide grading of student portfolios so the faculty who were the mentor for our students would not assess their portfolio, it'd be another faculty member and vice versa. And then we started talking about the logistics was we'd have to have a truck driving around carrying all these binders and...

Kristina Hoeppner 08:18

somebody just doing the job of driving portfolios around [laughs].

John Ittelson 08:21

And the other thing had happened is that because the whole portfolio thinking process wasn't really as refined, the students looked at this portfolio as 'Let me stick every piece of evidence that I have and hope the faculty member finds enough of it to pass me in the course.' And so we really started developing some of the, you know, what are the key principles, developing ourselves, but really, it was just reinventing the wheel. The key principles that people who've been active in the portfolio business had been around for a long time is letting students select their materials, but the selection materials and refinement and deciding what is your best example and making it not be a three inch binder, but really a very small electronic document that really represents the best work for the faculty members to assess. But the whole gathering process and reflecting process was good to have a three inch binder because students could see their work from the beginning of the process, halfway through the process, at the end.

As important as it is for our faculty to be able to assess the students' learning, it's really important for students to understand their path because they're not always going to have a faculty guide. So seeing their work as they started, we had teachers who would give their career goals and their teaching philosophy as part of the documentation at the beginning of the course, and we had them redo that at the end of the course and then reflect upon the changes, what happened during the two years of their credentialing programme.

Kristina Hoeppner 09:44

That is definitely also a very different process than the end of term final assessment. The multiple choice assessment or writing the essay because with a portfolio, you can document your learning throughout the entire semester and then go back to individual points of what you had learned. With your own personal background then, you mentioned that it influenced you and that it was probably one of the albeit more unconscious ways of how you ended up in portfolios or became really excited about portfolios, have you done any research actually on looking at these alternative ways of assessment and how they help students and how the portfolio can help them?

John Ittelson 10:29

My first 17 years at the university was teaching at California State University Chico or Chico State. That's when I taught some of the broadcasting but he also taught in our instructional technology programme. When I moved to Monterey when the campus opened in 1995, that was an interdisciplinary department. Essentially, we had computer science, we had graphic design, we had instructional design, all in one entity, and that campus also was going to be an outcomes based campus. We were going to try to do authentic assessment and even question whether we'd even have grades. And so the whole campus community was really trying to figure out how do you come up with authentic assessment that gives the students the greatest range of ability to show their skills. One of the goals was to serve traditionally underserved populations. In fact, even now I think, we're probably one of the campuses that graduates the most first generation students with bachelor's degrees in the California State University system.

I think some of the things that influenced me is - not to complain about Chico, but at Chico when I taught there, there were a set of core courses, and the faculty would rotate through these courses. They were sort of our large lecture courses that everybody had to take. You have to have some of those large courses so you get some of the small ones. I drew the short straw, had to teach it that term, and at the end of the term, we had the students do final paper, and as is typical, you know, they turned it in right at the end of the semester, you graded over the break, and then you leave them out at your front office so that the students could come pick them up. Students didn't come pick them up. They were just sitting there. 

This is very frustrating. I'd gone through and graded them and commented, it was part of the final project, and I was concerned - I went to the other faculty member who'd been teaching it many years before I got it, the last three years, in fact, and he said, 'Oh yeah, most of the students don't come back and get their papers.' I said, 'What? This was the capstone of their class, and they don't want to come back and get it and worse, they don't want to get the wonderful feedback that a faculty member gave them?'

That got me really focused on making sure any assignment that I gave a student was one that they had enough personal commitment to that they'd want to continue it beyond the class, or at least share it, if not only with the professor, but share it with their fellow students. So that I think influenced a lot of what I was doing with the portfolio research that I did with my own classes.

Part of building a portfolio is having the experiences, the assignments, and the activities that have you have something to put into portfolio? How do you make that being a meaningful assignment for the student? How do you get it to meet the educational objectives that you have? How do you make it broad enough that a wider range of students could excel at it?

Kristina Hoeppner 13:09

And also, then how can you learn from that feedback... 

John Ittelson 13:12

Definitely. 

Kristina Hoeppner 13:12

... that you get. Did you also involve students in giving feedback to each other?

John Ittelson 13:17

Yes, yes. It's interesting because other faculty members in the whole academy sort of trained students how to behave. The typical behaviour is if students do a class presentation or something, the students all say all the great things about it, which is good, you want to get positive reinforcement, but we really haven't been good at helping students learn how to critique other people's work. And I say critique, not criticise, but you know, to give good feedback. The whole idea is that you want people to want the feedback. You want to get authentic feedback, but maybe you don't want it 100%. And how do you do that? And you have to teach that. That's part of what worked well in the portfolio processing experiments that we did in that if you teach students how to critique each other, then they are better at hearing the critiques to themselves.

And so essentially, what I would do with my courses is start with very simple and easy things. First, make sure that there's something that's being shared and talked about that was probably low stakes for the student, but it was rich enough that there could be good comments and feedback so that as you get further into your course, when you're getting to more serious critiquing, was the script that you wrote really a good script, or are the graphics really visible in the production that you're doing? We did a lot of computer based instruction. Were the activities that the students in your computer design appropriate? Those are much meatier and there's not a right or wrong answer. And so you really want people to get more in depth critiquing, and you had to sort of help students build up to that.

Kristina Hoeppner 14:57

John, you mentioned the book 'Documenting learning' already that you, Helen Chen, and also Tracy Penny Light wrote together that came out now in 2012. So it's been on the shelf a bit, but it's actually not really been on the shelf because it is still quite relevant. You mentioned earlier while we were out on a bike ride that it actually still being referenced quite a bit, even a decade after. Why do you think that's the case?

John Ittelson 15:26

Well, yeah, Helen and I were talking recently. We were surprised because Helen and Tracy and I share the royalties, and so I think we've had more than one dinner based on it. I mean, it's not a great money maker, but the fact that it's still being sold means that there are still people buying it and using it. I think part of what made that book successful was that it really was more philosophy than, you know, particular techniques or things tied to particular hardware at any particular time. It was identifying the stakeholders, identifying what the folio thinking process is, and then the people who gave examples of what they were doing were from a wide range of disciplines. I think the content is still relevant. I look back to some of my old books in broadcasting and other areas where the hardware, the technology, the field has just shifted, but in many ways, the folio thinking and the portfolios are probably more relevant and the content in there is still very relevant.

I think back when I first met Helen...

Kristina Hoeppner 16:28

That is Helen Chen.

John Ittelson 16:29

Oh, yeah. There's also Helen Barrett. Both Helens but particularly Helen Chen and what she was doing with her engineering students. At that time, it was trying to be rich media portfolios. So they had cameras for recording video. They had audio recorders for recording audio. They had still cameras for documenting images. They had scanners so they could scan student work. And the whole idea was that the portfolio would be this rich portfolio of multimedia. In those early days, that wasn't the case. Some students were good at it, but most of the students were still pretty much paper and pencil, paper and keyboard, and text journaling. Fast forward now to 2023 that video camera, that still camera, that audio recorder, that scanning of digital images, that's all in everybody's phone.

Pre COVID getting somebody to do a video reflection would require, you know, really teaching people how to do it. Now you say, 'Do a Zoom call to yourself, record it, and send it to me.' So I think the ideas that were in 'Documenting learning' was trying to you know, talk about, figuring out who the stakeholders were, figuring out how the students can gather evidence about what they're learning, and do it in - back to my learning disability where writing long journal entries weren't going to work for me because spelling was a foreign subject to me, and filling in a bubble test wasn't gonna work as I'm dyslexic. I'd even know the right answer and fill in the wrong bubble. Having more ways of documenting the learning and then sharing it.

The sharing I think was critical, and that's when I talked about that where the students didn't pick up their papers. If they weren't even interested in picking it up themselves, they clearly weren't interested in sharing with others. And I found when students knew their work was not only to be seen by the professor, but seen by their fellow students, but even it was going to be more public, the commitment to that work becomes so much greater, and if it's going to be that public, then it has to be pretty good. Students rose to the occasion.

Kristina Hoeppner 18:28

You did mention the different media that can be used and are now much more accessible for us because typically all you need as a phone in order to do a video reflection or an audio reflecting and then upload that into your portfolio. Sometimes I found that instructors were quite hesitant in having those multimedia reflections because they thought that it is much more work to assess it. Is that your experience or how did you work with the many different formats that you had received?

John Ittelson 19:01

It could be more work. It takes longer to listen and although I found it is easier for me to give a video feedback than it was for me to critique a paper with the famous red pen. But I think when I was working with some of the faculty members in our teacher credentialing programme and even on my own campus when people said, 'Well gee, I don't really have the time to listen to hours and hours of self reflection,' this sort of designing how do you help the students select what they want to give to you? So I did a graduate course which was an experiment. It was back in the days, a short period when Adobe was trying to look at creating a portfolio process within the Adobe Acrobat.

So in this graduate course, essentially, I didn't use any course management system. Everything was based around the portfolio and Acrobat document. And it started with a PDF that I sent to the students that said, you know, 'here's this course syllabus, here's the course objectives,' and I said 'the first part of your portfolio is to write a reflection on your objectives compared to the objectives I have in the course,' put it in resume - was very descriptive. You know, I want this, this, this, and this to be in your portfolio. The way we do this class is I would send them my Adobe portfolio within blank pages, and then they would fill in dropping in their resume or a reflection or an assignment, and over the course of the 15 weeks, exchanging this every two weeks, my instructions became less formal, and the ownership of what went into the portfolio became more of the student's responsibility.

So by the second round of these, they had to do their first draft of a design document that I had the chapter, stuff that they had to have, and essentially describe what that document should look like. In the next exchange, they were beginning to try to implement that document. The last exchange in this Acrobat portfolio was I just said, 'beginning of term, I gave my objectives for the course and what the syllabus said, I asked you to reflect on how those objectives met with your career goals, and you had to define a project.' And I said, 'now, rather than saying, you need to do this, this, and this,' I said, 'show me something, and let me know how you met your objectives in mind.'

Someone said 'this is frustrating because up to now, you've been telling me what to do. Now, you're just saying you've got a blank canvas?' I said, 'Yep, that's it.' And I said, 'more importantly, it has to fit within - I forget how many pages and so many minutes.' So essentially, they had to figure out how they're going to condense it. It could take more time for a faculty member to do this work if they don't have the students constraint. But that's good, because students have to really reflect the what are the three artefacts that really represent that I've obtained the goals of this course. They're doing that selection. So you're not going through 30 or 40 of them. You go, okay, these three are there. And typically, the students really pick their best work.

The hardest thing I found is that one time I asked the students to turn in what they considered their first draft and their final draft of work. And there was great resistance to showing their first draft. They said 'it's horrible, it's so bad. I don't want to put it in the portfolio.' And I said 'to me, and it should be to you, is that's probably more important to show this is where I started, this is where I ended. If I were an employer, and I saw a great deal of improvement, that would be more exciting to me as a potential hire than somebody who spent 12 weeks in the unit and didn't see much improvement of what they're doing.

Kristina Hoeppner 22:28

Because you can see the learning process. 

John Ittelson 22:30

See them develop over time.

Kristina Hoeppner 22:32

With your activity with the PDFs, I think you can see also really nicely the folio thinking process, the selecting the organising, the curating, and then the connecting of all of the artefacts that are coming together and the reflection in order to tell your story, and that's kind of what I think now we are getting back to that point where it is more about telling your story and supporting students and telling their own story, telling their learning story, and making it accessible to others.

John Ittelson 23:01

Thinking back in the years that I've been working with this, I think all of us who've been in the portfolio business are a little bit like evangelists. 'Why isn't everybody in the world doing this?' You do it and it works so well, you know, why haven't everybody adopted it? And I think part of it is there's other traditions and other disciplines, and there's the accreditation, and there's the model - traditional of you know, you got to assessment and the testing models that we've had in college board's and the like.

But I really think the experience that we've had during the shutdown with COVID and more uses of electronic technology and a lot of reflection on what it means to have a college class, what it means to go to school, what does the diploma mean? I think there's a lot of self reflection now of students deciding is the university for me, is the time and the money worth the value? Faculty members thinking about where does learning really happen? I mean, I think sometimes faculty go, 'Okay, I have a 12 week course, and we meet three days a week, and it's for an hour a day, and that's the course. Where, you know, the learning doesn't take place in that time, and we're talking about the flipped classroom.

So I'm thinking there's probably going to be more interest in folio thinking because of what happened during COVID which increased the technical literacy and accessibility of a lot of things that you know, Helen very early on took a lot of different hardware and was difficult to do. Now a lot of the documenting of learning can be done with a ubiquitous device, your your mobile phone, which worldwide most of the stuff I was talking about was happening in the States where there is most of my work.

But there are very interesting portfolio movements in Europe, and particularly in Africa, where there really aren't enough qualified academics to teach the number of students who are there. So thinking of ways that electronic portfolios can help not only to documenting learning, but it really helps students take charge of their own learning. That's probably the part that I see the greatest potential for portfolios is still, it maybe hasn't been the revolution that a lot of us in the community have thought, but still the opportunity there to really revolutionise how students learn and how teachers teach through folio thinking method. I think it's going to happen.

Kristina Hoeppner 25:13

Do you see any other potentials for how portfolios can influence learning or teaching or career opportunities for students or how employers also in the workplace operate?

John Ittelson 25:27

Well, I think when I was looking at an assignment to put in portfolio, again, trying to make it engaging for the students, something that I think the students would want to show and use, it was always, you know, typically career oriented. Could you use some of the work from your portfolio as an example to get employment?

Probably my big success story in this was a very talented young woman who really wanted to get into multimedia poetry and dance of all things. The only class where we were doing multimedia was my instructional technology class. The students created a multimedia educational project. This student wanted to get access to the lab and the equipment. I said, 'well, that's fine, but you're gonna have to create the same design document and create all the stuff that the students are doing to create a multimedia project.' And there was a little bit of resistance, but she went ahead and created a design document. It was around this multimedia art performance that she was doing. And it became part of her portfolio. Then when she graduated, her parents who were sure she would never find a job, but she took that design document from her portfolio and got a job at a software firm and did very successfully. 

I think the interest of employers to see actual work that the students have done has increased more than when we first started talking about it, and particularly on our campus where we have a computer science type programme, they're not really concerned about 'did you get an A in your C+ coding?' They would, you know, 'here's a problem, code it,' and sometimes they'll do it as a test. But the ability for a student to, you know, show an actual project from start to finish with a documentation has helped. I think it's probably true in every profession that if we really think about it, there are certain documents or certain processes that can fit into portfolio that should help people. The challenge in anything is getting the right person for the right position.

Kristina Hoeppner27:26

Yeah. I find you see that much more easily when you have an example or a couple of examples and then also the reflection going with it, knowing how they improved rather than just seeing a grade.

John Ittelson 27:38

There were a lot of vendors of the different portfolio systems. Two that are really very active now. Of course, what you've got going in the open source market and then PebblePad and Digication. A lot of other ones have gone away. It shows that those concepts of giving the student a place to put their reflections, but their work, having a place where faculty and students could comment on each other's work, having a way of being able to present it publicly and not only internally within the institution, but publicly outside of it that has stayed consistent throughout the whole sort of portfolio movement.

You know, there's been certain things that have been consistent in the folio thinking that I think we've got the AAEEBL community that has done research and journals, we have a lot of the accreditation agencies looking at institutional portfolios. Our teacher credentialing programme still does a portfolio analysis, not only of the students' portfolio for their assessment, but then we look at all the students' portfolios for assessing how well the programme is doing. It's well established. Again, it just like it could be used more broadly throughout all academic and probably should be used in businesses and industry training also.

Kristina Hoeppner 28:51

That is a good point to stop and go to our three quick answer round questions, John.

John Ittelson 28:58

Well, asking a professor to give three quick questions is a dangerous step, but I'll try to be quick.

Kristina Hoeppner 29:03

Let's see how well we do. The first question is 'Which three words, if possible, or three short phrases, would you use to describe portfolio work?'

John Ittelson 29:14

Three words...

Kristina Hoeppner 29:16

or short phrases. I was reminded by somebody on social media that [laughs] a particular phrase never came up. [both laugh] Because I usually only ask for three words. So yeah.

John Ittelson 29:26

When I think about portfolios, I think about collecting, sharing, and presenting.

Kristina Hoeppner 29:32

What tip do you have for learning designers or instructors creating portfolio activities?

John Ittelson 29:40

Too many faculty members overthink it at times, and I think the tip for someone thinking about a portfolio is 'see it from the student's eyes.' The student perspective, how they see what you're asking them to do is key. If you could make the students see the value of what they're doing, it makes everything easier in the portfolio process.

Kristina Hoeppner 30:01

What is your tip for portfolio authors, for our learners, for our students? 

John Ittelson 30:06

For our learners? Don't be afraid to extend your reach.

Kristina Hoeppner 30:11

Reach in what sense? 

John Ittelson 30:13

I think many times students are trying to figure out what does the faculty member want or what's the right answer? And I'm saying extend your reach is don't just go for what you think is necessarily or wanted, but what you want to do. That's I said your reach.

Kristina Hoeppner 30:28

Because it is your portfolio. 

John Ittelson 30:30

That's right. 

Kristina Hoeppner 30:31

Thank you, John. It was wonderful to hear a number of the anecdotes from the past. It's great to hear those stories and learn from those as well and see how they have influenced what we are reading in books and reading in articles or hearing at conferences. So all of those stories that you've collected over the years, thank you so much for sharing some of them.

Now over to our listeners. What do you want to try in your own portfolio practice?

This was 'Create. Share. Engage.' with John Ittelson. Head to our website podcast.mahara.org where you can find links and the transcript for this episode. This podcast is produced by Catalyst IT, and I'm your host Kristina Hoeppner, Project Lead and Product Manager of the portfolio platform Mahara. Our next podcast will air in two weeks. I hope you'll listen again and tell a colleague about our podcast so they can subscribe. Until then, create, share, and engage.