[powerpress]
Feminine gamification viewpoint: Interview with Margaret Burnett
Podcast 8: Where to find inspiration for gamification?
In a question of gamification this week we are honoured to have the company of Margaret Burnett distinguished professor in Computer Science at Oregon State University. She has carried out research in relation to gender inclusiveness in software use and developed a research methodology to test for potential stumbling blocks your users may face.
| An Coppens: | Hi, I’m An Coppens from Gamification Nation and I’m delighted today to have with me Margaret Burnett who is a Distinguished Professor of Computer Science at Oregon State University in the USA. Now, I’m so delighted to have her on board because her research focuses on end user programming, end user software engineering and gender issues in this type of context. Welcome to Gamification Nation today Margaret.
|
| Margaret Burnett: | Thank you, I’m delighted to be here.
|
| An Coppens: | Thank you. Now, I noticed you have Distinguished Professor as your title, how does that happen?
|
| Margaret Burnett: | Well, it’s quite an honour and I just received it this last spring so I’m still just immensely delighted about the whole thing. The university passes this honour out to 2 professors per year so I got it this last spring. The criteria have to do with the quality and the reputation of your research and your teaching and mentoring and whether you’ve really made a difference in some way. I have some awards on some of those fronts and a long history I guess that … A really long history. Anyway, so I won.
|
| An Coppens: | Fantastic, it’s a great honour to have you so shortly afterwards, we’re not just talking to a professor but a distinguished professor. I love that and congratulations also.
|
| Margaret Burnett: | Thank you.
|
| An Coppens: | Now, the reason that I came across you and the research you have done is because as my listeners and readers know is we work a lot on the feminine view of gamification and I look for research that’s been done in that field. When I came across GenderMag, I got really excited, I said, “I want to know what this is, how does this happen, how did this come about?” First of all, what is GenderMag?
|
| Margaret Burnett: | GenderMag is a method. It’s a process for software developers or UX people or anybody really with some say into how software is turning out. Anyway, they can use this method to spot aspects of their software—the software itself—that might not be as gender inclusive as they liked. GenderMag stands for Gender Inclusiveness Magnifier, so that’s what it’s all about.
|
| An Coppens: | Fantastic. How did it come about initially, is there a background story to it?
|
| Margaret Burnett: | Yes, there is, there’s quite a background story. It all started with my PhD student, Laura Beckwith who was seeking a PhD topic to work on in about 2002 or 2003. We came up together with the idea of possibly looking at how gender differences might come together with software itself. At that time, there was starting to be a fair amount of understanding in the academic community—not in industry yet—the academic community that there were gender differences that were starting to play out in the workforce and in higher education. But nobody had thought about software itself yet.
|
| An Coppens: | That’s really amazing that it actually is recognized first in the university world and then further down the line becomes of interest. I think a lot of businesses and software developers still are only starting to recognize this field.
|
| Margaret Burnett: | That’s right. Laura started reading from 5 different fields and in every case what she was trying to do is ask herself the question, “What does this tell me about software?” As you mentioned earlier, I do human aspects of software development, some of it end user programming and some of it professional programming but always the human aspects of it. At the time she was thinking about an end user programmer. So she had this end user programmer living in one part of her head, and she was reading foundational work from psychology and from education and from computer science education and from the academic study of marketing and communication and just everything she could get her hands on from the feminist literature. Every time she would run across some sort of well-established statistical gender difference, she would say, “Well, what does that say about software?”
|
| Soon, we started concentrating on software in which the user is problem solving. It’s that aspect—that use of software—that we got interested in because that affects not only end user programmers but professional programmers or anybody who comes to the computer with a problem. For example, even an accountant that’s say for example trying to figure out their budget, they’re problem solving. That’s where our foundational literature applies.
|
|
| An Coppens: | I guess that would apply to a lot of enterprise software that people are using?
|
| Margaret Burnett: | Absolutely. Also decision support software, debugging tools, or any time the user has a problem that together with the computer, they’re trying to solve. It turns out that the other problem solving situation that’s all too common is that they didn’t actually bring a problem to the computer, they sat down thinking they were going to do something ordinary but the user interface is so obtuse they end up having to do a lot of problem solving just to figure out how to use it. That’s the other situation. Anyway, Laura spent her time investigating some of the hypotheses that dropped into her lap when she read this foundational literature from other fields.
|
| What she was trying to do then is say, “Well, do these things apply to computer usage by males and females?” She found over and over again that indeed these foundational things did apply. She did many studies, did some prototype building and evaluated those empirically and found the same thing over and over again. By the time she graduated, other people were starting to do that kind of thing too and so these same findings have been expanded upon and empirically established by a lot of other people. Anyway, we spent years just establishing those foundations. Then about 3 or 4 years ago, somebody from industry found me on the web and he wrote me and he said, “We have this terrible problem. I found you on the web because of your gender research so I hope you can help me.
|
|
| Here’s the problem, we write software for a particular branch of the medical profession that happens to be mostly female.” I think he said something like, “80% of the practitioners in this branch of medicine are female.” He says, “Females tend to hate our software so we’ve got a big problem because that’s who we’re marketing to and we don’t know what to change, we don’t know what to do about it. So please help.” That’s when I realized we didn’t have anything to give him except a bunch of research papers. That’s when I realized we needed a practical method that would distil the essence of those findings into a practical form that ordinary practitioners with no background in gender difference research could apply to their own situation. That’s how GenderMag was born.
|
|
| An Coppens: | Yes, it is this story I had come across. That is just such an amazing story because first of all, for someone to stand up and say, “Look, I’m not getting the main target audience to engage with my software.” That already takes courage. Then to then actually typically develop something that works and can engage them I think is absolutely fascinating. In research and in the work you did with the medical software, were there some key points in difference between men and women and how they go about problem solving in the first place and then how that impacts in the software that they use?
|
| Margaret Burnett: | I can’t answer that one as crisply as I’d like to because I didn’t end up doing a lot of background research with that company. Instead, what we did is we just brought the method, the very first iteration of it to them and they sat down with us and tried to use it based on what they knew. They did find some things that they understood immediately to be useful and made some changes. We also found many awkward and difficult to use aspects of the method so we started iterating on it as well. Both of us came away smarter.
|
| An Coppens: | and that’s the purpose of research and testing, right?
|
| Margaret Burnett: | That’s right. Since that time, we’ve iterated continually on the method and we’ve continued to work with various companies who are using it. For some of the teams I’ve worked with, I have quite a lot of information about their populations, and for some of them I have very little information. I guess one of the key points is every single software team that’s ever used GenderMag always finds some gender inclusiveness issue—every single one has always found something.
|
| An Coppens: | That’s amazing because I suppose as a female software user, I often and actually in my gamification business, I often get requests of new apps and different items for people. I’ve often given the feedback, “If you’re aiming at the ladies, this won’t work because we’d reject it, it’s not intuitive, it’s not this …” It was really interesting to see how that could work. When you present research findings around GenderMag and around human centred design that focuses on gender specific issues, how is that received?
|
| I’ll frame the question or position the question a bit because I gave a presentation last year at the Gamification World Congress about gender differences for gamification design. It really rattled a lot of feathers and ruffled a lot of men in the audience. Some of them loved it and came up to me, “Yes, that’s absolutely what we think too.” Then I had really long debates with others as to why there is a difference or not a difference. The reception was very mixed and very much questions. Do you experience that or do you experience a different side?
|
|
| Margaret Burnett: | Well, it’s a journey. Part of it I guess is learning the right ways to talk about this. I have experienced that kind of reception, maybe less so than you because when I’ve made my presentations, it’s almost always started out as being aimed at a research audience so I’ve always shown the data. Then when I started giving it to broader audiences, I learned from that but still it had a strong data background to it and graphs and stuff. People, when they see that there’s empirical evidence, some of these objections go away. Since I’ve been presenting it, I don’t have problems with credibility, I don’t have problems with people saying, “I don’t believe it.”
|
| There have of course been philosophical debates and I think I’ve gotten better at that too. At first … The whole attitude about gender differences, let me maybe explain that a little bit. First of all, everything that we’ve done of course is about problem solving, gender differences that pertain to problem solving in software. All of it really is about individual differences—so we can’t say that women are all one way and men are all the other way. It’s just a huge distribution of individual differences and there’s a fair amount of overlap between where a lot of the men are and where a lot of the women are. But there are also these long tails on both sides that have large fractions of males and females.
|
|
| If somebody is only targeting where that overlap is then they’re missing almost half of the population, both males and females. What happens more often is probably because so many software developers are themselves male, they’re targeting where most of them (males) are, which tends to be some of the overlap, not all of it and then way over on the male side of the tail. There too you’re eliminating almost half the population but much of that half you’re eliminating are female. That’s where this lack of gender inclusiveness comes in. It isn’t because females are all alike or males are all alike—it’s because of the portion of the wide distribution that’s been addressed in the past by software.
|
|
| Once I explain it that way, people start to understand that I’m not stereotyping, binning people into gender buckets but instead just saying, disproportionately, here’s the side of the distribution that you’re eliminating. There are a lot of females over there and a reasonable number of men too but more women. There’s a branch of feminism that embraces diversity and says, “Hey, we know there are individual differences and we know that the distributions don’t entirely overlap. Our missions as feminists under that branch of feminism is to embrace diversity and support all of it.”
|
|
| That’s the kind of feminism this is. There are other branches of feminism that take the stance that there absolutely are no gender differences. Those people are sometimes offended by this. When they understand the data and the distributions, typically, they aren’t any more but they could be at the beginning. Anyway so my presentations have evolved to try to make crystal clear that this is just absolutely not about binning people by gender.
|
|
| An Coppens: | Good, I’ll have to learn that technique as well. I don’t think it has offended but I make the same excuse, I give the health warning that there are men on the feminine spectrum, there are women on the masculine spectrum. I try to explain that but when you have a 10 or a 15 minutes launch at a big event, that fell into some gap. I say that to people, I say, “Well look, there are some men that have a very feminine way of working, there are some women that have a very masculine way of working.” It’s about I suppose, like you say, it’s how the distribution on the various data graphs, which end of the spectrum are you covering the most of? Yes, I agree, I think it’s great to see the differences.
|
| Now, you developed a number of personas in the GenderMag method. How do they help and what do they bring to the process?
|
|
| Margaret Burnett: | We have 4 personas. Abby is one of them … Her most recent version has 4 pictures, there’s the main Abby, and there are 2 other female Abby pictures that are different ages, different ethnic groups, and there’s also a male picture who we would call Abe. The idea there is we’re trying to communicate even at the very beginning that Abby could look like any of those pictures. Anyway, then we have a Tim and we have 2 Pats who are identical twins. Tim has 3 pictures of a male and a female and the Pats are equal.
Anyway, these personas are developed to be the sort the developers would find usable and useful. Sometimes developers don’t like personas because some of them feel that personas are too mushy or they don’t quite understand what pieces of a persona to bring.
|
| What we did is we made sure that each persona fits on exactly one piece of paper and has bullets and bold face so that the developers can see exactly what they’re supposed to focus on. Each of these personas is based on 5 facets that came from the foundational research that has to do with problem solving. One of the facets is the personas’ motivations for sitting in the chair in the first place. Are they there using that software because they love using software or technology or maybe that type of technology or are they there because there are some particular thing they’re trying to get done? That has multiple values, you could be somewhere in the middle, it could be situation dependent, you could be more on one side or the other.
|
|
| For motivations, Abby is there to get to get something done and Tim is there because he loves technology and the Pats are somewhere in the middle. That’s one of the 5 facets that actually has strong impacts on how somebody’s going to go about a problem.
Another facet is information processing style. If you run into a problem, a lot of times what you feel you need more information and this is where information processing style comes up. One style is to be extremely iterative and this is a style that Tim embraces. What Tim will do is he’ll gather a tiny little bit of information relevant to the process and then he’ll try to act upon it knowing that he might be wrong.
|
|
| He might take one more little action and then he may feel that he needs some more information or he may realize that it’s a mistake and back track and then gather a little more and proceed in this tightly iterative style. That’s called “selective information processing”. Then the other style that we cover is called comprehensive information processing. That one is much more bursty and so it looks like this: gather, gather, gather, do, do, do then maybe gather, gather, gather some more. It’s in bigger batches. So both styles have advantages and disadvantages. The iterative style is characterized by more mistakes and somebody might forget to fix them so those mistakes might stay in there. The comprehensive style is heavier on memory and so the memory load is a disadvantage there.
|
|
| As you can see, they’re equal but from a statistical perspective, females are more likely to be on the comprehensive side and males are more likely to be on the selective side. Well, when you look at software, typically the side that gets supported is just the iterative side: you get a tiny little bit of information then you’re in an opportunity to act upon it, you might get immediate feedback about that action, you might do a tiny little bit more and so typically, the [inaudible 00:22:56] that are available for this sort of thing with arrows and backwards colouring and that sort of thing, these are all developed for iterative processors.
|
|
| An Coppens: | Does that have an impact then on the outcome of the number of problems solved or are those actually the same?
|
| Margaret Burnett: | Yes. It has a big impact because if you are, say, a comprehensive information processor but the software is not supporting that style, then you’re probably really frustrated and this frustration is interfering with your ability to focus on the problem. You might be perfectly capable of adapting yourself to that other style but it’s not your best style. Why should somebody move to a style that they’re not as good at just because the software thinks they should? What we believe is that software should support either of these styles so that it doesn’t matter what your style is, you can find a way to get the information you need in the size of chunk you need it in.
|
| That’s the problem that GenderMag helps developers find: “Look, <users> can’t do comprehensive processing on this.” Once you find that, then often times a solution starts to suggest itself in your mind because you’re thinking, “Well, then how can we allow people to do comprehensive processing if they want to? We could allow them to pin the tools tips so they could have more than one on the screen at the same time.” This is just a matter of taking down a barrier, not a matter of somehow trading off 2 styles against each other. Some of these [crosstalk 00:24:54] fixes are very easy to do.
|
|
| An Coppens: | Yes, because I’m thinking of gamification design, what I work on a lot is that on-boarding into apps and on-boarding into an experience. It definitely makes sense to me from that perspective so it’s interesting to see how that actually played out in terms of research so that’s amazing. That’s 2 facets, do you want to run through more of the different aspects of the differences or there are 3 others to look at them.
|
| Margaret Burnett: | I’ll just list the other 3. There are of course more than 5 facets in the literature but what we did was chose a big 5 because we wanted the method to be practical. We felt like people using the method can’t really deal with more than 5 while at the same time they’re trying to evaluate software. The other 3 that we selected because they were so well established are learning style, when you’re trying to learn new features or new technology ranging from tinkering to a more process oriented learning style. Computer self-efficacy, your confidence about how well you are able to deal with technology problems if they should arise, that has a big impact on your problem solving style and your ability. Finally, there’s risk aversion: if you feel that there’s some risk, do you like risk or do you tend to shy away from it? Those are the other 3 facets.
|
| An Coppens: | Yes and they all make sense in my mind, they make pure sense from some of the experiences] I’ve had but also from what I’ve seen in software world or what worked and what hasn’t worked. It’s interesting to see them come back in the research and if I use the persona material I already get an idea of how that would work. You’ve also developed a GenderMag cognitive walkthrough, how would people have to look at that or understand that because the personas in gamification design, I think most well instructed individuals will already take on board that personas are a necessity because gamification focuses a lot on user centric design and developing a user profile. Adding the persona into the process I think a lot of people in gamification will probably tend to do quite easily. How does that transfer to the cognitive walkthrough?
|
| Margaret Burnett: | The persona helps the cognitive walkthrough go well and the cognitive walkthrough helps make sure people are processing the persona thoroughly. The 2 go hand in hand. The way it works (for those who don’t know what a cognitive walkthrough) is, you bring a scenario or a use case to your software and then you walk through with some user you’re imagining clicking or dragging or doing the thing that you’re imagining in that use case, and you answer particular questions about it. For our cognitive walkthrough, you do it with the persona in front of you. Let’s say you’ve chosen to do this from Abby’s perspective. You’ve chosen Abby and Abby is sitting in front of you and you say, “Okay, we’re imagining that the first thing somebody is going to do is log in and to do that they have to click on this thing.”
|
| Then there’s a question that says, “Will Abby know what to do here?” You answer this question and you have to refer back to Abby and suggest certain facets like her computer self-efficacy or her motivations or whatever that might be particularly relevant. You answer, everybody in the room answers yes, no or maybe. At this point, you don’t have to agree because GenderMag is all about diversity. Diversity of opinions is welcome as well. Some people might say, “Yeah, sure, look at that, it’s bright red, how could she miss it?” Someone else might say, “Wait, what it says is donate your firstborn child and log in, I don’t think Abby is going to do that because she’s risk averse.” Obviously that’s a only one example.
|
|
| You record all opinions and the facets that arose and then we force Abby to click on that and we see the next screen and it says, “Does Abby know that she’s making progress toward her goal?” Once again you answer those questions and then you move on to the next step. Those 2 are part of a standard cognitive walk through, but we refined them so that they refer constantly back to the persona you’ve chosen. You could instead do the whole thing through Tim’s eyes and then you’d be finding gender inclusiveness issues that apply to Tim as opposed to Abby, and sometimes you’ll find those too. Anyway, so that’s what you do. It’s just a walk through with those personas working hand in hand.
|
|
| An Coppens: | Exactly because that’s where I see the value in it also when you’re designing and before you go into development, it’s good to have a persona in mind. Then to afterwards test it once you have developed because you might have originally told something true in your design that was absolutely amazing and then when it comes out in development you say, “Right, that doesn’t really work.” I think it’s great to combine the 2 for sure. When I have an app or a software that I would redesign, what would be your top tips to engage on one side the ladies and I’m going to add on the other side the gents or should we even that discussion?
|
| Margaret Burnett: | It’s a great discussion. I think the first tip, and this won’t come as a surprise to anybody in the software industry, is evaluate early and continuously. If you have something that’s still at the design mark-up stage and you evaluate it and you find that 80% of the features you evaluated had gender inclusiveness issues, that’s actually good news because you didn’t spend a dime implementing it and you can go back to the drawing board early. That’s the 1st tip. The 2nd tip is that I think that GenderMag is a really good thing to do but not the only good thing to do. It’s good because it’s theory based. If you use it to find something, you often know why you found it because the facet is what helped you find it. You say, “It’s all about risk averseness so what we need to do is make this …” Give a more accurate picture of the amount of risk actually entailed, so that users better know what to do.
|
| Another good thing about GenderMag is it makes it easy to order up a user with exactly the facets you want to think about. On the other hand, any inspection method like this, because it doesn’t have a real user sitting there, is always subject to the evaluation skills of the people sitting the room. They may not actually be right what Abby might do or might not do. So, it’s good to also do empirical studies with real users, both males and females. By having these 2 together, you start to understand the validity and how widespread various problems are that you’ve identified with one thing or the other.
|
|
| A lot of times when people gather users for the empirical stuff, they neglect the women because they’re hard to find for some kinds of software. This of course is going to give you a very biased view of how users view your software. In fact, one conversation I had, somebody was working on a debugging tool I think it was and they brought in 5 users and I said, “Well, how many were female?” The response was, “Well only 1 because they’re really hard to find.” He had been telling me about how so many of these users had asked for a certain feature. I said, “Did she ask for that feature?” He said, “Well no she didn’t but …” Here’s the key point, he said, “But you can’t really generalize from her because there’s only one of her.” Basically what he’s saying is, “We will pay attention to our female users only if they agree with the males.”
|
|
| An Coppens: | Right, because that’s exactly the bias that we do need to overcome in some way. It’s actually totally congruent with one of the tips I gave last year is to actually have different sets of user test groups. Have a mixed group, have an all male and all female group. In some cultures, because I do a lot of work around the Middle East, in some of those cultures, you do need to have women together with women and a female facilitator to get the best answers. When in a situation where there’s a male facilitator or a mixed group, they may not actually share anything. I made that example very clear that there’s gender, there’s culture, there’s a lot of social facets to actually take on board.
|
| It’s fun to see that it’s construe as one of your tips too. It’s great that we’re aligned on that one. Is there any research or do you have views on how games may apply some of your knowledge or some of these processes? What I see in games and in gamification is that they’re often very competitive and very I suppose male driven. Are those findings similar to what’s there in the software industry at large or is that maybe just more game specific?
|
|
| Margaret Burnett: | It’s a good question. I think that nowadays since they’re gamifying so many things, things that happen in games tend to start to reach over into other kinds of software as well. I don’t have any findings on games and gamification itself yet, because we’ve only been doing field studies with real-world software teams for about a year and a half. My findings continue to just emerge. I would love to find out how well it works with games and gamification because on the one hand, games and gamification are all about problem solving which seems to make it a perfect setting for GenderMag but on the other hand, part of GenderMag is about facilitating people’s problem solving styles in a way that makes it easier for them to solve the problems. But games aren’t fun if they’re too easy.
|
| There’s a kind of tension that makes it a very interesting setting to try to investigate and see how well GenderMag does, but one thing I can say is for the user interface around the game, the part that’s not supposed to be fun, it’s just supposed to be there. There of course GenderMag would completely apply. But for the games themselves, I’m just as curious as you are about that.
|
|
| An Coppens: | Great. I definitely would love to do some more developing and researching in this area to see what can transfer because as you say, most gamification tools are a problem solving tool in some sorts to get somebody to do something a certain way. Often enterprise applications, you probably have already some research on. That leads me nicely into my final question, where can people find out about your research, where can people get the methods or GenderMag so that they can make sure all of our software has become better and that both games and gamification take this on board as well?
|
| Margaret Burnett: | Yes. GenderMag is freely downloadable from my webpage https://eusesconsortium.org/gender/
|
| Margaret Burnett: | Very good, okay. Anyway, freely downloadable. We continue to iterate and update and understand how well GenderMag is working for various people. So if somebody does download it, they might want to check back in a couple of months and get a newer version. If you use it and something good happen—and by you I’m talking to everybody who’s listening—I hope you’ll let me know. Or even if something bad happens as a result of using it, I hope you’ll let me know because it’s active research for us. We’re trying to learn everything that we can about it.
|
| An Coppens: | Fantastic. What’s the best way to do that, just reach out to you on the same link as the website?
|
| Margaret Burnett: | Yes, my email address is on my website and so you can get back to us that way.
|
| An Coppens: | Super. That is amazing so thank you so much Margaret, I really appreciate all the knowledge you’ve shared. I found it extremely interesting and enlightening in a lot of ways to hear what it is how it works. I’m very sure that the people listening in will think the same as well.
|
| Margaret Burnett: | Thank you so much for contacting me, it’s been a pleasure …
|
