Making Reality Easier Using Virtual Reality Part 1


Podcast Episode 55. The use of virtual reality and virtual worlds can be used to change the future for our young people and just how this is happening is the topic of this episode.My guest this week is Dr. Nigel Newbutt who is researching how virtual reality and virtual worlds can be used to help young people with additional needs access and manage real-world situations that they often find overwhelming.

Nigel explains the work he is doing to develop programs which can be used to help develop employment skills, prepare young people for social situations and provide a way to rehearse situations which could otherwise be overwhelming without a prior opportunity to experience what the encounter might be like.

This is the first of a two-part episode and in the next episode, Nigel talks about what the future of virtual reality might look like and also addresses some the concerns that people have around using these types of technology.

The work being done by Nigel is helping make these technologies accessible and relevant to our young people enabling them to develop specific skills in a controlled way which can then hopefully be translated to make their real-world encounters much easier.

Show Full Transcript
Podcast Transcript

DEBRA: Welcome to episode 55 of the Journey Skills podcast. This week, it’s all about technology and the role it will play in our young people’s lives in terms of helping them as they strive for independence. This is a two-part episode so this week is really about how virtual worlds and virtual reality work and some of the applications it can be used for. Part two will focus on the future of the technology and also address some of the concerns around using technology.

I’m talking to Dr. Nigel Newbutt who’s been working on how virtual worlds and virtual reality could be used to help our young people navigate the real world a bit easier. Nigel talks about how these technologies can help young people with autism but actually as you listen, you will realize that what he is talking about is transferable to any of our young people regardless of their additional need. I certainly know my daughter would benefit from some of the applications of virtual reality particularly around employability.

Nigel explains the work he’s doing to create scenarios that help young people navigate the real world. He also talks about the difference between virtual worlds and virtual reality partly because I’ll be honest, I didn’t clearly understand how they differed. This is also not an area I know an awful lot about, I suspect in part because I have daughters so video games, haven’t been such a big part of their lives. But I hope you’ll find what Nigel is doing as interesting as I did. It does seem that this type of technology will fundamentally change the way we help our young people get ready for the challenges of everyday life.

DEBRA: Today I’m talking to Dr. Nigel Newbutt who is a senior lecturer and researcher in Digital Education at UWE Bristol in England. Welcome, Nigel.

NIGEL: Thank you. Thank you very much for having me.

DEBRA: Can you tell me a little bit about yourself and also about the projects that you are involved in?

NIGEL: Yes. So, I arrived at this field of study all this sort of area or practice through having undertaken an undergraduate degree in Digital Media then I linked that with a qualification in education and then as I moved into PhD work, I could start to see the connection between digital media and education.

And it isn’t long when you start researching those two areas that you come autism and particularly the role that technologies can play in the lives of autistic people. My PhD work then focused on the use of virtual worlds and the use of digital technologies to help support autistic people. I was particularly interested in working with younger people with autism in classrooms and better understanding how autistic people engage with technology and the way that they represented themselves actually through that technology.

And then through that work, looking at where technologies have moved and where they’ve developed and where they revolved, it then wasn’t long that that work connected with the use of virtual reality and head-mounted displays. And we’ve been investigating those technologies used by autistic groups because we feel that there’s a huge potential in the opportunity to engage with those technologies for autistic groups for cognitive pressures for them to feel more comfortable, more able to do things at their own pace and in their own time own using those technologies. So, what we started to do was look at those technologies, both virtual worlds and virtual reality as a place to test social skills, to develop opportunities like routes to employment and more recently looking at how they can be integrated in schools as well with much younger groups. So that’s a snapshot, I guess, of where my work has come from and where it’s moving to.

DEBRA: So, could you make me talk me a scenario that you might have on one of those virtual worlds?

NIGEL: Yes, so we created a range of things and my PhD work for example, we created what’s called an Island which the students in the classroom could go into, digitally through a laptop and they could represent themselves by something called an Avatar or character in that game, if you like. We built a range of different things in collaboration with the students and the teachers including a fairground. So, we then encouraged them to think about going to the fairgrounds and they might do in the real world but through the Avatar and start to do that with a partner in the classroom. But again, do it virtually through the computer screen.

And what we want them to do was kind of engage with social conventions, queue in to take their turn, talking to each other, interacting socially. We wanted to give them a space virtually where they could do that because of the reasons we know that they feel comfortable, they can do things at their own pace, they can think about what they’re doing. So, giving them the chance kind of explore ways that they might do that through their character in the virtual world.

So that’s really one of the things we did in that piece of work but then that led into future work that we’ve done where we’ve actually built, again in a virtual world, we’ve built virtual job centers. So, we worked with the UK Department of Work and Pensions to actually create a virtual job center. Identifying that in some cases, job centers can be really daunting and difficult places for autistic people to potentially engage with. And as they can sometimes provide the gateway to employment opportunities for many people by building a virtual version of that, somebody with autism can sit at home, they could access all the same services that they can access by going there physically but through a virtual world.

So, in other words, building on the idea that in some cases, people feel more comfortable in a virtual space than they do in the real space and identifying that having to go to a physical space which is the only place where you can access the services can be quite a barrier. So, we wanted to reduce that barrier by creating a virtual version. We have had some good success stories in so far as the people who have accessed that virtually where they wouldn’t do in the physical world had have then gone on to find routes to employment. So, we found that a useful way of applying some of that technology.

DEBRA: Do you put challenges in their way as well? You mentioned queuing before and obviously that can be frustrating for many of us, but do you put other challenges in that virtual job center for them to work their way through it?

NIGEL: Well only in so far that they would need to navigate their way to the job center through a range of different pathways and roads and things like that. So potentially, yes, things like crossing roads or navigating your way through a space. There would be some of the virtual characters in that space so they might have opportunities for some social interactions. But what we are really trying to do is strip it down. In the first case anyway, strip down all those complications that one might typically interface with in the real world and really minimize them and reduce them.

I mean of course the affordance of this technology is that we can increase those stimuli so we can make the environment noisier, more populated but we haven’t experimented on that at this point, actually. What we’ve done is strip it down to its bare essentials essentially. When there might be a couple of other people there and that there would be somebody behind a desk that they would sit down in front of talk to and engage and access the services that they need at that point.

So really, the focus is about stripping back some of that stuff at this point but the work that we did with the school that you mentioned about queuing, turn-taking, going for coffee, you know, we try to simplify it as much as we could at this stage. I think as we move forward, the importance of this particular technology really lies in our ability to control it and the adapt it to different users. So, some users, we found really struggled with having more than one or two people in the environment and others found it fine. So, I think, it’s finding the happy medium between those things with the different types of users that we’re working with.

DEBRA: Is the idea that you would add in people, noise based on the individual needs and as they get more and more comfortable with the actual game?

NIGEL: You are absolutely right. That’s one of the keys. We’re currently working with the school. The school has a coffee shop attached to it and part of that is, so the pupils have a chance to work, to do some real-life work. So, what we’ve done is we’ve used the virtual reality version of coffee shop that takes them through the process of making a coffee; so, what they would need to do to make the coffee to help as part of that training. And what the students really like about that environment is they can learn to do that without anybody else in the room at all so they can really focus on the activity, so they can focus on the actual order that they need to do things in: they need to grind the coffee beans, put them into a dispenser, steam the beans, put the cup in the right place, steam the milk, that sort of stuff.

So, what they fed back to us is that they really liked the idea that they can rehearse all those things in an environment that doesn’t overstimulate them. We’ve only just started this piece of work but as they become comfortable doing that, we can add more noise, more people, more environmental stimuli into that environment. What’s really nice is we can control that so I think that’s what the technology in many ways does; it can either strip it right the way back or stage the process as they become more comfortable.

But the feedback at the moment is that they like being able to focus on what they’re doing which is of course what these technologies can do; they focus people’s attention which again is what autistic people tend to like to be able to do in many cases. So, this technology, as I see it, is about building on the strengths so utilizing the strengths that people have. But there’s obviously the opportunity to engage and increase stimuli as users become more confident.

DEBRA: Have you found it then that because they learn to make a coffee virtually, they then find it easier when they’re making it in the real coffee shop?

NIGEL: Yes, there has been a feedback, but it is only feedback at this point. We haven’t yet really conclusively identified that doing it in a virtual world or virtual reality environment definitely has the real-world impact. But you’re right, the feedback from the people that have been using this has been that it has certainly helped them when they’re going into the real-world context. But I think that’s the ultimate goal of this technology, potentially I think, the generalization. And that’s something the field, as a research-based hasn’t concluded yet but we’re still working really hard to do so. But it’s really pleasing when you hear people give you the feedback what they were able to focus on doing in this virtual reality environment they really were able to then reflect on that learning experience here in the real world.

DEBRA: What’s the difference then between virtual worlds and virtual reality?

NIGEL: I think the key difference for me would be a virtual world is essentially a multi- platform game where there are multi-users, something like Second World and Minecraft would be considered a virtual world. It’s typically accessed through a screen so through a laptop or through a computer where you would have an interface in your screen and you would be represented as the third person so you might be an Avatar in that environment and you might be seeing what you’re doing whereas virtual reality or head-mounted displays are kind of like wearing a big pair of glasses and you would then be presented with an all-encompassing 360-degree environment. So, you’ll see nothing else but that environment. Then what happens is typically you are presented as the first person in that environment so what you see is what you would potentially see in that environment.

So, the difference really is that instead of looking at yourself through a screen and engaging through a screen, you’re actually in that world and wherever you look, wherever you interface, you are seeing what you would see in that environment. So, there is the sense of what we call the sense of presence or the sense of immersion. So being in that space and feeling that you’re actually in that space is much more heightened in a virtual reality space compared to a virtual world space.

But that can throw up some issues and concerns as you might imagine. So, one of the pieces of work we’ve done initially with the VR head-mounted displays was to assess the sensory problems. We know that some autistic people have issues around sensory concerns and sensory issues and there were a lot of possible negative effects being communicated from virtual reality around things like being sick and feeling dizzy and eye strain.

On the first things we did was to work with autistic populations and ask them would they be willing to wear these devices, and would they feel any negative effects from them. So far by engaging with an age range of groups from 6 to 53 years of age we’ve concluded that there’s very low negative effects and in fact, all of those groups are willing to wear a virtual reality head-mounted display without any negative connotations.

But that was one of the first important steps because these technologies are only going to be useful if the communities, we’re willing to work will accept them, and that was the first concern and the first barrier that we had to investigate, and we have concluded that all of the people that we’ve worked are willing to use these.

That’s the main difference really, that sense of presence and that sense of being there. One of the reasons is quite important is because we then think the ability to generalize stuff is increased. So, the greater you feel a sense of ecological ‘validity or the feeling of being natural in the space, the greater the chance of transferring those skills in the real world. So that’s why we’re particularly excited about using the head-mounted display technology building on the work that we’ve done within the virtual worlds: Second Life and Minecraft.

DEBRA: So, what kind of things are you doing with the head-mounted displays now?

NIGEL: So, the example we’ve already talked about, the coffee shop is something we’ve worked with the school. We’ve also looked at building 360 versions of cultural spaces. So here in Bristol, we have a museum called We The Curious. Another issue that the teachers fed back to us was that sometimes the children can feel anxious about going to place they haven’t visited before. So, they were doing a Science Week and they wanted to take students to We The Curious and what we did was we created a virtual reality version of the museum.

So, in other words, the children were able to experience this museum in their school using some headsets and they were able to walk around and see what the space would look like and where they might go. So, they’re able to essentially alleviate some of their fears in visiting somewhere for the first time in a real world by visiting it, touring it, virtually. So that’s one application alongside the coffee shop that we’ve worked with at the moment, which is still quite a new piece of work that we’re doing and we’re looking to develop that further.

And most recently, we’re now looking to develop employment skills and so we’re going to be working with 16-18-year old’s in the school as they transition into university and/or employment. We’re looking at developing a virtual reality environment and scenario where they can better understand what it’s like to be in an interview environment and what to expect in an interview environment and how they might navigate that environment. Because, I think we’d all agree, that any job interview, is very stressful and for people with social-communication issues and/or anxiety issues, it can be very difficult and so we’re looking at providing a virtual reality environment and testing whether or not it can prove beneficial for them. That’s a new piece of work we’re embarking upon.

DEBRA: With the job interviews, do they get asked questions or is it just getting to the interview?

NIEL: No, in that environment, we’re going to be placing them in an interview context where we’re going to model what a good interview looks like. So, we’re going to be like game-based learning into it. So, they’ll have the chance to input what happens next. So, you’ve been asked these types of question, what might be the answer that you give: this, this, or this? And as a result of giving the answer, the scene will be played out. So, they get a chance to see what the response to that type of answer that might be.

So, we wanted to give them a chance to learn, what the type of responses are that they might give, giving them the chance to see how people respond to those things and because they are in this virtual environment, they’ll be very immersed and really feel the power of that sort of feedback, I think, of the interview environment. So, it gives them a chance to, which is all these technologies looking to do, give them a chance to make mistakes, test things out, find out what happens if. Doing that without real consequences, I think it will real benefit to this particular group that we’re working with. So that’s how we see that working and we are still in the process of planning that with the school and making it appropriate to their coffee shop environment. So, they will be applying for a job to working in this coffee shop in this particular environment.

DEBRA: Do you put program into the headset?

NIEL: Yes, so in this case, we’re looking at using 360 video content. It will have to programmed and coded and created in a way that enables them to choose what option they want to input and to follow those routes through and then choose the next option and follow those routes through. We’re working with the employers, we’re working with the teachers, we’re working with various stakeholders to inform what needs to happen in this phase. Also, what a good outcome looks like for these particular students.

So, it’s important that we work with these different stakeholders. And we’re working with VR developers and we’re working with educationalists. And in order for us to generate the evidence of this research and the efficacy of this type of work, you need to work in that way. And it can be challenging and it can be difficult but I think it’s important that we do. Because for me, it’s really important that all the different voices, from all different stakeholders have a role to play in forming this research. So, I work very closely with autistic individuals to feed into this and to shape the research from the outset because I think we’ve moved beyond researching about autistic people to researching with autistic people. So, involving autistic people and having their voices come through this work is something we’re very, the school and I, are very passionate about doing.

DEBRA: Key takeaway? Talking to the users of technology is key to making technology that actually benefits the young people that it’s meant to be working for.

Resources
More about Nigel

Please subscribe to this podcast and if you have a story you would like to share I would love to hear from you just email debra@journeyskills.com

Please Review Us On iTunes

If you liked this podcast and would like to help us, please do the 1, 2, 3:
(1) Click Review on iTunes (2) Click ‘View in Itunes’; and (3) Click on ‘Ratings and Reviews’ (just to the right of ‘Details’) and leave a review.