VOL. 32, No. 2 2017
This paper presents the development phase of the of the Virtual World that is going to be used by the Virtual Learning for People with Autistic Spectrum Disorder (VL4ASD) project, which aims to create training materials on conversation skills. This project is geared towards addressing the communication deficits of ASD populations, by exploring the extent to which the use of Virtual Reality (VR) could make an impact on the daily lives of people with those conditions and, thus, address a research gap. The initial results from the feedback sessions are positive towards using VR for training purposes. Feedback from both users and practitioners revealed that the training materials can be very beneficial to the target population.
The stated goal of this project of explaining the steps necessary to engage in effective and fruitful conversations for people with ASD through VR, could be helpful to these individuals in their social interactions on a one-to-one basis. A next iteration of the VR training programme should involve having conversations in a more complex group dynamic that would enhance group members’ self-confidence and self-esteem. Ultimately, this VR training would assist young adults with ASD in their pursuit of independent living by giving them the opportunity to gain work experience. This study, as a proof of concept, would open the door to developing training for other communication, life, academic and vocational skills, while this training would not only be useful for people with ASD, but the wider population.
Cet article présente la phase de développement du Monde Virtuel qui va être utilisé par la projet Apprentissage Virtuel pour les Personnes atteintes de Troubles du Spectre Autistique qui vise à créer des supports de formation aux compétences conversationnelles. Ce projet est conçu pour cibler les déficits de de communication des personnes présentant des troubles du spectre autistique (TSA) en explorant la mesure dans laquelle l’usage de la Réalité Virtuelle (RV) pourrait avoir un impact sur la vie quotidienne de ces personnes et, ainsi, combler une lacune de la recherche. Les premiers résultats des sessions de rétroaction sont positifs quant à l’usage de la RV à des fins de formation. Les retours des usagers comme des praticiens révèlent que les supports de formation peuvent être très bénéfiques à la population ciblée.
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
Virtual Reality (VR) has made a big comeback in recent years due to technological advances that have made it more affordable to the public. The term first appeared in the 1960s when a mechanical device called the Sensorama1 created an immersive VR. VR was not a commercial success in subsequent years, and even a few years ago there was scepticism by veteran industry leaders that this latest incarnation of the technology would end again with commercial disappointment (Cronin, 2015). However, now, VR has evolved to the point that it can imitate the real world very convincingly (Freina & Ott, 2015) and has become a viable, if not thriving, technology with applications in many sectors. Virtual World (VW) environments involve people (as avatars), that can talk to each other by voice or text chat, in public or in private, in a space that does not exist and yet that simulates the real world (Bartle, 2004). A VW is “a synchronous, persistent network of people, represented as avatars, facilitated by networked computers” (Bell, 2008; p. 2). A VW can either be a non-immersive (on a computer, laptop, or tablet screen) or immersive (VR glasses, head mounted displays) 3D space.
Virtual Worlds are bespoke environments that support training in a safe and less anxiety-provoking environment, avoiding potential real dangers, and where one can embed training and learning exercises and practice repeatedly in order to achieve goals (Freina & Ott, 2015). These are closely aligned with the major strengths of e-learning, which include comfort (using the technology at your own pace from where ever makes you feel comfortable) and user-friendliness of digital media, along with individualisation of the learning experience, since every person learns in different ways (Krämer & Schulte, 2008). The VWs have thus found application in an educational and rehabilitation context. For example, Cornell University’s Weill Medical College is currently exploring if Virtual Technology can be used to help individuals cope with posttraumatic stress disorder (PTSD) related to the World Trade Center bombings, the NOAA (National Oceanic and Atmospheric Administration) created a virtual world to simulate a tsunami or hurricane, while in the field of forensic pathology, Second Life (Virtual World platform) is being used to teach autopsy procedures.
Studies have shown that a Virtual 3D environment can stimulate users’ interactivity (Roussou, 2004) and motivation (Garris, Ahlers, & Driskell, 2002; Ott & Tavella, 2009). It affords opportunities for social interaction, especially in the classroom, by making conversations easy, structured and inclusive, through, for example, text-chat systems (Newbutt, 2013). Virtual Worlds can offer users a sense of co-presence and realism (Childs, 2010; Yee, Bailenson, & Ducheneaut, 2009) and provide an increased sense of control, allowing participants to have more engaging in-world experiences (de Freitas Rebolledo-Mendez, Liarokapis, Magoulas, & Poulovassilis, 2010). Moreover, a VW can improve students’ knowledge, their enjoyment and interest in the learning process, if they are structured correctly, just like game-based learning does (Papastergiou, 2009). The fact that training in a VW is done entirely online, means that it can be considered as part of the e-learning universe (Sangrà, Vlachopoulos, & Cabrera, 2012), which is defined as delivering educational material through electronic media for teaching or training purposes, either to compliment or to completely substitute the physical classroom (Guri-Rosenblit, 2005; Koohang & Harman, 2005). Regardless of the educational model employed for the use of a virtual environment, incorporating information and communications technology into the learning process involves practices (e.g., providing opportunity for peer to peer discussion, creating private spaces for study and offering opportunities for networking/socialization) that can be traced back to distance education (Sangrà et al., 2012). Moreover, some researchers are of the opinion that e-learning can be considered a concept that actually evolved from distance learning, or a new generation of distance education (Garrison & Anderson, 2003).
Extensive research has been carried out about the affordances and benefits of the use of VR to the general population. However, a recent systematic review of immersive VR technologies (Freina & Ott, 2015) showed that relatively little research has been carried out with people with disabilities and more particularly people with developmental disabilities (DD). Moreover, there is limited VR literature which deals with social skill deficits and related training for people with autism and intellectual disabilities (Matson, Hattier & Turygin, 2012). That is the type of population that could largely benefit from VR, given the fact that “learning in a virtual environment that reproduces the real one can minimize the problems related to learning transfer” (Freina & Ott, 2015; p. 139).
Developmental disability is a broad term that usually covers autism spectrum disorders (ASD), and other developmental disabilities (e.g., intellectual disability, cerebral palsy, Down syndrome) that begin during the developmental period, may impact day-to-day functioning depending on its severity level, and usually last throughout a person’s lifetime. One of the most common developmental disabilities is autism spectrum disorder (Matson & Kozlowski, 2011). About one percent of the world’s population has ASD, and the prevalence in the United States is approximately one in 68 children (CDC, 2014). People on the autism spectrum typically have deficits in communication and socialization skills, along with exhibiting repetitive behaviours and/or restricted interests (Fombonne, 2005; Hattier & Matson, 2012; Levy & Perry, 2011; Matson, Rieske, & Tureck, 2011; Rivet & Matson, 2011; Suzuki, 2011; Wing, Gould, & Gillberg, 2011).
People with ASD enjoy computer-based activities (Chen & Bernard-Opitz, 1993) most likely due to the limited real-world interaction (Moore, 1998). In fact, several studies have shown that the learning experience of people with ASD is enhanced when educators, therapists and service providers can provide them with a safe, unique and flexible virtual space (Georgescu, Kuzmanovic, Roth, Bente, & Vogeley, 2014; Kandalaft, Didehbani, Krawczyk, Allen, & Chapman, 2013; Newbutt, 2013; Parsons Mitchell & Leonard, 2004). This population finds Virtual Reality environments very engaging and quite immersive as well, where they can work on their soft, social and academic skills (e.g., employment-related skills training, such as job interview training) from the comfort of their home (Smith et al., 2014; Smith et al., 2015; Strickland et al., 2013). They also get the chance to explore the environment and the teaching/training material on their own and engage in independent practice (Rizzo & Kim, 2005), and have control over their level of participation (Parsons & Mitchell, 2002).
This paper presents the development phase of the of the Virtual World that is going to be used by the Virtual Learning for People with ASD (VL4ASD) project, which aims to create training materials on social communication skills (starting a conversation, picking appropriate topics for discussion, etc.). This project is geared towards addressing the social communication deficits of the ASD population by exploring the extent to which the use of Virtual Reality can make an impact on the daily lives of people with these conditions and thus addressing that research gap.
The VW is a first iteration of the HIVE-RD “R&D.Construct” platform, which was developed using Unity3D because of the customizable nature of the platform. The look and feel of the prototype of the VW environment was formed based on the developers’ shared aesthetic influences ranging from architectural books and lectures, to music, art, film and video games. They were guided in the prototype phase by internal critiques and software testing sessions, because, according to their professional experience, “people tend to ask for pre-made example environments, rather than planning anything completely by themselves”. When it comes to creative industries, studies have clearly shown that brainstorming groups produce fewer and poorer quality ideas than individually produced ideas (Furnham, 2000). This is because companies and, more generally, fields that depend on research and innovation, find external restrictions often kill creativity (Ciotti, 2013).
It is also worth noting that the VW can and has been used in different contexts. Clients that the VW developers have worked with, have commissioned VWs that would be appropriate for a variety of projects and thus have particular requirements on their design and use. The software testing sessions involved identifying a brief list of bugs or issues to be scrutinized, and then trying to find the cause of the bugs and what led to them; or in the case of usability issues, trying to make a feature more user-friendly by increasing its visibility, its ergonomics or by binding it to a hotkey. Given that the focus of the software testing session was on evolving a features’ usability, the developers identified when an existing feature or function overlapped with a potentially new feature that would improve user experience. Existing code was then reused and redesigned to rapidly develop that new feature.
Intervention research is helpful to individuals with ASD, their caregivers, and educators in translating information gathered from them into practice (Goodwin, 2008). Design-based research (DBR; Anderson & Shattuck, 2012; Barab, 2006) is situated in a real educational context (can inform and enhance practice), it has no restrictions on the method(s) employed, the process is iterative and empowers and encourages a collaborative environment (Anderson & Shattuck, 2012; Barab & Squire, 2004), where transfer of knowledge between all the stakeholders engaged in this process is possible.
A participatory design methodological approach (also referred to as co-creation or co-design) was deemed most appropriate for this project because it allows for a diverse group of stakeholders to get involved in the design process in a collaborative manner. This method is often used by designers in an effort to better understand their potential end user’s cultural, societal and financial situations. In doing so, a well-informed, empowered and active public is engaging in co-creation (Prahalad & Ramaswamy, 2004) with designers, sharing their knowledge, expertise and experiences.
The development of the VW was an iterative process, which, at the initial phase, involved software test sessions with two, 15-user, stress-test groups, where each user conducted a pre-prepared sequence of tests and was requested to report any anomalies or features that were difficult to use. Of the 15 users in each of the two software stress tests, eight had ASD (seven did not), because the intended user-base of the HIVE-RD “R&D.Construct” platform is potentially very diverse, so the developers value testers from a variety of backgrounds. Optimisations made based on the results of those stress tests have since led to significant improvements on performance.
A user test phase followed, where two feedback sessions were held with individuals with ASD and practitioners with backgrounds in special education. The engagement of people with ASD in the development process of the VW and the training material offers the opportunity for “the exploration and identification of presumably positive future possibilities” (Sanders & Stappers, 2008; p. 8); it allows people who can be excluded in society to contribute to the decision-making process, leading to the improvement of the end product through democratic practices (Greenbaum & Loi, 2012).
The creation of a minimally-viable product was the starting point for user participation. This was deemed necessary due to the lack of knowledge and expertise of the public in new technologies (in our case Virtual Reality), new platforms, techniques and approaches, which means that the public is usually unaware of their capabilities and potential; therefore, they do not know what they can ask for and, as Steve Jobs told Business Week (1998, November 23) people don’t know what they want until the innovators and pioneers show it to them.
Stress Test. Based on the data gathered from the stress test on the February 9, 2017, improvements to performance have yielded the following results between two sessions (February 9 and 16, 2017):
That software testing phase was primarily concerned with usability issues, since the platform's code is still in a development stage. Based on the two testing sessions, there was no noticeable difference in performance between the two groups (people with and without ASD), but if there had been different results the developers would have had to try to design solutions for those problems.
Usability Test. The first feedback session consisted of six people (users) with ASD (two female/four male) in their early twenties (mean = 20 years old, range = 18-22 years old). All of them had undergone an employment readiness skills training programme at a university in the Midwest region of the United States. Presumably, they would have first-hand experience of the type of skills necessary to have a successful job interview, which would include conversation skills. All participants were encouraged to try out the training program themselves during the session. Each of them was then asked to complete a feedback form regarding their experience with both the VR technology and the training material. The second feedback session consisted of two practitioners who have had experience working with the above-mentioned population. Both groups could therefore provide the researcher with useful and unique insights and recommendations in terms of how to improve the training programme.
The feedback sessions consisted of a brief presentation of the project, followed by a demonstration of the three elements of the intervention (PowerPoint presentations, videos and non-playable character) and time allotted to express their opinion about the VW and the training material, by filling in a brief questionnaire. There is one PowerPoint presentation for each one of the six training sessions, covering a topic related to having a conversation, namely: (1) What is conversation and how does it work?; (2) Why is conversation useful and approaching someone to have a conversation; (3) Starting a conversation and why are topics appropriate and inappropriate; (4) Finding common interests; (5) Taking turns/Answering questions; and (6) Switching topics/ Ending a conversation. In compiling the training material, numerous resources on social communication skills were consulted that addressed training in these areas for the general population, for people with ASD and training in a virtual environment (e.g., training in Second Life). The six key themes focusing on conversation skills, mentioned above, were identified and addressed in PowerPoint presentations. Colleagues with extensive expertise with the target populations helped with the refinement of the presentations in order to simplify the terms used and ensure that the language was familiar to the participants (English — the lead researcher’s language — versus American terminology). The training also included videos, which aimed to reinforce the messages from the presentations. They were found after a lengthy search on YouTube based on search terms from the six key themes. The last component of the training was Chatbot, a non-playable character that was programmed to have a controlled interaction with the users. A script was developed for the Chatbot that tried to anticipate a participant’s possible answers to a sequence of questions or statements, so it was kept as simple as possible, to avoid complications arising from unforeseen responses.
Feedback from Users with ASD. During the two feedback sessions, five out of six users reported that they enjoyed the VR experience (one user did not answer), five users claimed that they would like to use VR for training purposes in the future, two of them had some VR experiences prior to the demonstration/try-out and none of the six users had any complaint or experienced any negative effects (discomfort, dizziness).
Overall qualitative comments from written feedback forms and verbal discussions suggest that VR has a significant role to play in training for people with ASD (immersive VR possibly even more so). Comments from these users included: “It helps get rid of anxiety caused by real life,” “You get a surreal and interactive experience,” and “New learning experience.” The Chatbot, a non-playable character, was quite popular with the users. One of the users stated that he preferred “having the lessons (i.e., PowerPoint presentations) in the same area as practice (Chatbot)” whereas, another user liked the fact that the Chatbot could be used for “practicing social skills.”
However, the VW can be challenging for some. For instance, a user indicated that “it can be harder to concentrate when compared to real life,” as it is “not really talking to an actual person.” Additionally, there were some issues and concerns raised with regard to the “look and feel” of the VW (colour scheme, size of screen) and with usability (movement with the keyboard). Some comments reported by the users included “moving around as the avatar was challenging”, while another indicated “it (VW) is slightly drab.” In addition, the users offered several suggestions regarding: the content of instructional material (e.g., conclude each PowerPoint presentation with a quiz), organization of content (e.g., organise video playlist better), visual presentation of content (e.g., enlarge video screens and increase font size) and pacing of content display (e.g., enable Chatbot's questions to stay on for longer time).
Feedback from Practitioners. Both practitioners commented that VR is useful for educational purposes with people with ASD and would like to use it for training in the future. The practitioners appreciated the advantages of using the VW as it “cuts down on background noises” and potentially provides “comfort, ease of access.” Nonetheless, they also pointed out some general disadvantage of using VR as it “takes focus off human contact”.
The practitioners then offered specific comments on each of the elements of the intervention, including the content of instructional material (e.g., simplify the language on the PowerPoints), visual presentation of the content (e.g., add more variety in the background, tone down the background colours), pacing of content display (e.g., slow down the speed of some videos, reduce video/audio lag), choice of videos (e.g., ensure topics and/or characters are age-appropriate). Moreover, the practitioners indicated that the embedded videos could have a practical/demonstration element and would be beneficial for users with ASD. Lastly, while the practitioners highlighted that the Chatbot could have huge learning potential, they reported that it is not interactive enough at this point to achieve anything of real value.
While this paper only showed the results from two feedback sessions with users with ASD and with practitioners, the piloting/testing phase of the training materials is still ongoing. The initial results are positive about the VR/VW in general and about its use for training purposes in the future. Specifically, delivering the training materials in three formats (PowerPoints, videos and the Chatbot) is seen in a positive light by both users and practitioners. The three formats represent to users teaching, showing and practicing, respectively, and thus they can all contribute to the learning process in different ways.
The literature on this topic area suggests that it is the participants themselves who can best describe subjective conceptions of the surrounding world (Marton, 1981) and its beauty. The concept of aesthetics describes the desirable emotional responses evoked in the player (Hunicke, LeBlanc, & Zubek, 2004) with regards to appreciation of beauty, whether in the real (natural) or virtual world. It is imperative, when designing a virtual environment, that we get the users’ views on how they want the VW to look, feel and work, because the strongest motivating factor for them is an enhanced, more enjoyable experience (Crawford, 2004). The user feedback received to date, has revealed ways we can change the VW to make it more appealing, acceptable, and user-friendly.
Some of the comments and suggestions made by the users and practitioners regarding the visual presentation, pacing and organisation of the content can be easily addressed. For instance, using a mouse or a joystick for easier movement in the VW or makeing it possible that the avatar can move through furniture, as well as embedding the videos and enlarging them on the screen. It would also be feasible to increase the text font for the Chatbot and let it stay on the screen longer. The video playlist can also be simplified by either having a separate one for each session or by numbering them according to a specific session. Moreover, we have set up a browser link with a SurveyMonkey questionnaire with multiple choice questions as a quiz at the end of each PowerPoint presentation. The feedback received will help make navigation of the space easier, and will make the space less distracting or confusing.
Other comments made by the users and practitioners may be based more on individual perception and be rather subjective, such as the colour scheme of the VW. The current colour scheme is a balance between bland corporate and bright colours. While some commented that the colours were bland, others indicated that the colours were quite bright or even slightly distracting. Whilst it is possible to have a selection of colours to choose from dynamically, this comes with its own complexities when related to scene lighting. A compromise may be to have two to three schemes that the user could choose from when he/she is first introduced to the VW.
Having user involvement in the development and design phases of a product or service can be helpful and empowering, however it can also be distracting and cause a loss of focus. The approach would have a more successful outcome if the users have sufficient knowledge (subject or design) and experience (Yip et al, 2013), so that they could identify the areas where they need to focus their attention in order to meet the educational goals (Druin, 2002; Ke, 2014). For instance, the participants in our feedback sessions seemed to be focusing on design elements (narrative, sounds, characters, etc.), and somewhat neglected the educational content (the training materials in all three formats), which is supported by the literature in participatory design of educational games (Ke, 2014).
Finally, there are more complex issues to be resolved. The Chatbot, a non-playable character, was popular because it involved practicing the skills that were mentioned in the PowerPoints and were shown in the videos. However, the functions that could be executed by the Chatbot in its current form are quite limited and lack a useful level of interactivity. Developing an artificial intelligence is beyond the financial capabilities of this project but it would open up a great deal of options for training in the future. A possible solution for this project would be to work out the ways in which the conversation could potentially branch so that the Chatbot can respond appropriately when certain words appear in the user’s answer.
Overall, most of the feedback was constructive and will lead towards the creation of a better product. However, as was also stated in the review of the literature, studies have shown that user involvement does not necessarily produce a better result or more effective services. This would depend sometimes on the level of knowledge and experience of the testers with the subject matter. In addition, a few of the comments were either subjective or were lacking knowledge of the constraints that would make those suggestions difficult to address.
The relatively small sample size (six users with ASD and two practitioners) is a major limitation which restricts the generalizability of the study, and participants’ views might not be representative of the majority of the population. That being said, the study provides some insights and useful information for continuous revisions and alterations to the VW and of the training materials to achieve a better product. On that note, most of the comments received in the feedback sessions were helpful, constructive and could reasonably be addressed. However, there were some comments that were either subjective or would require a great deal of resources, in terms of money and time, which are the main limiting factors of the project. Therefore, there is a need to be very deliberate and selective about the changes that are both necessary and feasible in terms of meeting the needs and preferences of users within the above-mentioned constraints. Further study is strongly recommended to test all the elements of this project and its intervention through user-involvement in every stage of development.
While a participatory design method was adopted for the development of the VW (starting with a minimally viable version of it), the same cannot be said about the creation of the training material. The users did not have any input during the identification of the key themes of training for conversational skills but, rather, were asked to give feedback on all three formats of the training materials (PowerPoints, videos and Chatbot), when those materials were already at an advanced stage of development.
With regards to the Chatbot more specifically, its rather limited interactivity led to its withdrawal from the intervention that is currently being carried out in the study. However, since it was deemed to be – by the participant that provided the feedback – the most important training element, the features and functionality of the Chatbot will be revisited.
The stated goal of this project is to provide training that may improve social communication skills, more specifically the steps necessary to engage in effective and fruitful conversations for people with ASD through the use of Virtual Reality (VR). This could be helpful to these individuals in their social interactions with family, friends, peers, educators, employers and other people (on a one-to-one basis). A next iteration of the VR training programme would involve having conversations in a group dynamic, which is more complicated and nuanced but that kind of training would further build the individual’s self-confidence and self-esteem. A more advanced level of the VR training program would assist them in the pursuit of independent living by giving them the opportunity to gain work experience, such as training others to use the Virtual World (VW) or offering courses relevant and useful to job search through virtual job centres (Politis et al., 2017). This study, as a proof of concept, would open the door to developing training for other communication, life, academic and vocational skills, while this training would not only be useful for people with ASD, but also for people that struggle socially.
This project adopted a Participatory Design approach, in order to develop the VW with and for people with ASD. By getting their insights into the appearance and usability of the VW and the appropriateness and usefulness of the training materials, the final product that is being used in the on-going intervention is trying to meet their needs and preferences. However, as was mentioned above, when working with a creative technology there is a struggle to keep a balance between involving the users in the development phase and being truly innovative (Ciotti, 2013). There is, therefore, a need to examine the effectiveness of using Participatory Design with these populations; even though studies (e.g., Benton & Johnson, 2015) have shown that user involvement of any form offers people with ASD the chance to have a voice, Frauenberger et al. (2015) believe that is it almost impossible to gauge the effectiveness of a participatory compared to a non-participatory approach.
While the target populations have had a say throughout the process in the testing and feedback sessions on the technology and the training material, this could be done more and more effectively in the future. The ultimate target would be to have a controlled trial. Two groups of participants with similar types of conditions and levels of severity would undergo an intervention. The first group would use a revised version of the current intervention (based on the analysis of the results and of the reflections of the intervention, as well as the feedback by the intervention participants); the second group would initially identify the key themes they would like to have addressed in their conversation skill training, then co-create the training material and do the intervention with that material. The whole intervention would be carried out virtually, which means we could reach a much larger population, limited by just their access to hardware with the specifications to be able to run the VW.
VR training, if successful, would allow for a wider implementation and increased access, because this training can be delivered from/to any part of the world. This enables distance learning for people living in relatively remote locations or who have mobility issues. Lastly, creative technologies can be of great assistance in relation to dealing with the increased prevalence of ASD, high cost of treatment and limited availability of professional resources (Goodwin, 2008). Using a VW requires high up-front development costs, which have been estimated at $11,000-$35,000, depending on the complexity of the Virtual World (ThinkMobiles, n.d.), an investment in hardware (which is being reduced at a fast pace) and some investment in technical support; however, the subsequent back-end cost is significantly reduced ($5,000-$11,000). On the other hand, a traditional treatment, such as behaviour therapy, can cost upwards of $60,000 per year (Goodwin, 2008). The distance learning aspect to in-home computerized educational material has the potential to provide people with ASD and their families with skills and knowledge and thus reduce treatment related costs.
We are living through a period of rapid technological advancements in areas such as digital technologies and social media. Our societies have adopted a ‘participatory culture’, which has benefited the development of products and services, because companies have been involving users in their development process to get feedback by the potential end users. This paper has presented the design and testing phases of the VL4ASD project. Virtual Reality has potential in creating accessible and engaging products and services for training purposes, because it can be delivered remotely from anywhere and the user can undertake the training at their own pace. Moreover, despite the high up-front costs, it may prove to be a cost-efficient approach in the long run.
While there are benefits in getting the user involved in the development and design phases because it has been a cornerstone to adapting products and services to meet individual needs and preferences, VR designers, developers, and researchers should be aware of the caveat that all individuals can contribute in the co-design process and should feel they are welcome to do so but under the right conditions. Most importantly, though, is the fact that research has already shown that VR training can be effective for these populations (Politis et al., 2017). Now there is a need for a more concerted effort to investigate the effectiveness of participatory design approaches for the development of training for the ASD populations by having more in depth and longitudinal research conducted.
This research was supported by funding from the charity RESPECT and the People Programme (Marie Curie Actions) of the European Union's Seventh Framework Programme (FP7/2007-2013) under REA grant agreement no. PCOFUND-GA-2013-608728.
Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education Research? Educational Researcher, 41(1), 16-25.
Barab, S. (2006). Design-based research: A methodological toolkit for the learning scientist. Cambridge, UK: Cambridge University Press.
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal Of The Learning Sciences, 13(1), 1-14.
Bartle, R. A. (2004). Designing virtual worlds. New York: New Riders Publishing.
Bell, M. W. (2008). Toward a definition of “virtual worlds”. Journal of Virtual Worlds Research, 1(1), 1–5.
Benton, L., & Johnson, H. (2015). Widening participation in technology design: A review of the involvement of children with special educational needs and disabilities. International Journal of Child-Computer Interaction, 3(4), 23-40.
Centers for Disease Control and Prevention - CDC (2014). Data & Statistics. Accessed at: https://www.cdc.gov/ncbddd/autism/data.html
Chen, S. H. A., & Bernard-Opitz, V. (1993). Comparison of personal and computer-assisted instruction for children with autism. Mental Retardation, 31, 368-376.
Childs, M. (2010). Learners' experience of presence in virtual worlds. Thesis (PhD). University of Warwick.
Ciotti, G. (2013). Why Steve Jobs didn’t listen to his customers. Help Scout. Accessed at: https://www.helpscout.net/blog/why-steve-jobs-never-listened-to-his-customers/
Crawford, C. (2004). The art of computer game design. New York: McGraw-Hill Osborne Media.
Cronin, B. (2015). The hierarchy of needs in virtual reality development. Accessed on September 1, 2017 at: https://medium.com/@beaucronin/the-hierarchy-of-needs-in-virtual-reality-development-4333a4833acc
de Freitas, S., Rebolledo-Mendez, G., Liarokapis, F., Magoulas, G., & Poulovassilis, A. (2010). Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world. British Journal of Educational Technology, 41(1), 69-85.
Druin, A. (2002). The role of children in the design of new technology. Behaviour & Information Technology, 21(1), 1-25.
Fombonne, E. (2005). The changing epidemiology of autism. Journal of Applied Research in Intellectual Disabilities, 18, 281-294.
Frauenberger, C., Good, J., Fitzpatrick, G., & Iversen, O.S. (2015). In pursuit of rigour and accountability in participatory design. International Journal of Human-Computer Studies, 74, 93-106.
Freina, L., & Ott, M. (2015). A literature review on immersive virtual reality in education: State of the art and perspectives. eLearning & Software for Education, 2015(1), 133-141.
Furnham, A. (2000). The brainstorming myth. Business Strategy Review, 11(4), 21-28.
Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441-467.
Garrison, D.R., & Anderson, T. (2003). E-learning in the 21st century. A framework for research and practice. London: RoutledgeFalmer.
Georgescu, A.L., Kuzmanovic, B., Roth, D., Bente, G, & Vogeley, K. (2014). The use of virtual characters to assess and train non-verbal communication in high-functioning autism. Frontiers in Human Neuroscience, 8, 807.
Goodwin, M.S. (2008). Enhancing and accelerating the pace of autism research and treatment: The promise of developing innovative technology. Focus on Autism and Other Developmental Disabilities, 23(2), 125-128.
Greenbaum, J., & Loi., D. (2012). Participation, the camel and the elephant of design: An introduction. CoDesign, 8(2-3): 81-85.
Guri-Rosenblit, S. (2005). Distance education and e-learning: Not the same thing. Higher Education, 49(4), 467-493.
Hattier, M. A., & Matson, J. L. (2012). An examination of the relationship between communication and socialization deficits in children with autism and PDD-NOS. Research in Autism Spectrum Disorders, 6, 871-880.
Hunicke, R., LeBlanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. In Proceedings of the Challenges in Game AI Workshop, Nineteenth National Conference on Artificial Intelligence, AAAI Technical Report WS-04-04.
Kandalaft, M.R., Didehbani, N., Krawczyk, D.C., Allen, T.T. & Chapman, S.B. (2013). Virtual reality social cognition training for young adults with high-functioning autism. Journal of Autism and Developmental Disorders, 43(1), 34-44.
Ke, F. (2014). An implementation of design-based learning through creating educational computer games: A case study on mathematics learning during design and computing. Computers & Education, 73, 26-39.
Koohang, A., & Harman, K. (2005). Open source: A metaphor for e-learning. Informing Science Journal, 8, 75-86.
Krämer, B.J., & Schulte, D. (2008). The impact of technology on students at open and distance universities. In F., Agrusti, D., Keegan, G.,
Kismihok, B.J., Krämer, N., Mileva, D., Schulte. J., Thompson, & B., Vertecchi. (2008). The impact of new technologies on distance learning students. Brussels: European Commission, 28-48.
Levy, A., & Perry, A. (2011). Outcomes in adolescents and adults with autism: A review of the literature. Research in Autism Spectrum Disorders, 5, 1413-1420.
Marton, F. (1981). Phenomenography: Describing conceptions of the world around us. Instructional Science, 10(2), 177-200.
Matson, J. L., & Kozlowski, A. M. (2011). The increasing prevalence of autism spectrum disorders. Research in Autism Spectrum Disorders, 5, 448-425.
Matson, J. L., Rieske, R. D., & Tureck, K. (2011). Additional considerations for the early detection and diagnosis of autism: review of available instruments. Research in Autism Spectrum Disorders, 5, 1319-1326.
Matson, J.L., Hattier, M.A., & Turygin, N. (2012). An evaluation of social skills in adults with pica, autism spectrum disorders, and intellectual disability. Journal of Developmental and Physical Disabilities, 24(5), 505-514.
Moore, D. J. (1998, Summer). Computers and people with Autism/Asperger Syndrome. Communication, 20–21.
Newbutt, N. (2013). Immersive Learning Environments: The State of the Art. Learnovate Project Report. Dublin: Trinity College Dublin.
Ott, M., & Tavella, M. (2009). A contribution to the understanding of what makes young students genuinely engaged in computer-based learning tasks. Procedia-Social and Behavioral Sciences, 1(1), 184-188.
Papastergiou, M. (2009). Digital game-based learning in high school computer science education: Impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1-12.
Parsons, S., Mitchell, P., & Leonard, A. (2004). The use and understanding of virtual environments by adolescents with autistic spectrum disorders. Journal of Autism and Developmental Disorders, 34(4), 449-466.
Parsons S., & Mitchell P. (2002). The potential of virtual reality in social skills training for people with autistic spectrum disorders. Journal of Intellectual Disability Research, 46, 430-443.
Politis, Y., Robb, N., Yakkundi, A., Dillenburger, K., Herbertson, N., Charlesworth, B., & Goodman, L. (2017). People with disabilities leading the design of serious games and virtual worlds. International Journal of Serious Games, 4(2), 63-73.
Prahalad, C. K., & Ramaswamy, V. (2004). Co-creating unique value with customers. Strategy & Leadership, 32(3), 4-9.
Rivet, T. T., & Matson, J. L. (2011). Review of gender differences in core symptomatology in autism spectrum disorders. Research in Autism Spectrum Disorders, 5, 957-976.
Rizzo, A. S., & Kim G. J. (2005). A SWOT analysis of the field of virtual reality rehabilitation and therapy. Presence, 14, 119.
Roussou, M. (2004). Learning by doing and learning through play: An exploration of interactivity in virtual environments for children. Computers in Entertainment, 2(1), 10-10.
Sanders, E. B.N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. CoDesign, 4(1): 5-18.
Sangrà, A., Vlachopoulos, D., & Cabrera, N. (2012). Building an inclusive definition of e-learning: An approach to the conceptual framework. International Review of Research in Open and Distance Learning, 13(2), 145-159.
Smith, M.J., Fleming, M.F., Wright, M.A., Losh, M., Humm, L.B., Olsen, D., & Bell, M.D. (2015). Brief report: Vocational outcomes for young adults with autism spectrum disorders at six months after virtual reality job interview training. Journal of Autism and Developmental Disorders, 45(10), 3364-3369
Smith, M.J., Ginger, E.J., Wright, K., Wright, M.A., Taylor, J.L., Humm, L.B., & Fleming, M.F. (2014). Virtual reality job interview training in adults with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44(10), 2450-2463.
Strickland, D.C., Coles, C.D., & Southern, L.B. (2013). JobTIPS: A transition to employment program for individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43(10), 2472-2483.
Suzuki, M. (2011). Mental development and autistic behavior in children with pervasive developmental disorders. Research in Autism Spectrum Disorders, 5, 1517-1525.
ThinkMobiles. (n.d.). Accessed on September 5th, 2017: https://thinkmobiles.com/blog/how-much-vr-application-development-cost/
Wing, L., Gould, J., & Gillberg, C. (2011). Autism spectrum disorders in the DSM-V: Better or worse than the DSM-IV? Research in Developmental Disabilities, 32, 768-773.
Yee, N., Bailenson, J. N., & Ducheneaut, N. (2009). The Proteus effect: Implications of transformed digital self-representation on online and offline behaviour. Communication Research, 36(2), 285-312.
Yip, J., Clegg, T., Bonsignore, E., Gelderblom, H., Rhodes, E., & Druin, A. (2013). Brownies or bags-of-stuff? Domain expertise in cooperative inquiry with children. In Proceedings of the 12th International Conference on Interaction Design and Children, 201-210.
Authors
Yurgos Politis is a Marie Curie Research Fellow, ASSITID programme, Michigan State University. E-mail: yurgos.politis@ucd.ie
Louis Olivia lives in London, UK and is the co-director of HIVE-RD LLP. He has a BSc. in Computing and specialises in developing 3D virtual worlds, most prominently, R&D.Construct. E-mail: enquiries@hive-rd.com
Thomas Olivia is a London-based 3D artist and co-director for HIVE-RD LLP. He specialises in the development of 3D virtual worlds and assistive technology, and is interested in 3D technologies and their application to social causes and business. E-mail: tom@hao2.eu
Connie Sung is an assistant professor of rehabilitation counselling at Michigan State University. She holds a PhD in Rehabilitation Counselling Psychology from University of Wisconsin-Madison and a Master’s degree in Rehabilitation Sciences from the Hong Kong Polytechnic University. She is also the director of Neurodevelopmental Disability and Transition Research Lab. E-mail: csung@msu.edu