In this 5-month project, which was issued by Veiligheidsregio Twente, we aim to find a way to assess the self-reliance of individuals by designing a serious game. More specifically, to find a way to assess the ability of individuals to rely on their own powers and resources to survive a crisis by making them play a serious game and gather information through it. The information collected from the game could help the Twente safety region to make better-informed decisions. Our ideal target group was citizens of the Twente region. But in the real case, we expanded the target group to the students of University of Twente, which is more inclusive and diversified.
Assessing the resilience of people in advance is crucial for the relevant body that deals with crises.The Veiligheidsregio Twente (VRT) is one such local body that is responsible for crisis management in the region of Twente. For simulations to be effective and practical the researchers need to collect accurate information about what will happen in alive scenario, with years of experience VRT has a lot of information on the operational side of crisis management. After discussing with the related personnel from VRT, the initial concept of designing a serious game about the crisis which should be relatable to the users' daily life for accurate results had been determined. Moreover, in order to obtain the real response of users, the scenario in the game should be realistic and common to make sure people can immerse themselves in the situation and get accurate responses.
At the define phase, we focused on determining the specific scenario for our project, we initially discussed several different crises based on mock simulations that are run by VRT for internal training, some of them are natural calamities, road accidents, and more. We used these as references and explored some scenarios on our own. There are four crises we talked about in varying degrees in the beginning, they were Earthquakes, Storms, ChemicalMaterials Leaking, and Power outage. There are certain amount of basic protection facilities which have been established since a serious flood disaster happened in 1953. There are accordingly enough protection measures in place by relevant departments, hence we went on with exploring other topics. As for the chemical materials leaking, a very detailed scenario setting already exists in the training system of VRT. And taking into account that it is highly difficult to achieve in a mobile game to simulate the effect of gas or liquid leakage very real and immersive. However, for power outage, there can be a variation in difficulty depending on the time of duration of the outage. Predictably, long-term power outages will cause the lack of many social functions. Thus, after exploring different kinds of crisis, we finally determined that the power outage is relatively suitable for our demand to build the scenario in the project.
Game Design
From Jesse Schell’s research(2020), there are certain keywords that include voluntary control systems(Elliott Avedon and Brian Sutton-smith), interactive structure(Greg Costikyan), users/players, and unequal outcomes(Tracy Fullerton, Chris Swain, and Steven Hoffman ). In our case, we mainly focus on developing a serious game which is a digital game used for purposes other than mere entertainment(Sushi et al., 2015)’. And based on our goal of this project, the SixFacet approach(Marne et al., 2012) and the MDA framework(Hunicke et al., 2004) can be accordingly referred to as the design framework. That will be elaborated in section of Research Approach.
Competitive Research
To understand better how the serious games work in different context, and to learn the working flow, aesthetic, and interactions in serious games. The researchers went through four existing games based on usability in our context, aesthetics, user-friendliness, drawbacks, and implementation & feasibility. Here are the games(see figure on the right) that we took inspiration from when developing the game design:
From these related works we can understand the different interaction methods used by game designers in serious games, the type of graphics, and the complexity. They have used gamification instead of using questionnaires, we used these inspirations to design outlines for our interaction screens.
The development process of this project is mainly divided into 4 phases: Discover, define, ideate, prototype and test. There were certain adaptations to the design process which has been made, and we also added a pictorial representation (figure below) of the design process we would follow in this project. Accordingly, we divided the project into 4 phases. The first is the empathize phase or Discover phase as it is described in some diagram.
Additionally, the MDA framework can be introduced at the following stage after the earlier definition. The design methodology of MDA stands for Mechanics,Dynamics, and Aesthetic. Separately, Mechanics refers to the technical principles of a game such as the algorithms and data operation support in the process of designing a game.
Generally, the approach of Six Facet afforded a collaborative library of relevant design patterns to game designers and researchers in need, which can be used to help us construct the theoretical outline of the entire design inception.
The main purpose of the game is to provide information to VRT with apt information about people faced in a crisis situation, information like what are their first responses, who do they reach out to in a crisis, and much more. This information will be used by VRT to make quick decisions during a crisis situation and also help them to develop realistic crisis simulations for internal training of VRT staff. To finalize the information that VRT expects to extract from the game, we had a semi-structured interview session with the safety officer the questions. After the interview, we did some competitive research and an inspirational mood board, this is explained in the literature review section earlier. In a brainstorming session, we mapped requirements from the VRT to the questions that we would ask in the game.
We simulated a crisis situation with a card sorting workshop to understand the community dynamics when a crisis is going on, this is a very vital part of activating the users to respond to the crisis. To finalize the initial six cards that we gave in the workshop, we as a group came up with a list of actions each corresponding to power outage crisis similar to the card sorting workshop. We chose the best six amongst all the actions we pooled together by dot voting is shown in figure below.
We formed a template in Miro (online collaboration tool) as shown in Figure below. There are four sections for each participant indicated by the 4 sections in the figure highlighted in blue, the sections in the middle are ‘Interventions and announcements’ and ‘Interpersonal communication'.
Card-sorting workshop has been completed and when starting the analysis of the result we get into the define phase. With the scores from all of our interview participants, we can look at what people consider the most important activities. We had two different sections to identify the differences between International and Dutch students. The columns in the following table represent the average total scores, the average scores when we segment the population by nationalities, and the average score difference between these two groups.Everything is classified by individual cards. The golden color represents the highest value per row.
We also conducted an individual interview after the card-sorting session. That helped us understand unique insights about each question. Several questions were asked mainly to know participants' basic actions on reacting the power outrage crisis. For example, we intended to know whether people experienced a prolonged power outage before, and to know what they will do in the first place to stabilize yourself in the face of the crisis, and to see what the supplies are that they generally have at their house if such a power outage suddenly happens, etc.
The activities we did in the card storing were taken as a base for the game screens, but still, they required some refinement. To refine this we defined the game and had a long in-person meeting where we discussed the activity that the user would do during each day, the mode of interaction, and made a user flow. This gave us an overview of how many screens and assets are required for the game. The user flow also highlights the key action at each screen.
%20(1).png)
The image on the left bottom of figure showed below was selected as the base figure to identify the color scheme and tone of the graphics we are selecting for the game. During the dot voting session, we had a discussion on the same and finalized this.
We initially discussed what could be the possible game interactions, it was unanimously agreed that we did not want to get answers in a questionnaire and wanted to make the game as interactive as possible. From the inspiration mood board, we borrowed ideas and made collaborative sketching to develop some initial concepts. The game introduces the player as a University student staying in his dorm room (a situation any student can relate to). We have used the game mechanics and converted the questions into interactable gameplay with click interaction, press and drag options. Most of the questions are displayed as pictorial options. An example of the introductory scenario can be “On a winter day you are in your dorm room, suddenly you hear a big sound and your power goes. What is your first response?". Below is the image where you can find some of the actions (first iteration sketch) that the user can do on the screen.
We were able to do all the interactions in Figma except drag and drop interaction, which we were ready to sacrifice for the speed of prototyping. Any correction that may arise from this interaction could be directly prototyped in the final version. We preferred to go for digital tools over paper prototypes as this could help to reuse some elements that we need to create for the final prototype. Initially, we planned to conduct one or 2 user testing to evaluate the different modes of interaction like sound + Graphics, various game interactions like swipe to move, point and click interactions. We then realized it's much faster to incorporate them into the hi-fi version prototype as without actually conducting 5 to 8 user testing the data we gather may not be a good enough representation of the population. All in all, Figma had been determined to be the main design tool in this project.
To make the game, there were multiple frameworks available. But the two frameworks we were a bit torn about, in the end, we’re either using Unity, a proper gaming engine, or simply creating a web application using JavaScript. The main advantages of using Unity are the 3D aspects it provides both in resource development and the rendering engine. Creating a 3D game using Unity takes a lot of time which we currently lack. Moreover, we wanted to keep the graphics light so that the people feel they are playing a game while handling the serious subject. So we went ahead with a 2D game. As it was just a simple click-and-drag adventure in 2D, Unity does not really bring in any meaningful advantage over using a web application. Hence, we went ahead with JavaScript.
The final version of the game was made as a 2d adventure game with illustrations and graphics used to convey the scenario to the user. The hi-fi prototype was made in Figma with a point and click interaction. In our research the prototype would consist of a series of screens and certain elements in each screen will be interactible. The interactions in these screens will replicate the actual interaction that the user would do on the final prototype. We have shown the image of the actual dorm room which helps the user to relate his stay at a dorm room during his studies, and the image of a snowstorm to show the severity of the storm(as you can see in figure below).
Based on the stakeholders’ needs and the results of user testing, we determined the basic game machines of this project. Essentially, the game can be realized on mobile devices and on the web. Exploration and decision-making will be the primary part of this project. Some operations and decisions of users in the game will accumulate corresponding points for them, which is reflected in the form of a progress bar in the shape of stars.
Generally, the game's difficulty level was designed to simulate a 3-day power-off crisis. We intended to make our users experience the game’s entirety as much as possible. So that they could have a more holistic and realistic response through this game. As you can seethe basic game experience flow in figure on the right.
We planned to include both four INTERNATIONAL and four DUTCH students in this activity.After having interacted with the HiFi prototype, the participants were asked to fill in some general impressions questionnaires which gives them an opportunity to comment on various aspects of the prototype, overall experience, parts that were difficult to understand, parts they liked and disliked. After this, they were asked to fill out a System Usability Scale(SUS) and User Experience Questionnaire(UEQ). Before the main session of testing, participants were filling out a personal information questionnaire that collected some generic data about them. This activity took place for around 30-40 minutes per user individually. The activity was going to be very simple. We would ask the users to play through the prototype of the game. These are some interactions in the game: Items in the fridge, users can drag items into their fridge from a list we provide, User in his dorm room, gets a notification on his phone, etc.
First of all, participants need to get through the introduction part of the game. Subsequently, users need to explore all the interactable items in the first level. Next, the first announcement was published. Users would be informed about the last time about this power outage crisis. After this task, the first intervention was set for users. The user can make multiple choices and try to survive under a given situation. Then, the second day was coming. A news from University has been notified to users, which needs users to decide if they can be the volunteer to support the Uni for a series actions. The third day as well as the last level of this game. Users complete the last task, which is more about knowing if the user is competent enough to help others after two days surviving on their own in this power outage crisis.
After the session of testing observation and information collection, we analyzed all the problems that participants met during the testing of the hifi prototype. More specifically, we listed corresponding confusions from participants and recorded some misoperation based on our observations as we can partly see in figure below. Based on the analysis, in order to obtain a relatively objective result with a web prototype in the next step, we did corresponding improvements in details to new version.
For example, the button had been magnified to get more attention from users. And about the changes made on some items, this was basically due to that there being no distinction between clickable and non clickable objects in the game. This can be resolved by simply greying them out and visually confirming that they are non clickable.
The final web-prototype was built in Javascript. In order to be able to render the graphics, the P5.js library was used. This library provides easy-to-use methods to interface withWebGL and display images, shapes and text. It also makes dealing with user-input and programming all the interactions a bit easier as compared to vanilla Javascript. And the game of web-version shows below. As with hi fi prototype, we had an observation sheet to record the way people went about playing the game. We then followed it up by noting down general impressions of the people (same procedure as the hifi testing) followed by SUS and UEQ questionnaires to evaluate the system.
Game Link
To compare the UEQ scores of hi fi prototype and web prototype to see where exactly we have made improvements and what are the margins made in each category. As one can directly infer from the table below, the improvements were made in every possible facet. The attractiveness score went up and so did the variance in the score, but the difference is not too much. This was expected as the graphics were still mostly the same with only minor improvements made. The perspicuity is one factor that improved hugely in both score and variance. This is mostly due to the fact that users now have more control over the game attributes. Dependability improved theoretically, but as mentioned earlier does nor really make sense in this iteration due to the fact that this game still is about testing the interactions.Stimulation and Novelty are two factors that improved in values but their variance also drastically increased. These are expected to change since we got in a lot of new users and these attributes are ever changing amongst participants.
From the SUS table below, it can be observed that for most questions there is a high degree of polarization towards either extreme (strongly disagree or strongly agree). This proves that the participants have a strong consensus on those questions. Overall, the consensus is strong that the prototype is simple, easy to learn and use. The web prototype has a very marked improvement from the hi-fi prototype, which only scored 40.63 while the web prototype scored 75.63. Since the SUS score is improved and consensus is stronger, we canconclude that we have overall achieved better user satisfaction for the web prototype.
During the whole process of completing this project, we have put more effort into improving the usability issues, which generally helped us to improve the user experience and fluency of the game playing. On the other hand, we partly ignore the aspects of user engagement. In addition, the reasons why users perceive the game in a less engaging way may be due to the lack of awareness of the serious game definition.Similarly, some necessary interactions should be added to provide user feedback, which can help improve users’ feel of engaging. At the same time, the customized user feedback would be a key factor in users wanting to share this game. This is the part that we are missing in the game at present.
Let's get in touch!