Some years ago, when we first started to think about developing conversational simulations, we were obsessed by finding a way to design truly realistic experiences.
In fact, the market was responding to the need for interactive training with simple branched solutions, where each choice of the user was leading to one pre-determined reaction from the simulation. The first “serious games” were popping up, presenting very simple scenarios where users could interact with puppets or avatar while making their way through short and very predictable branched stories.
Not bad, if you consider that the alternative at the time was that of reading through html-based documentation, suitable maybe for collecting information, certainly not for shaping new behaviors.
But still too basic. In real life, we do not interact in such a simple way. Each decision we make and every emotion we express are the results of multiple factors merging, making the possible nuances of our behaviors almost unlimited.
Observe the way we interact in real life: each decision we make and every emotion we express are the results of multiple factors merging, making the possible nuances of our behaviors almost unlimited.
The problem is, even nowadays, the idea of simulating conversations through branched design is so common that each time we present SkillGym to anyone, the most predictable reaction being asked, “What type of branching did you design here?”
So, the real question is: why you should start considering that simulations can evolve into interactive conversational training, to the point where the experience becomes truly authentic?
The Need for Authentic Interaction
Short answer: because branching is not the answer to immersive learning.
We never pondered branching, not even for a second, while designing the SkillGym experience. Branching is too basic, it’s too black-or-white, it’s too much about understanding basic concepts rather that experiencing in a natural way the complexity of reality where internal perceptions and external circumstances continually influence the content and the intentions of the conversational interaction.
Let’s look at some examples to frame what I mean, considering three situations that may happen while you are playing a conversational simulation:
- You are discussing a topic with a fellow colleague. Suddenly you say the wrong thing. It happens all the time. How do you recover? Which strategy do you use?
But, most of all, how will your colleague react after your next interaction? A branched simulation would much probably offer you the possibility to go back on track, proposing a better option for your next phrase.
And that’s OK. However, how does it manage the fact that your fellow colleague may still be not happy with your previous sentence? How does branching deal with the flow of emotions that each of us carries along the conversation and that affect the ongoing dynamics?
- During the conversation, you hesitate to answer. You are trying to choose your best option and time flies, so your answer comes a bit late. Does it affect the quality of the reaction of the other person? Does the other person slightly change the overall mood while waiting? In real life, almost always yes. In branched simulation, no.
- Again, your fellow colleague decides to mark the spoken sentence with a subtle expression of his face. Almost nothing, but meaningful. Will your next sentence be interpreted with influence from this feeling? In real life, of course yes.
In a branched simulation? Most likely, the entire simulation would be designed without those nuances, to avoid having to deal with them when they are expressed instead of when they are not.
- You play the simulation again. It’s training, after all. You are supposed to practice more and more. Do you expect that things develop exactly in the same way?
I mean: at a certain point you decide to say the same thing you said last time. Even if the circumstances are slightly different. What would happen in a branched simulation?
How can you possibly evolve interactive conversational training to the point where the user experience becomes truly authentic?
These examples show how limiting a branched simulation can be, to express all the complexities that a real-life conversation brings to the table. Life is not branched at all. Even if every day we try to reduce it to a mere sequence of left-or-right turns, hoping to make it easier to find the reasons behind people’s behaviors.
These examples are only scratching the surface of the complexity behind a true and authentic representation of human reactions and feeling that get expressed during a conversation.
The picture I chose to open this article is a good representation of our feelings here at SkillGym: nurture new ideas beyond those branched “trees” and come up with something unique that can help users experience a completely new approach to training simulations, facing at least three challenges:
- The flow of the conversation influences the character
- The profile of the characters determines their own reactions
- The user’s approach influences the interaction
1. The flow of the conversation influences the character
The first challenge for developing authentic Digital Role Plays is that of considering how the dynamics of the conversation -happening even before it starts- can influence the emotions and thus the reactions of the character.
In fact, each step is linked to the previous ones and influences those that follow.
In other words, the instant reaction of the character, in terms of verbal, para-verbal and nonverbal communication is continually influenced, whatever you are going to say, by several factors, including:
- Whatever happened to him/her before your conversation started, especially at the very initial steps of the conversation
- What happened so far: how you handled things and how he/she reacted
- What the character expects to happen next based on the ideas from you and of the ongoing conversation quality that is slowly developing inside his/her mind
The first challenge in developing authentic digital role plays is that of considering the continuous flow of emotions that happens during a conversation.
So, for example, you may expect that a conversation that starts is a bad way -maybe due to external factors outside of your control- will be more difficult to manage than one that started with a happy fellow colleague or report.
At the same time, the more you dig into the conversation, the more any reaction of the other person will be somehow influenced by the growing perception built around you.
2. The profile of the characters determines their own reactions
According to the specific type of character -and here at SkillGym we have developed a full set of types, in accordance with the most well-known psychometrics models- other factors influence the way the conversation goes.
Think about:
- The way this character processes information, the way he/she thinks
- The way he/she shows/hides emotions and the way their inner world is expressed when your approach changes
- How inbound and outbound emotional peaks are managed along the conversation, whether they are single peaks or continuing interferences
Each of us is different, and even within ourselves we are a blend of emotions and reactions that can be described according to models, but that eventually must be displayed in real time along the simulation. And this requires considering several variables, all intersecting with one another in real time.
3. The user’s approach influences the interaction
Finally, the user is an active part in the conversation, influencing the ongoing outcome of the meeting through the application or misapplication of:
- A style of communication
- A coherent set of behaviors along the path
- The recognition of any weak signals coming from the character
For example, one trainee could systematically apply a seemingly winning leadership style that is not appropriate to the circumstances. Much often the style has to change according to the development of the context, and recognizing it on-time can make the difference in going north or south.
Also, the attention to weak signals as the elapsed time, the nonverbal nuances of the character in the emotional peaks and other elements that are potentially under the control of the trainees can have an influence on the outcome of each single reaction of the character.
Enter AI
Managing all of the above variables in real time, within a limited canvas of conversational materials -defined as the balance between the user interaction and the character reaction domains’ variables- requires a good deal of technology.
This is where our SkillGym algorithms become a key ingredient of our recipe for authentic and immersive training experience.
The scheme below summarizes what happens during the ongoing conversation between the trainee and a SkillGym character.
- SkillGym Digital Role Play is designed around a canvas of pre-recorded (character’s) video and (user’s) sentences
- These assets are the result of the combination between the variables that define the character’s domain (needs, profile, prequel) and those belonging to the user interaction space (meaningful topics, styles and behaviors)
- Based on the PREQUEL of the character and according to the information available in the overall user’s history, SkillGym Algorithms (SA) decide how to start-up the conversation
- Each single interaction is tracked and stored in real-time and sent to the SA
- At each step of the conversation, the SA decides about the options for interaction available to the user and the relevant reaction of the character–topic, intention and nonverbal
The twelve SAs work together to keep into account all the variables that we have seen above:
A. The conversational factors:
- What happened to the character before the conversation started (“prologue”)
- What happened during the conversation so far
- What is happening now
- The character expectations on the conversation’s outcome
B. The personality factors of the character:
- The way he/she thinks
- The way he/she behaves
- The continuous flow of in/out emotions
C. The interaction approaches available to the user:
- The style of communication
- The underlying behaviors
D. The quality of the interaction (user and circumstances):
- The attention to the character’s weak signal
- The timing of interaction
- Unexpected external factors influencing the context
Managing the Flow of Emotions
Clearly the challenge is that of making the conversation authentic, thus immersive.
SkillGym Algorithms are designed exactly for this purpose and the level of authenticity we reach in representing the character’s flow of emotions allows the user to literally collect a series of “Deja-vu moments” that contribute to the unconscious storing of new experiences.
I have discussed earlier in this article (“8 Key Metrics to Ensure a Successful Practical Training on Critical Conversations”) the subject of Deja-vu, one of the key learning triggers in SkillGym. In short, a Deja-vu is the perception of a situation that you feel like you experienced before and are strangely connected to what is happening now in front of your eyes.
This is not easy to achieve, but very powerful, since it is the equivalent of real-life stored experience that excites the brain much faster and more persistently than any knowledge-based piece of information.
Deja-vu, to pop-up, requires the perception of natural and coherent behaviors as perceived from the way the character acts and communicates, especially through nonverbal signals.
The twelve SkillGym Algorithms are made to measure and deliver this immersive and authentic experience, tuning the entire flow of the conversation in real time based on the multiple triggers we discussed above.
The impact on the user is very effective:
- You have the feeling of dealing with an “alive” character, just as if the person was there, real-time, in front of you
- You have the feeling that time is a key factor during the conversation
- You have the feeling you are really practicing your confidence, self-awareness and self-control by storing a large amount of ready-available new experience
We are constantly working on new ways to make the training more engaging, more realistic and more effective.
I believe that the limits of what we can do are still far off, however the implementation of this technology is a big step ahead already in the delivery of an increasingly authentic experience for our trainees.
I hope you enjoyed this tour of one of our favorite characteristics of SkillGym and hopefully I was able to make it very clear about the tremendous benefits of adopting AI.
You are most welcome to contact us to learn more about this subject and please feel free to book a 1-hour discovery call of SkillGym.
