Hi everyone! Welcome to the first installment of the *new!* Floreo Engineering Corner: a column of the newsletter in which an engineer writes about topics related to the engineering behind Floreo’s app.

I’m Elena, a software engineer who’s been working at Floreo since August 2022. I’m highlighting how the “gaze fixation” feature is used in the Cross the Street By Yourself lesson. In this lesson, gaze fixation is by far the most important game-play mechanism.

“Gaze fixation” refers to instances in which the lesson is responsive to where the Learner is looking. For example, if a Learner looks at a Fish in Find the Fish, then a musical tone sounds and the lesson moves forward. The lesson’s logic is based on detecting where the Learner’s gaze is directed, and ascertaining whether the Learner has “found the fish” based on that. In a headset without hand controllers, it is an important way for the Learner to move through the lesson.

There are three different use cases for this mechanism in Cross the Street By Yourself.

First, gaze fixation is used to simulate cases in which Learners are literally expected to direct their gazes in certain directions. In Cross the Street By Yourself, this mechanism is really important because it is used to verify that the Learner has looked left, right, and at the cross signal before crossing the street. The lesson only progresses once these actions have happened. These are the literal cases in which Learners may also direct their gazes this way in real life to make sure that cars aren’t coming.

However, gaze fixation also is used symbolically as a way for Learners to interact with items. In the Teamwork lessons (Communication, Cooperative Work, and Collaboration & Compromise) Learners pick up and set down cans by looking at them. In the Time Management lessons (Multitasking, Organizational Habits, and Interruptions and Distractions), Learners sign into work on the computer by looking at it. And, in the Cross the Street by Yourself lesson, Learners may “press the crosswalk button” just by directing their gazes towards the crosswalk button.

The final use case for gaze fixation in Cross the Street by Yourself is for lesson progression. In the city scene, the Floreo character Emma is waiting for the Learner across the street. In order to motivate and provide context for why the Learner wants to cross the street, Emma says “Hey there!” and invites the Learner to her side of the street. However, before Emma says this, the Learner must find the character first using gaze fixation so that it doesn’t appear that the voice comes out of nowhere.

One of the continual engineering efforts at Floreo is making sure that the game-play mechanisms used in our lessons work accurately. For gaze fixation in particular, that means making sure that it is accurate and intuitive—and not difficult or frustrating.

When building lessons with gaze fixation, we pay attention to the placement and sizing of objects that serve as “gaze fixation targets.” Objects that are far away present a challenge because it is difficult to accurately look at the object. Instead of just using the object as the “gaze fixation target,” we create an invisible fixation target that is sized appropriately for the object the Learner is supposed to look at. The invisible fixation target is often centered around the original object but is larger than it, so that when the Learner looks towards it, the lesson gives credit accordingly.

Another important way to ensure that the game-play mechanisms are effective is by updating functionality and introducing new features. One of my co-workers recently introduced a “progress timer” (currently in-use in the Teamwork lessons) to ensure that Learners receive visual feedback on their gaze fixation actions. It appears in the lesson as a progressively-loading green circle over the object the Learner is expected to look at.

What has your experience been with gaze fixation in Floreo lessons? Do you have any ideas for new features? We love hearing from Floreo users about experiences, suggestions, and feedback!