Keeping up with new technology

Hololens Tech Demo: Pathfinding using Voice Commands

After breaking the ice with this demo, we decided to take it a step further and explore ways that we could provide functionality in larger spaces. The goal of this prototype was to create a system to navigate to a waypoint, and also to empower the user to create points of interest as well. We also wanted to explore voice commands, and we saw this as a great opportunity to accomplish that as well.

One of the first tasks, and an initial challenge when we sat down to plan the prototype, was to research how to use an existing path finding algorithm, called Dijkstra’s Algorithm.  By fully understanding the principles behind this, we were armed with enough information to apply it to our demo.  The alternative – pathfinding through sheer number crunching, evaluating every possible path – may not be feasible on Hololens, if dealing with a large amount of data.  We knew that we not only wanted the best path to be provided, but we also wanted it to update based on the user’s current position.  Both of these are implemented in this prototype.

There are many utility functions available that make setting up a room easy and fun to do. All functions can be triggered by simply saying a command. Here is a list of possible commands:

  • Create – creates a new node
  • Create Orphan – creates a new node without linking it to the previous one
  • Go to [node] – begin path finding to the specified node
  • Idle all – clears your current selection
  • Select all – selects all nodes
  • Tag [name] – applies a label to a selected node
  • Untag – removes the label from a selected node
  • Link – adds a viable path between two or more selected nodes
  • Unlink – removes the path between two or more selected nodes
  • Delete – deletes all selected nodes
  • Pinch – selects a node
  • Grab – allows the selected nodes to follow your view
  • Drop – releases the selected nodes from following your view

There are many practical applications for a solution such as this. Imagine a hospital, where the nursing staff is provided updated information on patient status by simply looking down the hallway. Or perhaps an art gallery experience, where approaching a painting on the wall triggers a hologram of the artist describing the work to you in person. Virtual tours of museums or visitor centers could come alive, with a personal guide discussing the room around you, speaking directly to you through the Hololens’ built-in spatial sound system.

For further reading on this prototype, please refer to Brendon’s project documentation here.

Greg Foxworthy is the Interactive Director at VisionThree. He is responsible for planning and leading the development team in the creation of all of our experiences, along with guiding our R&D efforts.
ResearchJacob Leeman