Voice Interactive Games

Document Type

Conference Proceeding

Publication Date


Publication Title

2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference



First page number:


Last page number:



Computer games are being controlled by joysticks or the keys of the computer keyboard, especially the four arrow keys, or the WASD, or both, and other keys of the keyboard. In this research paper we describe a new algorithm that replaces control with joystick or the arrow keys with voice commands. Thus, we replace the left arrow key with the voice command “left”, the right arrow key with the voice command “right”, the up-arrow key with the voice command “up”, and the down arrow key with the voice command “down”. In order to do that we first develop a new convolutional neural network architecture, then we teach the architecture how to recognize the words “lef”, “right”, “up”, “down”, with extremely high accuracy and very low probability of misclassification. Once the Convolutional Neural Network (CNN) is taught to recognize these words, we use the feedforward part of the network in our game programs so that they can capture, real time, the voice input commands of the player and play the game. The advantage of using the voice commands is that for many players it is easier, faster, eliminates the chance of pushing the wrong key by mistake, and provides a better player experience. We also present a pong game in Unity where the paddle controller uses our algorithm to control the two paddles of the game.


CNN; Voice recognition; Voiced and unvoiced Part of the voice signals; Deep learning; Machine learning


Game Design | Other Computer Sciences

UNLV article access

Find in your library