This project aims to enable an operator to more intuitively communicate with a 6-axis, ABB robotic arm more intuitively so that the robot can easily recognize and perform the procedures it has been assigned. By harnessing the power of voice commands, an operator can more easily interact with an industrial instrument.
To achieve this, my colleagues and I began modeling the setup, including the structure's components, such as the table, platform, and saw. I then undertook the next steps independently. Procedures were then broken down into simpler tasks, which were translated into a syntax suitable for Machina Bridge, an app that allows online connections to industrial robotic arms. To facilitate clear instructions, procedures are labeled, allowing the operator to easily communicate with the system.
The key feature of this project is the use of a machine learning model to recognize and interpret spoken labels. By training these models on the sound of background noise and the corresponding numbered procedures, the system becomes attuned to the operator's voice commands. This enhances the operator's ability to seamlessly convey their intentions to the robotic arm.
This project achieves its goals by piecing together many separate tools. A machine learning model into p5js, a versatile JavaScript library. In turn, p5js connects to Machina Bridge, an application that allows online connections to industrial robotic arms. Notably, the software tools employed in this project play a vital role in its success. Rhino3D serves as a robust modeling platform, enabling precise design of the structure and its components. Teachable Machine provides the framework for training the machine learning models on the voice, while p5Js integrates the ML model into the project workflow. Finally, Machina Bridge is the conduit connecting the operator's commands to the robotic arm.
Through comprehensive procedure planning, the implementation of machine learning algorithms, and robotic simulation, this project successfully achieves a pipeline that enables operators to communicate their intentions using voice commands. This project serves to speculate new possibilities for more natural interactions in the realm of human-robot collaboration.