We conduct research to bring xR (VR/AR/MR), UX/UI, and sustainability (SDGs) to the real world, leading to next-generation technologies. These broadly link ergonomics, digital transformation, information engineering, educational technology, and manufacturing (electronic and mechanical engineering) to create things that are useful to people. Through research, you will acquire skills that integrate image processing, mechatronics, CG, sensing, and machine learning. Currently, we are focusing on the development of xR (especially VR/AR) and research that combines xR and machine learning. Boldly challenge yourself in a new world without fear of failure.

This study aims to develop a PC file (Office and programming) creation tool using augmented reality (AR) technology and graphic block manipulation. The promotion of digital transformation (DX) requires the creation of human resources skilled in software and electronic data creation, maintenance, and management. However, the difficulty of keyboard and mouse operation presents a barrier, leading to an increasing number of people struggling with complex and detailed command operations such as creating shapes and entering text into shapes. By combining AR technology with graphic block manipulation, we developed a tool that quickly and easily creates files using intuitive operations of physical block placement and voice input. By elucidating user creation trends using AR block manipulation, the tool functions required for creation, and the effectiveness of a step-by-step learning method, we aim to clarify the possibility of adapting AR block manipulation to PC operations. This research contributes to the visualization of PC operation UI, and by applying these findings to PC skills education and other areas, we hope to develop human resources to support digital transformation. Recently, we have also been working on developing tactile programming diagrams for the visually impaired (known medically as "totally blind").

When communicating the movements of an expert to a beginner, words and gestures alone are difficult to convey, making it difficult to quantitatively evaluate superior skills. In this study, we convert the movement trajectories of multiple joints into curved surfaces (behavior surfaces) to clarify the differences in the movements of experts and beginners. The behavior surfaces are evaluated using factors such as surface shape, curvature, and area, and the curvature is expressed visually using a color gradient. To visualize the rhythm and timing of movements, we create behavior surfaces that add not only the trajectory but also speed, acceleration, and propulsive force information using a color gradient, and use them to evaluate the movements of experts. Currently, in addition to sports movements, we are conducting joint research with the Hamamatsu Innovation Promotion Organization and Fujimoto Kogyo Co., Ltd. to target the transfer of skills in artisan movements such as deburring. We are also attempting to convert finger movements into curved or three-dimensional movements.

Skilled motion training in vocational training often requires learning by watching and imitating the movements of experts, which requires equipment, time, and manpower. Simulations such as flight simulators offer a solution to this problem, but their high level of reproducibility requires specialized equipment, and their training effectiveness is unclear due to their excessive emphasis on realism. Therefore, in this study, we focus on entertainment value by adding a competitive scoring system and developing a motion training game with multiple viewpoints that would be impossible in reality. Previously, we proposed games for lathe centering, milling parallel alignment, and arc welding, and evaluated the effectiveness of motion training games by comparing them with real-world vocational training movements. We are currently promoting the gamification of vocational movements in the food service industry and the promotion of tidying and organizing.