Current Research (4)

Biomechanics of Human Movements

sDIMS is the fundamental software originally developed by Dr. Katsu Yamane (Disney Research) and includes kinematics, dynamics, and optimization libraries for complex mechanisms. The dynamics engine for simulation (Forward Dynamics computation, FD) is based on Dr. Yamane’s Ph.D study (2002) known as Assembly-and-Disassembly Algorithm, which performs FD in the order of N (number of bodies) for single thread computation and in the order of log(N) for multi-threads parallel computation. The earlier version of the engine was used in OpenHRP and released from AIST. Ichiro Suzuki  in 2002 modeled the wholebody musculoskeletal system, as a rigid-body system of over 150 degrees-of-freedom connected by nearly 1000 massless wires representing muscles, tendons, ligaments, and cartilages. The model is represented in the format for computation by sDIMS. Yusuke Fujita in 2003 worked on estimation of the wholebody muscle tensions from the information of motion capture, a set (typically 16) of wireless EMG sensors, and force plates (6 axes force/torque sensing for each) based on Inverse Dynamics (ID) computation and optimization in sDIMS. The assumptions were that the mass properties are attributed to only bones and that tensions of the massless wires generate the motion of bones in accordance with signals of the force plates and the EMG sensors. The musculoskeletal model and the optimization for muscle-tension estimation are now included in sDIMS.

The biomechanical studies started using sDIMS. Dr. Yamane analyzed the motion data of patients of cerebellar atrophy. Prof. Gentiane Venture (Tokyo University of Agriculture and Technology) studied on identification of joint visco-elasticity of patients of neuromuscular disease.  The both studies were done in collaboration with Dr. Tomotaka Yamamato (University of Tokyo Hospital).

The estimation of muscle tension a priori assumes that the mass properties (such as mass, the center of mass, and the moments of inertia) of body-segments are known. We have developed two solutions to this problem. The first solution is to estimate the body shape first. Then, the mass properties of body segments are computed assuming a constant density. The body shape was estimated by Principal Component Analysis of the statistical data of Japanese population, which is published by AIST, based on which we can estimate the body shape from a few measurements of length. The second solution is based on identification theory developed in robotics. Prof. Venture’s Ph.D study in Ecole Centrale de Nantes was on the subject. She started to apply the theory for human body-segment identification. The main difference between robots and humans is in the fact that the joint torques are not measured for humans. The question is whether we can identify the wholebody mass properties only from motion data (position, velocity, and acceleration) and contact force data measured from force plates on the floor, which implies to use only partial (six) equations from the whole equations of motion (N+6). The identifiablity was proved by Prof. Venture and Dr. Ko Ayusawa (AIST). It was a chapter of Dr. Ayusawa’s Ph.D dissertation. Dr. Ayusawa extended the idea and developed the method to identify physically consistent mass parameter in his dissertation. It is noteworthy that the mass properties are identified only from motion data (without contact force) when a subject is jumping in the air. In this case, we have to measure the total mass separately. Although there is such a technical problem to resolve that the identification of small segments has low accuracy, it is the only method to measure the subject-specific mass properties of body segments.

The digitized model of wholebody skeleton was made by Dr. Akihiko Murai (AIST) in collaboration with AIST and University Museum of University of Tokyo, from a specimen of the museum. The digitized model is released and available from AIST.

Based on the above mentioned developments we have had opportunities to analyze body skills of athletes and performers, some results of which were shown in Miracle Body and Einstein’s Eyes (TV programs of NHK) and the other occasions. The figures are from the recent movies produced by overlaying the results of analysis on the video image. The muscle activities are shown by color gradation, for examples, green (0%), yellow (50%) and red (100%). The left figure is two soccer players, where an offender’s skill for carrying a ball through a defender is analyzed. The offender in the left is Andres Iniesta Lujan of FC Barcelona. The measurements were done in 2013 by Dr. Murai and Kazunari Takeichi in Barcelona as a part of field shooting by NHK crew, at a laboratory of Technical University of Catalonia. We carried some of our equipments to Barcelone. The right figure shows analysis of the performance of Noh with three musicians. The performer is Keisuke Shiozu, a Noh player of the Kita school, playing Ranbyoshi of Dojyoji. The musicians’ voice and sound of instruments were measured independently in synchronization with the performer’s motion. The minimalism of Noh requests a performer to hide the anticipation of movements. We investigate to find what body skills make the abstract expression of minimalism possible. The figure in the center shows the analysis of a double-leg circle on a pommel horse, a typical skill of gymnastic performance on a pommel horse. The measurements were conducted in collaboration with Prof. Kyoji Yamawaki (Gifu University, a Bronze medalist of Los Angeles Olympic Game), Dr. Motoyuki Nawa (Rakusho Gymnastic Club), and Prof. Kazuie Nishiwaki (Ritsumeikan University). According to the experts, the skills on a pommel horse need further mechanical and biomechanical investigation to establish the scientific training program. The measurements of the figures in the right and the center are conducted in Cyber Behavior Studio (University of Tokyo).

The movies overlaying muscle activities like the figures above were made off-line. In 2009 Dr. Murai and Kosuke Kurosaki used simplified optimization and pipe-lined image generation in multi-thread programing to show the overlaid image at the rate of 15 frames per second. Dr. Ayusawa in 2010 solved the original large optimization by using an efficient LCP (Linear Complementarity Problem) solver and showed the overlaid image at the rate of 30 frames per second, which is still lower than the highest measurement rate (200 Hz) at Cyber Behavior Studio, but sufficient for video rendering. The technology is called Magic Mirror, after the mirror telling the truth to the Queen in Cinderella story.