Masaaki Shibata

The paper reports the experimental results of applications of the visual feedback control to track the high-speed target. The stereo vision robot with high-speed cameras keeps tracking the moving tip of the double pendulum. The double pendulum is well known to behave in chaotic motion, and then has been employed in our experiments in order to evaluate the performance of the visual tracking control on the robot. The frequency characteristics of the double pendulum is analyzed and the sufficientperformance of the robot is confirmed in several experiments and in the frequency analysis.

The paper proposes the way of visual tracking control for the stereo vision robot, which consists of the active vision system. The robot keeps tracking to the moving target in arbitrary motion, with using the high-speed cameras installed on itself. The double pendulum is employed as the moving target in the experiments, which contains wide distribution of frequency in its tip motion, so that it is preferable to evaluate the performance of the visual tracking control. It is important to estimate the target position and velocity accurately for realizing the non-delayed visual tracking to the moving target. The way of using the estimated target position and velocity in the stereo vision robot is detailed, and the performance of the visual tracking control in the proposed method is analyzed in several physical experiments in the paper.

This paper addresses the joint limit avoidance problem in kinematic control of redundant manipulators. A kinematically redundant manipulator has more Degrees of Freedom (DOFs) than are required to perform a given task. An inverse kinematic problem for such a manipulator admits an infinite number of solutions. In its kinematic control, by utilizing the redundant DOFs of the solutions, we can compose a primary task in consideration of a constraint condition. We usually select one of the solutions by introducing a Performance Criterion Function (PCF). The PCF performs a secondary task that has no effect on a primary task. Thus, we can introduce, e.g., joint limit avoidance as a secondary task to the design of a PCF. However, it is hard to perform joint limit avoidance in the case where a joint behaves in the close vicinity of the limit. In this paper, we propose a new PCF for avoiding joint limits. The main feature of the function is to admit the joint range of motion maximally. We adopt the proposed function in the gradient projection method to obtain the redundancy resolution. The effectiveness of the function is demonstrated by both numerical and experimental results. The differences among the proposed and conventional functions are also discussed.

This paper addresses a depth estimation problem for a robotic system with a single camera. For this problem, several depth observers have been proposed so far. Most of them are based on the Persistency-of-Excitation (PE) condition. But, the PE condition is not always satisfied during a visual servoing task. The estimation of the depth observer stops if the PE condition fails. In our previous work [13], such a case has been illustrated via simulations, and also an estimation recovery method based on periodic IBVS has been proposed. In this paper, we evaluate the effectiveness of our proposed method experimentally. On the basis of experimental results, we discuss how the amplitude and the period of the periodic behavior contribute to recovering estimation.

This study aims to achieve the smooth going up and down the step for a biped robot whose leg has 6 DOFs with kinematics redundancy. If COG position is not controlled in the going up and down motion, the robot falls down forward or backward. The robot has achieved the motion of going up the step in the proposed approach by its COG out of shifting the waist position. In our proposed approach, the COG position is moved with the waist position shifting, and then the climbing and down motions have been realized. It is also confirmed that the robot could climb and down the known height step safely and climb down the unknown height step by the waist position shifting and the COG position changing. It is found that the required COG displacements depend on the depth of the step in the case of climbing down. The relationships among the COG displacements and the depth of the step have been made clear in the experimental studies in this paper.

This paper addresses the problem of visual target following for a nonholonomic wheeled mobile robot with a single fixed camera. For the problem, we propose a control scheme composed of vision-based trajectory planning and tracking control. We present online trajectory planning based on image-based visual servoing. The key idea is to compensate the target object motion. In particular, we focus on visual feature motion resulting from the target object motion. Tracking control to the generated trajectory can therefore achieve visual target following precisely even if the target object is moving. Our proposed scheme can be applied to, e.g. robotic systems for autonomous guard and observation. The effectiveness of the control system is evaluated experimentally. (c) 2012 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

This paper addresses an avoidance problem of robot joint limits in visual servoing. Many robotic systems are practically subject to some physical constraints. In particular, violation of joint limits can deteriorate control performance and lead to the breakdown of the system. Redundancy appears frequently in the visual servoing formulation. To this redundancy, the authors introduce the maximal admission function. Consequently, both avoiding joint limits and exploiting joint range of motion effectively are achieved in visual servoing. Here, we consider a moving object as the target of visual servoing. In visual servoing, the case of a moving object is more severe than the case of a stationary object. The effectiveness of the proposed scheme is validated experimentally. The obtained results are also evaluated through a comparison with the conventional schemes. © 2012 The Institute of Electrical Engineers of Japan.

In this paper, we propose a visual tracking control method of a hand-eye robot for a moving target object with multiple feature points. The hand-eye robot is composed of a 3-d.o.f. planar manipulator and a single charge-coupled-device camera that is mounted on the manipulator's end-effector. The control objective is to keep all feature points of the target object around their desired coordinates on the image plane. In many conventional visual servo methods, it is assumed that the target object is static. Consequently, the visual tracking error arises in the case of a moving target object. We have already proposed a visual tracking control system that takes into consideration the target object motion. This method can reduce the visual tracking error, but can only deal with a single feature point. Therefore, this paper extends such a visual tracking control method to multiple feature points. The effectiveness of our control method is evaluated experimentally. (C) Koninklijke Brill NV, Leiden and The Robotics Society of Japan, 2011

The paper presents recent advances of vision-based motion control for robotic systems. The visual feedback loop is very effective for increasing the dexterity and flexibility of robot's task. With the development of vision and computer technology, the visual feedback applications are remarkably expanded for various robotic systems. In the paper, at first, a variety of vision-based approaches in Japan are overviewed from the viewpoint of motion control scheme, and fundamental formulations of visual servo controller are described. And two kinds of visual feedback applications are introduced with several experimental evaluations. The first one is non-delayed visual tracking approach using active stereo vision. It is found that the target motion estimation is an important issue for fast tracking. The other is the motion controller for wheelchair-type mobile robots based on optical flow field. The vision-based feedback controller conducts the force-related motion. Then, the vision makes dynamic environmental change transmitted to human operator through force interaction. The validity of these approaches are evaluated by experimental results. (C) 2009 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

In this paper, we propose an image-based visual tracking control method of a hand-eye robot for a moving target object. The hand-eye robot is constructed from a three-DoF planar manipulator and a single CCD camera that is mounted on the end-effector. This robot is a typical example of an eye-in-hand system with a single camera. The control objective is to keep the target object around the center of the image plane. In many conventional visual servo methods, it is assumed that the target object is static. Consequently, the visual tracking delay arises in the case of a moving target object. Although we have already proposed a non-delayed visual tracking control method for a moving target object, this method is developed only for a stereo vision robot with two CCD cameras. Therefore this paper provides such a visual tracking control method for the hand-eye robot. The validity of our control method is evaluated by an experiment. © 2009 The Institute of Electrical Engineers of Japan.

In this paper, we propose a visual tracking method for a moving object without tracking delay. On visual servoing, an image-based method with image Jacobian is often taken for target-centering camera motion control because it does not require the target's 3D position estimation and is robust to modeling errors and noises. In an ordinary visual servoing approach, it is assumed that the target object would be fixed in the workspace. Therefore, for the purpose of tracking to a moving target, the image errors always exist between the target position and camera optical axis, that is, the visual tracking delay. In our proposed method, the target object's 3D position is estimated with the technique based on triangulation on stereo vision system. Then, the moving velocity is calculated in the camera coordinate frame. The image Jacobian concerns the camera velocity with the target velocity in the image plane. Additionally, the target velocity in the 3D worksplace is also converted into the velocity in the image plane. The converted velocity compensation the camera velocity to compensate the camera motion delay. The approach enables us to achieve nondelayed tracking for the moving target. (C) 2009 Wiley Periodicals, Inc. Electron Comm Jpn, 91(11): 19-27,2008; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecj.10184

In this paper, we propose a visual tracking method for a moving object without tracking delay. On visual servoing, an image-based method with image Jacobian is often taken for target-centering camera motion control because it does not require the target's 3D position estimation and is robust to modeling errors and noises. In an ordinary visual servoing approach, it is assumed that the target object would be fixed in the workspace. Therefore, for the purpose of tracking to a moving target, the image errors always exist between the target position and camera optical axis, that is, the visual tracking delay. In our proposed method, the target object's 3D position is estimated with the technique based on triangulation on stereo vision system. Then, the moving velocity is calculated in the camera coordinate frame. The image Jacobian concerns the camera velocity with the target velocity in the image plane. Additionally, the target velocity in the 3D worksplace is also converted into the velocity in the image plane. The converted velocity compensation the camera velocity to compensate the camera motion delay. The approach enables us to achieve nondelayed tracking for the moving target. © 2009 Wiley Periodicals, Inc.

This paper proposes a method for gait trajectory generation using an artificial vector field for stable walking for a biped robot. The tip of the robot while walking can often deviate from the desired trajectory as a result of the disturbances forced by unexpected outside factors. In our approach, though no prepared trajectory is specified a priori, the tip follows the artificial vectors designed in the workspace. Moreover, the prediction simulation is performed online. The simulator judges the stability under comparison with the present state and the prediction results, and then the gait parameters are adaptively improved in feedforward for the stable walk. The numerical and physical experimental results show the validity of the proposed method in a continuous walk. (C) 2007 Wiley Periodicals, Inc.

The paper describes a novel method for visual tracking of a moving object with compensating tracking delay, Many of conventional visual feedback techniques based on image-based control law dealt with static target object, while the moving objects are tracked with the techniques based on position-based one. In case to track a moving target with the image-based controller, the tracking delay should occur, so that it must be suppressed in some methods. In our approach, the image-based control law is simply improved and then the source of the tracking delay is compensated with the estimated target speed in camera coordinate frame.

In this paper, we propose a visual tracking method for a moving object without tracking delay. On visual servoing, image-based method with image Jacobian is often taken for target-centering camera motion control because it does not require target's 3D position estimation and is robust to modeling errors and noises. In an ordinary visual servoing approach, it is assumed that the target object would be fixed in the workspace. Therefore, for the purpose of tracking to a moving target, the image errors always exist between the target position and camera optical axis, that is, the visual tracking delay. In our proposed method, the target object's 3D position is estimated with the technique based on triangulation on stereo vision system. Then, the moving velocity is calculated in the camera coordinate frame. The image Jacobian concerns the camera velocity with the target velocity in the image plane. Additionally, the target velocity in 3D workspace is also converted into the velocity in the image plane. The converted velocity compensates the camera velocity to compensate the camera motion delay. This approach enables to achieve non-delayed tracking for moving target.

This paper proposes a visual servo control method for exploring motion of eye-in-hand robot to recognize a three-dimensional object. Optimal motion is based on the strategy in which the position estimation precision of the feature points gets improved when the feature points move widely, and the optimal robot motion provides the widest transitions of the feature points per unit robot motion, so that the robot should be controlled to purse such motion. In addition, this paper describes the way to reduce a large number of calculation efforts. The validity of our approach is confirmed in the several experimental results.

In this paper, we propose a gait control method for redundant legged biped robot based on leg center of mass (COM) position control at constant velocity and constant height of the waist. The developed biped robot has redundant legs, which have four degrees of freedoms (DOFs) on each in the sagittal plane. The redundant DOF enables movement of its leg tip position and its leg COM position independently. Therefore, the proposed robot has structural capability to control the leg COM position for maintaining the projection of the total COM inside the support polygon without upper body motion. Such capability enables the stable static walk in the arbitrarily desired velocity and height of the waist motion. The validity of the proposed method for the static walk at constant velocity and constant height of the waist in the leg COM position control for redundant legged biped robot is confirmed by several results of simulation and experiment. (C) 2006 Wiley Periodicals, Inc.

The paper proposes a novel method of visual tracking for moving object without camera calibration. An active stereo vision robot tracks a moving target with image-based visual tracking. The target velocity in the 3D workspace is estimated in the camera coordinate frame, and the estimation contributes to the suppression of tracking delay, so that no steady-state error is achieved in visual tracking. The tracking delay is fully suppressed, less than 10 [pixel], and then the camera calibration is theoretically and experimentally eliminated in the proposed method. The validity of the proposed approach is confirmed in the physical experimental results.

This paper proposes a visual servo control method for exploring motion of eye-in-hand robot to recognize a three-dimensional object. The CCD camera attached at the manipulator tip captures the images of the target object, and then the robot varies its own posture with referring the position transitions in the images of the feature points on the object. Our approach for optimal motion is based on the strategy in which the position estimation precision of the feature points in the captured images gets improved when the feature points move widely, and the optimal robot motion provides the widest transitions of the feature points per unit robot motion, so that the robot should be controlled to purse such motion. In the proposed method, the primary feature point is made located at the center of the image, while the other points are moved for achieving the optimal transition in the image. In addition, this paper describes the way to reduce calculation effort for the proposed method. The numerical and physical experiments have been executed with use of 3-DOF planar manipulator. The validity of our approach is confirmed in the several experimental results.

This paper proposes a way of gait trajectory generation with artificial vector field for stable walking of a biped robot. The tip of the robot on walking can often deviate from the desired trajectory by the disturbances forced by unexpected outside factors. In our approach, though no prepared trajectory is specified a priori, the tip follows the artificial vectors designed in the workspace. Moreover, the prediction simulation is performed on-line. The simulator judges the stability under comparison with the present state and the prediction results, and then the gait parameters are adaptively improved in feedforward for the stable walk. The numerical and physical experimental results show the validity of the proposed method in the continuous walk.

In this paper, we propose a gait control method for redundant legged biped robot based on leg center of mass (COM) position control at constant velocity and constant height of the waist. The developed biped robot has redundant legs, which have 4 degree-of-freedoms (DOFs) on each in the saggital plane. The redundant DOF enables to move its leg tip position and its leg COM position independently. Therefore proposed robot has structural capability to control the leg COM position for keeping the projection of the total COM inside the support polygon without upper body motion. Such capability enables the stable static walk in the arbitrarily desired velocity and height of the waist motion. The validity of the proposed method for the static walk at constant velocity and constant height of the waist in the leg COM position control for redundant legged biped robot is confirmed by several results of simulation and experiment.© 2005, The Institute of Electrical Engineers of Japan. All rights reserved.

This This paper describes a control method for moving object tracking on an active stereo vision robot. Our developed robot has 3 rotational axes; two are for camera-pan motion and one for base rotation, and then has structural redundancy to track the target object. The aim of our approach is to achieve both requirements of rigid tracking and accurate position estimating, and then the characteristics of each rotational axis are considered form the viewpoint of dynamics. Since the base axis has heavier inertia than the camera axes, the camera-pan motion is preferable to realize rigid tracking in the high gain servoing. On the other hand, the base is required to face to the target to prevent the position estimation accuracy from getting worse, so that the base would turn to the target in the slow response. The former motion is realized in the image-base visual servo control law, while the latter in the mechanical impedance control law, and then these control laws are unified in the proposed method. Additionally, the Kalman filter is introduced for prediction of the target motion. The filter contributes to more precise target position estimation. The validity of our method is confirmed in the physical experimental results.

This This paper describes a control method for moving object tracking on an active stereo vision robot. Our developed robot has 3 rotational axes; two are for camera-pan motion and one for base rotation, and then has structural redundancy to track the target object. The aim of our approach is to achieve both requirements of rigid tracking and accurate position estimating, and then the characteristics of each rotational axis are considered form the viewpoint of dynamics. Since the base axis has heavier inertia than the camera axes, the camera-pan motion is preferable to realize rigid tracking in the high gain servoing. On the other hand, the base is required to face to the target to prevent the position estimation accuracy from getting worse, so that the base would turn to the target in the slow response. The former motion is realized in the image-base visual servo control law, while the latter in the mechanical impedance control law, and then these control laws are unified in the proposed method. Additionally, the Kalman filter is introduced for prediction of the target motion. The filter contributes to more precise target position estimation. The validity of our method is confirmed in the physical experimental results.

The paper describes a control technique for an active stereo vision robot. For the purpose of achieving both rigid tracking and accurate estimating, the proposed approach unifies the image-based high gain PD control law for tracking and the mechanical impedance control law for stable motion of the base axis. © 2005, The Institute of Electrical Engineers of Japan. All rights reserved.

The paper describes a control technique for an active stereo vision robot. For the purpose of achieving both rigid tracking and accurate estimating, the proposed approach unifies the image-based high gain PD control law for tracking and the mechanical impedance control law for stable motion of the base axis. © 2005, The Institute of Electrical Engineers of Japan. All rights reserved.

The paper describes an approach for applying advanced image differential operation, for purposing to realize the visual servo system based on the advanced image processing. The conventional and ordinary visual servo system, in general, has employed binarized image processing approach, and/or preferred the simple operators so as to suppress the image processing time. On the other hand, the precise object recognition system has introduced the advanced image processing operators and filters which require much more processing time.© 2004, The Institute of Electrical Engineers of Japan. All rights reserved.

In this paper, we propose a novel approach for stepping forward action of biped robot with minimum kicking force. In walking or starting to walk, the general biped robot kicks the ground with its own hind leg as well as human being does. The condition of the ground often restricts the gait because of stiffness, slipperiness and so on. In order to surmount the difficulty on stepping forward action, we introduce the redundant legged biped robot, which has 4 degree-of-freedoms on each leg. Our robot enables to move its tip position and its center of mass (COM) position independently. Controlling COM acceleration without moving its tip position realizes the stepping forward action with little kicking force. The physical experimental results show the significant validity of the proposed approach. © 2004, The Institute of Electrical Engineers of Japan. All rights reserved.

A swinging leg of a biped robot landing, impact force usually occurs between the sole and the ground, and then it causes instability of the gait. The paper describes the advantages of adopting redundant legs to the robot in order to conquer the difficulty, ancl proposes a novel way of the motion control for the redundant legged biped robot. Tn general, each leg of a conventional biped robot consists of 3 joints, namely, hip, knee and ankle in the sagittal plane. On the other hand, the proposed robot has been added extra joints, and thereby has redimdancy in terms of degrees-of-freedom. Since the redundant leg can select its arbitrary posture, regardless of the tip position, the structure enables to move the position of the center of mass (COM) of the leg independently. The impact force is suppressed by controlling the COM acceleration of the landing leg. In order to achieve the decoupled motions between the tip and the COM, the unified decoupling controller is introduced. The controller includes three types of the disturbance observers together, and both desired motions are realized consequently. The validity of the proposed approach is confirmed in physical experimental results. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

The paper describes proper initial posture of the swinging leg of a redundant legged biped robot for impact force suppression in landing onto the ground. The proposed robot has structural advantage for absorbing the impact because of its own redundancy of posture, and then, the proper posture of the landing leg contributes the further suppression. The validity of the proposed approach is confirmed in physical experimental results. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

The paper reports a structural approach to reduce impact force in landing the swinging leg of biped robot. The robot introduces redundant legs in the sagital plane, which have redundancy in terms of DOF that is, extra joints besides hip, knee and ankle in each leg. The leg independently changes the posture, regardless of the tip position, while the center of mass (COM) of the leg moves to an arbitrary position even if the tip is fixed. Since the position of COM moves in landing the swinging leg toward the ground, therefore the impact force is reduced. The validity of the proposed approach is confirmed in physical experiments. © 2002, The Institute of Electrical Engineers of Japan. All rights reserved.

The paper reports a structural approach to reduce impact force in landing the swinging leg of biped robot. The robot introduces redundant legs in the sagital plane, which have redundancy in terms of DOF that is, extra joints besides hip, knee and ankle in each leg. The leg independently changes the posture, regardless of the tip position, while the center of mass (COM) of the leg moves to an arbitrary position even if the tip is fixed. Since the position of COM moves in landing the swinging leg toward the ground, therefore the impact force is reduced. The validity of the proposed approach is confirmed in physical experiments. © 2002, The Institute of Electrical Engineers of Japan. All rights reserved.

The paper descries a novel method to extract line segments and circular arcs from the image taken by CCD camera for object recognition. Model based matching is one of the widely used techniques of the object recognition in computer vision, and the position and orientation of the objects are decided with referring the extracted line segments and circular arcs. For the precise and stable decisions of the position and orientation, the accurate extractions of characters are required. These characters are. fast and accurately extracted in the proposed method, while its validity is confirmed in some experiments.