日浦 慎作

コンピュータビジョンにおいて,画像中の処理対象物体の領域を抽出する手段として背景差分法が広く用いられているが,単純な背景差分法は背景の変化に対応できない.そこで多くの拡張がなされているが,背景物体の動きに再現性がある場合についてはあらかじめ得た背景画像列を固有空間により表現し,入力画像を固有空間に射影することで侵入物体の領域と背景画像とを同時推定する手法が提案されている.ここで侵入物体の領域は,画像修復問題における欠損領域と同様に扱われる.一方,固有空間を用いた欠損画像推定では,カーネルトリックを用いた非線形部分空間を利用する手法が提案されている.ところが,この手法では欠損領域が変化するとカーネル行列を再計算する必要があるため,侵入物体の検出のように,欠損領域が未知である場合には計算量が膨大になるという問題がある.そこで本論文ではカーネルトリックと繰返し投影により背景画像と未知物体領域を高速に同時推定する手法を提案する.

シーンへの入射光分布と物体表面の輝度との関係であるライトトランスポートは照明シミュレーション等に有用であるが，その計測には長時間を要し，データ量も膨大である．そこで本論文では，光源からの光がシーン内で最初に反射した結果観測される直接反射成分と，シーン内で複数回反射した結果観測される間接反射成分を分けて計測することで高速化する手法を提案する．まず，プロジェクタとカメラの各画素の幾何的な対応関係を取得し，それに基づき並列的に直接成分を計測する．また間接成分については，その空間周波数が低周波数成分に限定されることを利用して高速化を図る．Light transport is one of the useful representations of complex optical phenomena inside a scene. However, it is difficult to obtain enough sample under various lighting condition with large degrees of freedom. Therefore, in this paper, we propose a fast and efficient sampling method using the characteristics of direct and indirect components of reflected light. At first, we obtain the geometric relationships between projector and camera, then repetive dot pattern is projected to the scene for acquiring direct reflection component. Finally, stripe pattern in a small rectangular is used to measure the indirect light distribution spread within the low-frequency domain.

In computer vision, background subtraction method is widely used to extract a changing region in a scene. However, it is difficult to simply apply this method to a scene with moving background object, because such object may be extracted as a changing region. Therefore, a method has been proposed to estimate both current background image and occluding object region simultaneously by using eigenspace-based background representation. On the other hand, image completion method using eigenspace have been extended to non-linear sub-space using kernel trick, however such existing method takes large computational cost. Therefore, in this paper, we propose a method for rapid simultaneous estimation of a background image and occluded region in non-linear space, using the kernel trick and iterative projection.

本論文では,三次元空間中に任意に設定した領域への侵入を複数の未校正カメラによって実時間で検知するシステムを提案する.このシステムは三次元形状復元手法の一つである視体積交差法に基づいているが,平面間透視投影を用いた限定的な形状復元を行うため検知にかかわるすべての処理を二次元空間で行うことができ,三次元空間中で完全な形状復元を行う手法に比べて効率的に物体の侵入を検知することが可能である.また,複数カメラによる形状計測において校正は避けて通ることのできない問題であり,基準物体や複雑な手順を要求される場合が多く,専門知識をもたないユーザは行うことが困難な作業である.このため,提案システムではユーザがマーカを用いて大まかに入力した点群を射影復元し,その凸包を侵入禁止領域として自動生成することにより,校正作業を不要にした.したがって,ユーザは個々のカメラの仕様や配置を意識することなく直観的に侵入禁止領域を設定することが可能である.

シーンの三次元モデルから生成したCG画像と入力画像の差の最小化によりシーンに対するカメラの位置・姿勢の変化を追跡し,同時にステレオカメラによりシーンの三次元形状を求めることで,自己位置と環境マップの同時推定(SLAM)を行う手法について述べる.ステレオカメラによる距離画像取得処理計算は濃淡画像入力に比べ著しくCPU負荷が高いため処理サイクルが不均一となりがちで,この間に追跡が不安定になったり,シーンの三次元形状の位置合せ精度が低下するおそれがある.そこで本論文ではモデリングと追跡の2種の処理を並列化し,加えてステレオ計算中の運動追跡処理を改良することによって追跡の安定度と位置合せ精度を向上させる方法について述べる.実験ではシーンモデルの更新の有無にかかわらず高精度に追跡とモデリングが同時実行できることが確認された.

We propose a system with camera and projector to show specular reflection of surrounding environment on the real object. At first, active shape measurement with pattern light projection is performed to obtain precise shape of the object rapidly. Then, rendered specular pattern using the shape information is projected on the real object. Our system also recognizes the status of work operated by the designer, and shape measurement and visualization are done only when the worker wants to evaluate the shape. The view point of the evaluator is measured using LED marker and spherical mirror, and the projected reflection pattern is adjusted for the viewpoint adequately in realtime.

In this paper, we propose a practical intrusion detection system using uncalibrated multiple cameras. Our algorithm combines the contour based multi-planar visual hull method and a projective reconstruction method. To set up the detection system, no advance knowledge or calibration is necessary. A user can specify points in the scene directly with a simple colored marker, and the system automatically generates a restricted area as the convex hull of all specified points. To detect an intrusion, the system computes intersections of an object and each sensitive plane, which is the boundary of the restricted area, by projecting an object silhouette from each image to the sensitive plane using 2D homography. When an object exceeds one sensitive plane, the projected silhouettes from all cameras must have some common regions. Therefore, the system can detect intrusion by any object with an arbitrary shape without reconstruction of the 3D shape of the object.

There has recently been much research into real-time tracking of object location and orientation tracking. However, many of the current methods use explicit image features such as corners, edges or silhouettes, etc. and are, therefore, not well suited to tracking curved and smooth objects from which it is difficult to extract features. On the other hand, with methods that attempt to estimate the movement of an object from the optical flow obtained from the pixel intensity gradient, it is difficult to avoid tracking errors accumulated during tracking. Therefore, in this paper we propose a method for estimating the movement of an object with six degrees of freedom without extracting features from the images. In particular, the difference between real images and CG (computer graphics) images is minimized based on the principle of intensity gradient images, CG images can be generated using an object model which consists of shape and color information with location and orientation parameters of the object. Here we show that by using images obtained simultaneously from multiple cameras at different angles, the stability of this tracking method is improved. We present experiments in which we construct an object tracking system that uses multiple cameras and show that we can perform object tracking with six degrees of freedom at a speed of approximately five frames per second for a rigid and arbitrarily curved object from which it is difficult to extract features. © 2006 Wiley Periodicals, Inc.

コンピュータビジョンにおいて,画像中の処理対象領域を抽出する手段として背景差分法が広く用いられている.背景差分法は,あらかじめ背景となる画像を取得しておき入力画像との差分を求めることにより,背景から大きく変化した領域を処理対象領域として抽出する手法であるため,背景の変化に対応することが難しい.これに対応する手段として背景画像の更新によるものが多く提案されているが,一定の条件を満たせば新規物体を背景画像の一部と自由に置き換えられるため,背景の確からしさを保証できない.そこで本論文では固有空間を用いて動的背景の画像列を学習し,学習段階に存在しなかった未知物体領域を実時間で抽出する手法を提案する.提案手法は単純な固有空間への投影と逆投影によって構成されるが,従来手法であるBPLPによる解へ収束することを示した.また,未知物体領域と背景画像を同時推定する場合,従来手法より高速かつ高精度に推定することができる.

We have been engaged in the researches on the active lighting techniques for twenty years. At first, we proposed a method to measure the 3-D shape of the object by projecting multiple patterns of structured light, because it is much faster than the method with scanning slit of light, and dense range image can be measured. For this purpose, we made a projector which consists of mechanically-changeable masks, but it was soon replaced by a specially-made liquid crystal panel with many slit segments. From the fact, we can proudly declare that the projector is not a diverted device but exclusively developed as a part of projector-camera systems, and the projection target is originally not a planar screen but a real scene with varied depth. Actually, this infant device was successful and commercially enough practical for 3-D shape measurement, but unfortunately we had to wait for the evolution of panel display technology more than a decade, to make a step to the studies which utilizes full duality between cameras and projectors. In the middle of 1990s, many researchers started using video projectors. Since we were computer vision researchers, we used the flexibility of video projector not for a interaction technique but improving performance of 3-D measurement such as motion tracking. But we were also aware of its advantage as a visualization tool in a real scene. Alike to the other research activities, we have confirmed that the projector can change the seen color, texture and specular of the real object. In addition, faked shade projected on the object make it possible to change our interpretation of its shape. Furthermore, the traditional shape measurement method we had developed is seamlessly combined with the visualization technique. Therefore, we have a common view to the possibility of intuitive interface systems with realistic visualization for communicating, training and designing works, so now is a good time to looking back on the history of projector-camera systems for our community. Of course you are already familiar with most researches in this field, but we regret our less presence if you are not aware of our research activities. Therefore, in this paper, we summarize the history and progress on light projection methodologies in our laboratory. © 2006 IEEE.

There are many methods to extract moving objects in a scene using background subtraction. However, most methods assume that there are no moving objects except intruders in the observing space. In this paper, we propose the iterative optimal projection method to estimate a varied background in real time from a dynamic scene with intruders. At first, background images are collected for a while, because we assume that the motion of background is well known. Then, the background images are compressed using eigenspace method to form a database. While monitoring the scene, new image is taken by a camera, and the image is projected onto the eigenspace to estimate the background. But however the estimated image is much affected by the intruders, so the intruder region is calculated by using background subtraction with former estimated background to exclude the region from the projection. Thus the image whose intruder region is replaced by the former background is projected to eigenspace and we have updated background. We proved that the cycle converges to a correct background image and we confirmed we can calculate the right region of the object through some experiments.

In this paper we propose an appearance-based tracking method without any feature extraction from the images. At first, range-finder system or stereo cameras measure the shape of the incoming object to prepare a CG model of the object. Then many images are generated from the model and compared with input images captured by the cameras. Motion parameter of the object is modified to minimize the difference between generated and captured images, and the parameter tracks the motion of the real object precisely when minimization is converged. At the image generating stage, occlusion is also simulated and we can easily handle the visible region of the object.

This paper introduced a user interface based on virtual shadow derived by projector for Spatial Augmented Reality (SAR) environment. Shadow is a daily phenomenon in our daily life and may contribute to build an effective and intuitive connection between physical and digital world. Taking advantage of spatial and optical characteristics of shadow, user can realize remote interaction for ubiquitous interface. The prototype system is implemented and adaptive image processing algorithms for shadow are proposed. The authors has demonstrated in this paper that shadow has possibility to realize simple but effective interface between human and computer systems, which may yield to many useful applications without massive device resources. © 2006 ICASE.

A method for 3D motion tracking without feature extraction is necessary for monitoring human action in a normal civil-life scene. To create a fast and robust object tracking method, we propose a model-based method using intensity images taken with a multiple viewpoint camera connected to a PC cluster system. First, the whole 3D shape and reflectance model of the object are prepared using several rangefinders. Each rangefinder is constructed with a camera, projector and PC, and all PCs are connected via network to each other. For tracking the object, several CG images with varied object pose and position are generated in each PC using the object model, and then compared to the input intensity image in parallel. The result of the comparison is transferred to a master PC, and the pose and position of the object are estimated by minimizing the residual of the CG and input images. We made a special CG generator, which is a precise simulator of the real camera to generate a CG image identical to the input image. We confirmed the ability of our method and achieved a 3DOF real-time object tracking system at the rate of 4.5 frames/s, and 6DOF tracking using four cameras at 5 frames/s.

We developed a direction system for assisting the work in a real world from a distant site. At first, the 3-D shape of the object is measured and sent to the distant PC. A supervisor at the distant site can observe the CG of the object and draw annotation figures on it. The figures of the direction message are projected onto the object using projectors. The worker is free from any wearing equipment, ex. HMD, and multi projectors avoid the problem of occlusion by the worker body.

We developed a direction system of real-world operation from distant site. The system consists of several sets of cameras, projectors and PCs connected via network each other. At first, the 3-D shape of the object is measured using pattern light projection method, and it is sent to the distant PC. A supervisor can observe the CG of the object from any viewpoint and draw annotation figures on the CG. The direction message is sent to the real field and projected onto the object using projectors. The projected annotations are well aligned geometrically because the all cameras and projectors are calibrated with single reference object. The worker is free from any wearing equipment, ex. HMD, and multi projectors avoid the problem of occlusion by the worker body. Direction of alignment task of both existing and new object are also implemented.

In a Cooperative Distributed Vision system, a group of communicating Active Vision Agents (AVA, in short, i.e. real time image processor with an active video camera and high speed network interface) cooperate to fulfill a meaningful task such as moving abject tracking and dynamic scene visualization. A key issue to design and implement an AVA rests in the dynamic integration of Visual Perception, Camera Action, and Network Communication. This paper proposes a novel dynamic system architecture named Dynamic Memory Architecture, where perception, action, and communication modules share what we call the Dynamic Memory. It maintains not only temporal histories of state variables such as pan-tilt angles of the camera and the target object location but also their predicted values in the future. Perception, action, and communication modules are implemented as parallel processes which dynamically read from and write into the memory according to their own individual dynamics. The dynamic memory supports such asynchronous dynamic interactions (i.e. data exchanges between the modules) without wasting time for synchronization. This no-wait asynchronous module interaction capability greatly facilitates the implementation of real time reactive systems such as moving object tracking. Moreover, the dynamic memory supports the virtual synchronization between multiple AVAs, which facilitates the cooperative object tracking by communicating AVAs. A prototype system for real time moving object tracking demonstrated the effectiveness of the proposed idea.

Measurement and recognition of moving objects by a visual system is crucial to realization of robots that can handle a changing environment flexibly. In this article, we propose a system that tracks 6-DOF motion of a solid object of any shape in real time using a range image sequence. We compare range images measured by the Silicon Range Finder high-speed range image sensor with those synthesized from a computer model, and we assert that motion can be estimated independently of the form of objects. Furthermore, we propose a method to generate range images from a model very rapidly with CG rendering hardware. We also describe a process to model the shape of objects simultaneously with tracking, and a method to keep the object within the measurement region by rotating the measurement direction of the sensors. We performed experiments to verify the effectiveness of our method and found that all processes were performed at a speed of 7 frames per second and that the measurement/errors were less than 1% of the scope of measurement.

Range images are helpful in tracking the motion of objects defined by smooth curved surfaces. However, combined use of range images with intensity data such as texture data is desirable in order to improve measurement stability. In this paper, a method is proposed for model-based tracking of 6-axis object motion. Pose estimation based on intensity image is not merely added to a range-image-based tracking system but instead, a texture-mapped model is built through range and intensity measurement and then used for tracking. We first explain a method for direct acquisition of object motion by applying the object rigidity to the gradient method, and then this method is shown to be easily fused with range-image-based motion tracking. An algorithm is also offered for high-speed image generation and reflectance estimation using CG hardware. Finally, experiments are described that demonstrate the feasibility of real-time tracking of the 6-axis motion of an object and of acquisition of the shape and reflectance model. © 1998 Scripta Technica.

滑らかな自由曲面により定義される物体の運動を追跡するためには距離画像が有効であるが, より計測安定度を向上させるためには物体表面のテクスチャなど濃淡情報を融合することが望ましい. 本論文では, 単に濃淡画像による姿勢推定手法を距離画像による追跡システムに追加するのではなく, 距離・濃淡情報の融合により対象のテクスチャマップモデルを得たと考え, モデルベース的に対象の6軸運動を求める手法を提案する. まず, こう配法に対象物体の剛体性を適用することで直接対象の運動を求める手法について述べ, 次にこの手法が簡単に距離画像による運動追跡と融合できることを示す. また, CGハードウェアにより高速な画像生成と反射率推定を行う手法を導入し, 実験により対象物体の6軸運動追跡と形状・反射率モデル獲得が実時間的に実行可能であることを示す.

本論文では, 距離画像を用いた固有空間照合より, 物体の姿勢検出を行う手法を述べる. 従来, 固有空間を用いた画像照合では, 濃淡画像の2次元照合の手法として用いられたが, これを距離画像の2次元照合ヘ応用した. 距離画像の2次元照合に固有空間を応用するにあたり, アクティブレンジファインダによる距離画像計測で生じるオクルージョンや, テクスチャによる反射光量の変化から生じる画素欠損の扱いが問題となる. 欠損画素は距離情報をもたないため, 固有ベクトルとの内積より固有空間へ射影することができない. そこで, 距離画像を用いた固有空間照合を実現するために, 画素欠損に依存しない固有空間照合を用いた姿勢検出手法を提案し, 本手法の有効性を検討する.

We propose a new method to track 3-D motion of an object in real-time, using range images. A very high speed range sensor "Silicon Range Finder" makes it possible to acquire range images at more than 30 frames per second, and rotated to keep the object in the measurable area. Rough but fast results of range image processing controls the angle of the range sensor, and the object centered range images decide the pose of the object. The 3-D CG rendering hardware of the graphics workstation generates range images from the surface model very fast, so it makes the comparison algorithm simple. Tracking speed and accuracies of estimated motion seem to be feasible for the man-machine cooperative systems. © 1996 IEEE.