日浦 慎作

In this paper, we develop an active stereo system for endoscope which requires auto-calibration, because a micro pattern projector is inserted through the instrument channel during an operation and cannot be fixed to the endoscope. For solution, a new auto-calibration technique with full 6-DOF estimation of an active stereo system without any extra devices nor extra pattern projections is proposed. In the technique, the pattern projector itself is simultaneously captured with a target scene by an endoscope camera and the silhouette of the pattern projector is used to conduct 2D-3D matching by using the knowledge of the shape of the projector. In addition, the markers which is included in the projection pattern are extracted and the distances from the closest epipolar lines are calculated as for the cost function. To enhance the robustness of the reconstruction, we also propose a simple high dynamic range (HDR) imaging system for an endoscope by alternating the input power of the pattern projector ON and OFF to blink the pattern so that exposure time will vary with beat frequency, realizing a virtual multi-exposure camera. By applying our auto calibration technique with HDR imaging system, we achieved a robust and accurate reconstruction of tissue in metric 3D under practical operation of the endoscopic system, such as reconstruction of the inside of a real stomach of a pig.

Depth from Defocus (DFD) is known as a technology which is able to estimate depth by a monocular camera, without any additive devices. However, it has to get two blurred images with different focused distance sequentially so that subjects or the camera itself can move in the short interval of consecutive shots. Conventional DFD is intolerant to such image shift especially in the weak texture part. In this paper, a new evaluation function for DFD is shown in order to achieve the motion robustness in frequency domain. The Conventional DFD algorithm is modified considering the image shift, and in the derived formula both shift and phase components vanished. As for DFD technology, the formula is insensitive to the image shift because the defocus blur appears only in the amplitude part. As a result, we also confirmed that our algorithm overcomes conventional methods especially for the real life with rapidly moving subjects.

When an image is projected onto a non-planer surface, the resolution is not spatially uniform because projectors are normally designed to display an image with spatially uniform pixel density on a fat surface. We propose to insert a transparent layer between a projector and a non-planar surface and optimize its shape to make the projection pixel density spatially uniform, which can be controlled by non-uniformly refracting a projection light through the transparent layer. We use a multi-material 3D printer to fabricate the transparent layer combined with a non-planar surface. From the result of a projection experiment with the surface, we validate our proposed method.

Polarization is a phenomenon that cannot be observed by the human eye, but it provides rich information regarding scenes. The proposed method estimates the surface normal of black specular objects through polarization analysis of reflected light. A unique surface normal cannot be determined from a polarization image observed from a single viewpoint; thus, we observe the object from multiple viewpoints. To analyze the polarization state of the reflected light at the corresponding points when observed from multiple viewpoints, the abstract shape is predetermined using a space carving technique. Unlike a conventional photometric stereo or multiview stereo, which cannot estimate the shape of a black specular object, the proposed method estimates the surface normal and three-dimensional coordinates of black specular objects via polarization analysis and space carving. (C) The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

近年，実際の風景を撮影しているにもかかわらず，写っているものが小さなおもちゃのように見えるミニチュア風の写真が注目を集めている．このような写真の撮影には，チルトレンズ（傾けたレンズ）を装着したカメラが使用される．このチルトレンズの効果をCGで再現するためには，チルトレンズの高精度なカメラキャリブレーションとレンダリング手法が必要となる．本研究では，ぼけと幾何特徴を利用したカメラキャリブレーション手法を拡張し，撮影した平面ターゲットの画像から，チルトレンズのレンズの傾き角を推定する．レンズの傾きを考慮した分散レイトレーシング法を用いることにより，有限な距離で合焦しない点が画面内に含まれる場合にも適切に画像生成が行える．撮影画像を使用した実験では，実写と同等のCG画像を生成できることを示し，本手法の有効性を確認した．In recent years, photographs and videos of actual scenery look like the miniature diorama have been gaining attention. This method of photography is involves the use of cameras equipped with a tilt-shift lens. In this study, we proposed the method to estimate the camera parameters of real tilted lenses. After estimating the camera parameters based on the camera calibration method using geometry and blur of feature points, we estimated a tilt angle of a real lens. Also, we proposed a rendering algorithm considering the lens tilt using the distributed ray tracing method. By using our rendering algorithm, we generated images with arbitrary focal plane including the case of infinite focal distance. At the experiments, by comparing generated images with real images, we demonstrated the ability of our proposed methods.

We propose a new system to visualize depth-dependent patterns and images on solid objects with complex geometry using multiple projectors. The system, despite consisting of conventional passive LCD projectors, is able to project different images and patterns depending on the spatial location of the object. The technique is based on the simple principle that multiple patterns projected from multiple projectors interfere constructively with each other when their patterns are projected on the same object. Previous techniques based on the same principle can only achieve (1) low resolution volume colorization or (2) high resolution images but only on a limited number of flat planes. In this paper, we discretize a 3D object into a number of 3D points so that high resolution images can be projected onto the complex shapes. We also propose a dynamic ranges expansion technique as well as an efficient optimization procedure based on epipolar constraints. Such technique can be used to the extend projection mapping to have spatial dependency, which is desirable for practical applications. We also demonstrate the system potential as a visual instructor for object placement and assembling. Experiments prove the effectiveness of our method.

3D endoscopic systems have been researched and developed to measure the actual shape and size of living tissues for the purpose of remote surgery and diagnosis, to name a few. For such systems, active stereo that consists of a camera and a pattern projector (i.e., structured light systems) is a promising solution because of simple system with high accuracy. Recently, an active-stereo-based 3D endoscope system has been proposed, in which many practical problems were solved such as shallow focal range of the pattern projector or strong diffusion by living tissues. To use the laser pattern projector for endoscopic systems, two fundamental issues arise a limited dynamic range of the endoscopic camera and a calibration of the system. In this paper, we proposed a new high dynamic range (HDR) image synthesis technique for a laser pattern projector as well as an auto-calibration technique for dynamic motion. Quantitative experiments are conducted to show the effectiveness of the method followed by a demonstration using real endoscopic system.

Metamerism is the phenomenon by which two objects are recognized as having different colors under one light source and the same color under another light source. In this article, the authors propose a method for creating trick artwork using metamerism. Two illuminants are designed to achieve metamerism such that two oil paints used in a piece of artwork look the same under one light but different under another light. The experimental results show that metamerism is generated between the two light sources and the two object colors. (C) 2016 Society for Imaging Science and Technology.

非合焦画像からぼけを取り除くために符号化開口が有効であることが知られている．しかし，その開口形状の設計に関する研究は十分に行われていない．そこで本論文では実用的な設計法とその詳細な実装を示すことを目的とし，特に最適化に用いる遺伝的アルゴリズムの性能を大きく左右する交叉法の比較と評価を行う．また，ぼけ除去画像を認識に用いる場合，その画像の良し悪しは画質（視覚的品質）ではなくその後の認識率により評価されるべきである．そこで本論文では，PSNRを基準とした画質評価と，その画像を認識した際の認識率との関係を示す．Although coded aperture is known as an efficient technique to remove the optical blur from the defocused image, studies on designing the shape of the aperture have not been well explored yet. Therefore in this paper, we will show a practical method with detailed implementation of the optimization of the coded apertures. Especially, crossover process which strongly affects to the performance of genetic algorithm is intensively evaluated and compared with the other methods. In the context of object recognition, the performance of the deblurring should be evaluated by not the visual quality but the resultant recognition rates. Therefore we will show the relationships between PSNR-based image quality criterion and actual recognition rates through experiment.

We propose a new projection system to visualise different independent images simultaneously on planes placed at different depths within a volume using multiple projectors. This is currently not possible with traditional systems, and we achieve it by projecting interference patterns rather than simple images. The main research issue is therefore to determine how to compute a distributed interference pattern that would recombine into multiple target images when projected by the different projectors. In this paper, we show that while the problem is not solvable exactly, good approximations can be obtained through optimization techniques. We also propose a practical calibration framework and validate our method by showing the technique in action with a prototype system. The system opens up significant new possibilities to extend projection mapping techniques to dynamic environments for artistic purposes, as well as visual assessment of distances.

メタメリズムとはある光源下では2つの物体が異なる色として知覚されるが別の光源下では同じ色として知覚される現象である．本研究では，メタメリズムを利用したトリックアートの作成手法を提案する．与えられた光源下でメタメリズムが生起する条件を満たす油絵の具を提案手法によって計算する．本研究では，トリックアート作成のために与えられた光源下でメタリズムが生起する条件を満たす油絵の具の混合比率の計算法を提案する．実験では，2光源・4物体色の間で生起されるメタメリズムについて計算を行った．これにより擬似的な陰影を表現することができ，二次元の絵画が三次元の物体として知覚されるトリックアートを実現することができた．

The central projection model commonly used to model cameras as well as projectors, results in similar advantages and disadvantages in both types of system. Considering the case of active stereo systems using a projector and camera setup, a central projection model creates several problems, among them, narrow depth range and necessity of wide baseline are crucial. In the paper, we solve the problems by introducing a light field projector, which can project a depth-dependent pattern. The light field projector is realized by attaching a coded aperture with a high frequency mask in front of the lens of the video projector, which also projects a high frequency pattern. Because the light field projector cannot be approximated by a thin lens model and a precise calibration method is not established yet, an image-based approach is proposed to apply a stereo technique to the system. Although image-based techniques usually require a large database and often imply heavy computational costs, we propose a hierarchical approach and a feature-based search for solution. In the experiments, it is confirmed that our method can accurately recover the dense shape of curved and textured objects for a wide range of depths from a single captured image.

For endoscopic medical treatment, measuring the size and shape of the lesion, such as a tumor, is important for the improvement of diagnostic accuracy. We are developing a system to measure the shapes and sizes of living tissue by active stereo method using a normal endoscope on which a micro pattern projector is attached. In order to perform 3D reconstruction, estimating the intrinsic and extrinsic parameters of the endoscopic camera and the pattern projector is required. Particularly, calibration of the pattern projector is difficult. In this paper, we propose a simultaneous estimation method of both intrinsic and extrinsic parameters of the pattern projector. This simplifies the calibration procedure required in practical scenes. Furthermore, we have developed an efficient user interface to intuitively operate the calibration and reconstruction procedures. Using the developed system, we measured the shape of an internal tissue of the soft palate of a human and a biological specimen.

The paper addresses the recognition problem of defocused patterns. Though recognition algorithms assume that the input images are focused and sharp, it does not always hold on actual camera-captured images. Thus, a recognition method that can recognize defocused patterns is required. In this paper, we propose a novel recognition framework for defocused patterns, relying on a single camera without a depth sensor. The framework is based on the coded aperture which can recover a less-degraded image from a defocused image if depth is available. However, in the problem setting of "a single camera without a depth sensor," estimating depth is ill-posed and an assumption is required to estimate the depth. To solve the problem, we introduce a new assumption suitable for pattern recognition templates are known. It is based on the fact that in pattern recognition, all templates must be available in advance for training. The experiments confirmed that the proposed method is fast and robust to defocus and scaling, especially for heavily defocused patterns.