Shinsaku Hiura

In this paper, we develop an active stereo system for endoscope which requires auto-calibration, because a micro pattern projector is inserted through the instrument channel during an operation and cannot be fixed to the endoscope. For solution, a new auto-calibration technique with full 6-DOF estimation of an active stereo system without any extra devices nor extra pattern projections is proposed. In the technique, the pattern projector itself is simultaneously captured with a target scene by an endoscope camera and the silhouette of the pattern projector is used to conduct 2D-3D matching by using the knowledge of the shape of the projector. In addition, the markers which is included in the projection pattern are extracted and the distances from the closest epipolar lines are calculated as for the cost function. To enhance the robustness of the reconstruction, we also propose a simple high dynamic range (HDR) imaging system for an endoscope by alternating the input power of the pattern projector ON and OFF to blink the pattern so that exposure time will vary with beat frequency, realizing a virtual multi-exposure camera. By applying our auto calibration technique with HDR imaging system, we achieved a robust and accurate reconstruction of tissue in metric 3D under practical operation of the endoscopic system, such as reconstruction of the inside of a real stomach of a pig.

Depth from Defocus (DFD) is known as a technology which is able to estimate depth by a monocular camera, without any additive devices. However, it has to get two blurred images with different focused distance sequentially so that subjects or the camera itself can move in the short interval of consecutive shots. Conventional DFD is intolerant to such image shift especially in the weak texture part. In this paper, a new evaluation function for DFD is shown in order to achieve the motion robustness in frequency domain. The Conventional DFD algorithm is modified considering the image shift, and in the derived formula both shift and phase components vanished. As for DFD technology, the formula is insensitive to the image shift because the defocus blur appears only in the amplitude part. As a result, we also confirmed that our algorithm overcomes conventional methods especially for the real life with rapidly moving subjects.

When an image is projected onto a non-planer surface, the resolution is not spatially uniform because projectors are normally designed to display an image with spatially uniform pixel density on a fat surface. We propose to insert a transparent layer between a projector and a non-planar surface and optimize its shape to make the projection pixel density spatially uniform, which can be controlled by non-uniformly refracting a projection light through the transparent layer. We use a multi-material 3D printer to fabricate the transparent layer combined with a non-planar surface. From the result of a projection experiment with the surface, we validate our proposed method.

Polarization is a phenomenon that cannot be observed by the human eye, but it provides rich information regarding scenes. The proposed method estimates the surface normal of black specular objects through polarization analysis of reflected light. A unique surface normal cannot be determined from a polarization image observed from a single viewpoint; thus, we observe the object from multiple viewpoints. To analyze the polarization state of the reflected light at the corresponding points when observed from multiple viewpoints, the abstract shape is predetermined using a space carving technique. Unlike a conventional photometric stereo or multiview stereo, which cannot estimate the shape of a black specular object, the proposed method estimates the surface normal and three-dimensional coordinates of black specular objects via polarization analysis and space carving. (C) The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

We propose a new system to visualize depth-dependent patterns and images on solid objects with complex geometry using multiple projectors. The system, despite consisting of conventional passive LCD projectors, is able to project different images and patterns depending on the spatial location of the object. The technique is based on the simple principle that multiple patterns projected from multiple projectors interfere constructively with each other when their patterns are projected on the same object. Previous techniques based on the same principle can only achieve (1) low resolution volume colorization or (2) high resolution images but only on a limited number of flat planes. In this paper, we discretize a 3D object into a number of 3D points so that high resolution images can be projected onto the complex shapes. We also propose a dynamic ranges expansion technique as well as an efficient optimization procedure based on epipolar constraints. Such technique can be used to the extend projection mapping to have spatial dependency, which is desirable for practical applications. We also demonstrate the system potential as a visual instructor for object placement and assembling. Experiments prove the effectiveness of our method.

3D endoscopic systems have been researched and developed to measure the actual shape and size of living tissues for the purpose of remote surgery and diagnosis, to name a few. For such systems, active stereo that consists of a camera and a pattern projector (i.e., structured light systems) is a promising solution because of simple system with high accuracy. Recently, an active-stereo-based 3D endoscope system has been proposed, in which many practical problems were solved such as shallow focal range of the pattern projector or strong diffusion by living tissues. To use the laser pattern projector for endoscopic systems, two fundamental issues arise a limited dynamic range of the endoscopic camera and a calibration of the system. In this paper, we proposed a new high dynamic range (HDR) image synthesis technique for a laser pattern projector as well as an auto-calibration technique for dynamic motion. Quantitative experiments are conducted to show the effectiveness of the method followed by a demonstration using real endoscopic system.

Metamerism is the phenomenon by which two objects are recognized as having different colors under one light source and the same color under another light source. In this article, the authors propose a method for creating trick artwork using metamerism. Two illuminants are designed to achieve metamerism such that two oil paints used in a piece of artwork look the same under one light but different under another light. The experimental results show that metamerism is generated between the two light sources and the two object colors. (C) 2016 Society for Imaging Science and Technology.

We propose a new projection system to visualise different independent images simultaneously on planes placed at different depths within a volume using multiple projectors. This is currently not possible with traditional systems, and we achieve it by projecting interference patterns rather than simple images. The main research issue is therefore to determine how to compute a distributed interference pattern that would recombine into multiple target images when projected by the different projectors. In this paper, we show that while the problem is not solvable exactly, good approximations can be obtained through optimization techniques. We also propose a practical calibration framework and validate our method by showing the technique in action with a prototype system. The system opens up significant new possibilities to extend projection mapping techniques to dynamic environments for artistic purposes, as well as visual assessment of distances.

Metamerism is a phenomenon where two objects recognized as having different colors under one light are also recognized as having the same color under another light. This short paper proposes a method that automatically calculates the blending ratios of oil paints satisfying the condition to occur metamerism under the specific light sources for actualizing artistic illusion. It entails metamerism occurring between four types of object colors under two types of light sources. This enables us to create false shading where the observer recognizes the 2D oil painting as if it is a 3D object with plausible shading.

The central projection model commonly used to model cameras as well as projectors, results in similar advantages and disadvantages in both types of system. Considering the case of active stereo systems using a projector and camera setup, a central projection model creates several problems, among them, narrow depth range and necessity of wide baseline are crucial. In the paper, we solve the problems by introducing a light field projector, which can project a depth-dependent pattern. The light field projector is realized by attaching a coded aperture with a high frequency mask in front of the lens of the video projector, which also projects a high frequency pattern. Because the light field projector cannot be approximated by a thin lens model and a precise calibration method is not established yet, an image-based approach is proposed to apply a stereo technique to the system. Although image-based techniques usually require a large database and often imply heavy computational costs, we propose a hierarchical approach and a feature-based search for solution. In the experiments, it is confirmed that our method can accurately recover the dense shape of curved and textured objects for a wide range of depths from a single captured image.

For endoscopic medical treatment, measuring the size and shape of the lesion, such as a tumor, is important for the improvement of diagnostic accuracy. We are developing a system to measure the shapes and sizes of living tissue by active stereo method using a normal endoscope on which a micro pattern projector is attached. In order to perform 3D reconstruction, estimating the intrinsic and extrinsic parameters of the endoscopic camera and the pattern projector is required. Particularly, calibration of the pattern projector is difficult. In this paper, we propose a simultaneous estimation method of both intrinsic and extrinsic parameters of the pattern projector. This simplifies the calibration procedure required in practical scenes. Furthermore, we have developed an efficient user interface to intuitively operate the calibration and reconstruction procedures. Using the developed system, we measured the shape of an internal tissue of the soft palate of a human and a biological specimen.

The paper addresses the recognition problem of defocused patterns. Though recognition algorithms assume that the input images are focused and sharp, it does not always hold on actual camera-captured images. Thus, a recognition method that can recognize defocused patterns is required. In this paper, we propose a novel recognition framework for defocused patterns, relying on a single camera without a depth sensor. The framework is based on the coded aperture which can recover a less-degraded image from a defocused image if depth is available. However, in the problem setting of "a single camera without a depth sensor," estimating depth is ill-posed and an assumption is required to estimate the depth. To solve the problem, we introduce a new assumption suitable for pattern recognition templates are known. It is based on the fact that in pattern recognition, all templates must be available in advance for training. The experiments confirmed that the proposed method is fast and robust to defocus and scaling, especially for heavily defocused patterns.