by Marina Kolesnik and Gregory Baratoff
State-of-the-art inspection of inaccessible sewers is by teleoperated visual surveying. In practice, the inspection process is both complicated and labour-intensive. This inspired research activities targeted at developing an autonomous robot for sewer surveying. Out of four institutions from research and industry a multidisciplinary team was chosen, working on the three-year project ‘MAKRO’ which started in September 1997. A Hybrid Model-Based Vision System has been developed at GMD-Research Center for Information Technology at Sankt-Augustin. The Vision System will facilitate the orientation and navigation of the autonomous MAKRO robot while moving in the sewer.
Maintenance of thousands of kilometres of sewers requires their permanent surveying. State-of-the-art inspection of inaccessible sewers is by visual surveying. A human teleoperates a wheeled platform equipped with a high-resolution color camera and a lighting system, producing a video record of the sewer condition. An operator-guided process of sewer surveying is labour intensive, costly, and prone to human error. A radical change in sewer surveying technology, based on the use of autonomous mobile robots, could accelerate the inspection process, enhance inspection quality, and reduce cost.
The research project MAKRO is being conducted by four partners from research and industry and aims at developing an autonomous robot for surveying of modern concrete sewers. The task of such a robot will be to collect a video record about sewer conditions. The multi-segment robot consists of a mobile base platform, and is equipped with a set of sensors, a light source, a laser crosshair projector and a camera, it must be able to drive autonomously through a long system of sewer pipes. An on-line Hybrid Model Vision-Based System has been developed at GMD to navigate the MAKRO robot when it moves along sewer pipes, approaches manhole junction areas, and proceeds to enter the next pipe. The Vision System comprises two different algorithms. The first one, called the orientation algorithm, assesses the robot’s relative orientation with respect to the pipe axis, in order to guide the robot through a pipe. The second one, called the 3D interpretation algorithm, recognizes the end of the current pipe at a manhole entrance and locates the next pipe opening to move into when the MAKRO robot is positioned before a manhole area of the sewer.
The Vision System relies on a hybrid sensing strategy consisting of two modes: a fast laser footprint analysis mode and a slower, but more thorough, full image analysis mode. In the first mode the laser crosshair projector is on and the flashlight is off, causing the camera to record a predominantly dark image with a bright laser footprint. The analysis of such an image is very fast, and in most cases it can adequately support the robot’s navigation as if travelling through a pipe segment. At some points in time, however, the robot needs to acquire more accurate information about the sewer. For example, when the robot approaches a manhole area, or when it decides to move into a pipe which has a smaller radius than a manhole area, it needs to determine the distance and direction of the pipe end with high accuracy in order to execute a collision-free turn. In these cases, the flashlight is used to illuminate the sewer. Since the structure of such an image is much more complex, the processing is more time-consuming. But it allows to obtain more accurate information about the location of relevant sewer structures.
The robot relies on the ‘fast & rough’ sensing mode to navigate through a pipe segment in a closed-loop fashion, continuously analyzing the laser footprint to determine its orientation within the pipe, and using this information to correct its heading, so as to keep its orientation in the direction of the pipe axis. The laser footprint also allows the end of a pipe to be detected. Using this information, the robot knows when it has arrived in the manhole area. Upon arrival it stops, snaps a few images, analyzes them, and makes a decision in which direction to proceed, ie which pipe to enter next. In this situation, the robot relies on the ‘slow & thorough’ sensing strategy to get an accurate evaluation to base its decision on.
A further characteristic of the Vision System lies in the fact that the visual processing is model-based. Modern concrete pipes are mostly cylindrical with a standard diameter of 60cm or 30cm. When illuminated by the laser crosshair projector they give rise to a pair of conic sections whose shape encodes relative orientation of the robot within the pipe. This information is extracted and used to align the robot with the pipe axis. In addition to the pipes being cylindrical, their ends are circular. Important circular landmarks are pipe ends and entrances at sewer manhole areas as well as ends of house inlet pipes. These circular structures appear in the fully illuminated image as ellipses. Using a calibrated camera, the 3D orientation of these circular structures is determined by their image ellipses. This provides the robot with an estimate of its instantaneous position relative to these landmarks.
In general the image of a circle is an ellipse, except when the camera’s viewing direction is perpendicular to the supporting plane of the 3D circle, in which case it is a circle. The vision algorithm processes ellipses when reconstructing circular structures generated by house inlets. It uses the orientation algorithm which is based on the images of a laser footprint to ensure an approximately parallel viewing direction of the robot camera with respect to the pipe axis. This setup allows to work with circles in the images, hence eliminating two of the main problems with the general ellipse as compared with the circle case: (1) the more time and memory consuming detection procedure, and (2) an inherent two-fold 3D orientation ambiguity of the 3D circle of which it is an image.
The Hybrid Model-Based Vision System operates on-line and is integrated in the onboard control system of the MAKRO robot. Future work will be directed at introducing new functions to the onboard Vision System, which will enable the MAKRO robot to perform automatic vision monitoring of sewer conditions while moving in the sewer.
Links:
MAKRO project and related publications:
http://ais.gmd.de/~kolesnik/
Please contact:
Marina Kolesnik - GMD
Tel: +49 2241 14 2097
E-mail: kolesnik@gmd.de