Issues in Vision, Semi-autonomous Control, Haptics and Manipulation in Robotics for Nuclear Decommissioning
SD Monk and D Cheneler*
Engineering, Lancaster University, UK
Submission:March 18, 2019; Published:March 26, 2019
*Corresponding author: David Cheneler, Engineering, Lancaster University, UK
How to cite this article: SD Monk, D Cheneler. Issues in Vision, Semi-autonomous Control, Haptics and Manipulation in Robotics for Nuclear Decommissioning. Robot Autom Eng J. 2019; 4(2): 555635. DOI: 10.19080/RAEJ.2019.04.555635
Introduction
Traditionally, the nuclear industry has preferred the use of tele-operated control within robotic applications such as decommissioning. This is due to obvious safety reasons, along with other less apparent motivations such as the safeguarding of industry jobs, and a lack of coding expertise in the industry. However, problems with the use of such techniques have been evident within the past few years, mostly associated with operator fatigue leading to errors. A typical modern autonomous robotic system will utilise some sort of stereoscopic 3D vision system (often based on LIDAR) to aid recognition. However, this information can be hard to relay to a human tele-operator not used to such information. Further, tele-operation of a modern robot is a truly complex and specialised skill, and there are a lack of people with in the nuclear industry (and indeed industry as a whole) with these skills. A potential solution to alleviate these problems may be in the use of semi-autonomous control where the robotic artificial intelligence may be used for low-level tasks while the human operator would handle the higher-level decisions. Instead of the operator having to directly control the robot via two joysticks, the operator would be more likely to be confronted with a large touchscreen complete with a list of tasks and highlighted objects on which the robot can perform them upon.
Keywords: Nuclear; Decommissioning; Robotics; Vision; Haptics; Semi-Autonomous control; Manipulators
Discussion
Vision is obviously vital within all robotic control, and research is currently ongoing within such diverse areas as fruit picking and emergency repairs in space [1,2]. There are many modern vision sensors available using techniques such as cameras, IR range finders and LIDAR to provide advanced vision to the operator and low-level autonomous control systems [3]. However, due to the sacrificial nature of the work being undertaken during nuclear decommissioning owing to the high levels of radiation, the vision systems employed must be quite cheap. The system utilised within our work thus far has been the mass-produced Microsoft Kinect sensor mounted at a fixed position independent of the moving robotic manipulators. The Kinect is a cost effective and commonly used RGB-D sensor, originally developed for gaming applications but since used widely for research into robotics. It couples a standard RGB camera with a structured light depth sensor, allowing both colour and depth data to be used and combined. A live RGB video stream is displayed to the user on the graphical user interface (GUI) as the mobile base unit is positioned and stabilised. Although many systems utilising the MS Kinect make use of 3D point clouds, a different, less computationally expensive approach, is used by us, utilising edge detection on the acquired RGB image to separate objects for user selection.
This obviously reduces the accuracy of object recognition and means the associated control algorithms require the use of some significant assumptions about shape. However, the processor load is significantly reduced and therefore the speed of the process is increased leading to a reduced latency. The standard 2D image is then combined with the depth data to locate the coordinates of all objects in 3D space. This information is then all fed back to the operator via a touchscreen display offering options and objects on which to operate. The user of the system can change sensitivity of the algorithm allowing more or less of the objects in the surrounding area to be considered within this vision system. Higher levels of sensitivity lead to more options, although greater computational power leading to slower operation. A typical application for robotics in nuclear decommissioning is the grasping and cutting of pipework. Once the operator has chosen their object (i.e. pipe) and procedure (i.e. cutting with saw), there are now four key positions:
A. The position directly in front of the grasp location
B. The cutting operation start position
C. The cutting operation end position
D. The final grasp location
Once the operator has chosen the grasp location and cutting location, the control program can calculate the four above positions in 3D space. There are numerous inverse kinematic solvers available of varying complexity and each has their own individual strength and weakness [4]. A pseudo inverse Jacobian transpose inverse kinematic solver was chosen here as it offers the best solution to this problem in terms of speed and accuracy [5]. This solver is then utilised to find the associated target joint angles using the 3D co-ordinates of the four locations above. The joint angles calculated represent set points for the feedback control algorithms that determine the required position of the actuators. The algorithms behind the vision system, GUI and inverse kinematics solver were all implemented in MATLAB. However, the interface to the robotic actuators is via National Instruments LabVIEW, with these elements connected via TCP-IP locally on the same PC. The LabVIEW control software currently uses Proportional Integral Derivative (PID) control to smoothly move the joints to the set points provided by the MATLAB algorithm. Throughout the process, the user can view the live colour video and terminate manipulator movements at any time. This method is currently implemented on two seven degrees of freedom hydraulically controlled arms attached to a BROKK 40 industrial robot. However, the principle of control could quite easily be adapted to a multi-armed robot of any size as long as dimensions were known prior to implementation. The current work using the Hydrolek arms is described in more detail in [5-10].
While high level decisions can be made by an operator controlling a robot using visual information alone, haptic and tool feedback is required to ensure that local operations such as grasping hold of pipes and cutting or drilling are performed adequately. A major disadvantage of robotic manipulation is that the operator’s hands are not at the site of operation. The lack of kinaesthetic and tactile information makes automated tasks more difficult; hence, haptic feedback is a crucial source of information [11]. Indeed, studies have shown haptic feedback helps reduce task completion time and error rates [12]. Haptics describes both the cutaneous (tactile) and kinaesthetic (force) information obtained during exploration or manipulation of an object. Tactile information may include pressure, and the local shape and slipperiness of an object, which is important information for handling objects, and kinaesthetic perception includes discernment of proprioception and force. Standard techniques utilise simple force or torque sensors incorporated in between the last robotic link and the robot end effectors [12-13]. This limited information informs the operator as to when the end effector has made contact and allows for direct feedback that acts as a safety control preventing excessive loading. Similar systems have been incorporated into many simple tele-operated power tools [14].
However, these simple strategies, while cheap, are not sufficient for the manipulation of end effectors for the machining, i.e. drilling/hot tapping and cutting of complex shapes such as pipes [15]. In such a scenario, a robot will have to first reference the location of the site of processing, approach the pre-position while checking for collisions, confirm orthogonality and fine positioning, initiate contact and clamp with work piece, perform the operation, i.e. drilling, then release. The referencing can be achieved by way of utilising the visual data and encoders on the joints of the robot and will part of the high-level control by the operator. Safe approaching can be achieved by way of proximity sensors such as noncontact inductive sensors or laser/IR range finders [15-17]. The selection of which will depend on the particular task. It could be that surface finish/geometry variations precludes the use of laser-based systems, in this case ultrasonic range finders can be useful but they are best for locating large flat objects [18-21]. When in contact, tactile sensing can be employed as a means to augment the initial grasp and manipulation strategies by addressing inconsistencies in the contact forces during object contact and manipulation, usually by way of monitoring an array of compliant pressure sensors of which many types are available [20].
One issue with a robotic system is that the more degrees of freedom a system has, the more compliant it becomes. This makes it difficult for it to apply a cutting or drilling force and ensure a successful operation. This is compounded if the object being cut is also compliant [21]. It is therefore recommended that the tactile sensing capability be incorporated into a clamp module that attaches the end-effector nose to the structure to be operated upon. This will help avoid vibrations between the tool and the surface as well as any unwanted flexion. Several systems exist for monitoring tool operation, for instance piezoelectric sensors in the chuck of the cutting tool, or 3-axis load cells that will inform the load applied to the tool as well as any lateral skating of the tool [22,23]. These sensors could also be used to ensure the tool is normal to the surface if needed by way of antennae, which are in contact with the tool and surface and monitor of asymmetric loading [23]. If a sufficient clamping system is used, vibration, asymmetry and skating should be negated by design and so simple load cells could be used to ensure contact between the tool and workpiece and to inform the operator when the tool has broken though. What this data cannot inform the operator is how well the tool is cutting. Proximity sensors, as used before, could be used to monitor cutting and drilling progress by measuring how far the tool has moved into the workpiece. However, it won’t inform the operator if the tool is having difficulty machining due to the hardness of the material or because of tool wear, beyond the time taken to perform the operation so far.
One strategy for monitoring tool wear and load, and hence cutting efficiency, indirectly is by monitoring the power required to drive the spindle motor of the machine tool or local temperature rise due to cutting [24,25]. Such a technique could be employed as it simply requires a power monitor device and an algorithm that takes into account the power consumed by the idle-running spindle and its dependence on the thermal state of the machine tool. While this will provide useful data, the interpretation requires a significant amount of empirical, historical, and environmental data, i.e. temperatures and previous cutting tasks. A lot of operator skill is required to monitor tool performance, and it is unlikely that full autonomous control of the cutting operation will be as successful as with operator input.
Conclusion
Here a number of common issues regarding the implementation of semi-autonomous robotics, as compared to tele-operated robotics that are currently commonly employed, for decommissioning and related applications in the nuclear industry have been described. This strategy has a number of advantages when conducting complex tasks such as grasping and cutting of pipes, as seen in the example employed here, as many local decisions can be automated, reducing operator error due to incomplete information. However, as has been made clear here, this requires significant technological development to the system, with many sensors needing to be integrated, the data collected, analysed and simplified so that the operator has just enough information to make informed decisions in a timely manner.
References
- Silwal A, Davidson JR, Karkee M, Mo C, Zhang Q, et al. (2017) Design, integration, and field evaluation of a robotic apple harvester. Journal of Field Robotics 34(6): 1140-1159.
- UK Robotics & Autonomous Systems Network (2016) Space Robotics & Autonomous Systems: Widening the horizon of space exploration.
- Shengyon C, Youfu Li, Kwok NM (2011) Active vision in robotic systems: A survey of recent developments. International Journal of Robotics Research 30(11): 1343-1377.
- Serrezuela RR, Chavarro AFC, Cardozo MAT, Toquica AL, Martinez LFO (2017) Kinematic modelling of a robotic arm manipulator using MATLAB. ARPN Journal of Engineering and Applied Sciences 12(7): 2037-2045.
- West C, Montazeri A, Monk SD, Duda D, Taylor CJ (2017) A new approach to improve the parameter estimation accuracy in robotic manipulators using a multi-objective output error identification technique. In 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Lisbon, Portugal, pp. 1406-1411.
- Montazeri A, West C, Monk SD, Taylor CJ (2017) Dynamic modelling and parameter estimation of a hydraulic robot manipulator using a multi-objective genetic algorithm. International Journal of Control 90(4): 661-683.
- Burrell T, Montazeri A, Monk S, Taylor CJ (2016) Feedback Control-Based Inverse Kinematics Solvers for a Nuclear Decommissioning Robot. IFAC-PapersOnLine 49(21): 177-184.
8. West C, Montazeri A, Monk SD, Taylor CJ (2016) A genetic algorithm approach for parameter optimization of a 7DOF robotic manipulator. IFAC-PapersOnLine 49(12): 1261-1266.
- West C, Burrell T, Montazeri A, Monk SD, Taylor CJ (2015) Research and teaching using a hydraulically-actuated nuclear decommissioning robot. In International Conference on Systems Engineering, UK.
- Burrell T, Montazeri A, Monk SD, Taylor CJ (2015) Review of autonomous navigation for indoor aerial robots within a nuclear decommissioning context. In International Conference on Systems Engineering, UK.
- Gwilliam JC, Mahvash M, Vagvolgyi B, Vacharat A, Yuh DD, et al. (2009) Effects of haptic and graphical force feedback on teleoperated palpation. In 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, pp. 677-682.
- Williams LE, Loftin RB, Aldridge HA, Leiss EL, Bluethmann WJ (2002) Kinesthetic and visual force display for telerobotics. In Proceedings 2002 IEEE international conference on robotics and automation, Washington DC, USA, pp. 1249-1254.
- Kazanzides P, Zuhars J, Mittelstadt B, Taylor RH (1992) Force sensing and control for a surgical robot. In Proceedings 1992 IEEE International Conference on Robotics and Automation, Nice France, France, pp. 612-617.
- Esen H, Yano K, Buss M (2004) A virtual environment medical training system for bone drilling with 3DOF force feedback. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sendai, Japan, pp. 3631-3636.
- Frommknecht A, Kuehnle J, Effenberger I, Pidan S (2017) Multi-sensor measurement system for robotic drilling. Robotics and Computer-Integrated Manufacturing 47: 4-10.
- Fericean S, Droxler R (2007) New non-contacting inductive analog proximity and inductive linear displacement sensors for industrial automation. IEEE Sensors Journal 7(11): 1538-1545.
- Hsiao K, Nangeroni P, Huber M, Saxena A, Ng AY (2009) Reactive grasping using optical proximity sensors. In 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, pp. 2098-2105.
- Zhang R, Yuan P, Gong M (2012) Intelligent surface-normal adjustment system and application in aeronautical drilling robot. In 2012 Second International Conference on Intelligent System Design and Engineering Application, Hainan, China, pp. 696-699.
- Liang J, Bi S (2010) Design and experimental study of an end effector for robotic drilling. The International Journal of Advanced Manufacturing Technology 50(1-4): 399-407.
- Tegin J, Wikander J (2005) Tactile sensing in intelligent robotic manipulation-a review. Industrial Robot: An International Journal 32(1): 64-70.
- Eguti CCA, Trabasso LG (2014) Design of a robotic orbital driller for assembling aircraft structures. Mechatronics 24(5): 533-545.
- Shum LY, Sienkiewicz LJ, Gogniat T (1975) US Patent No. 3,872,285. Washington,DC: US Patent and Trademark Office, USA.
- Olsson T, Haage M, Kihlman H, Johansson R, Nilsson K, et al. (2010) Cost-efficient drilling using industrial robots with high-bandwidth force feedback. Robotics and Computer-Integrated Manufacturing, 26(1): 24-38.
- Cuppini D, D'errico G, Rutelli G (1990) Tool wear monitoring based on cutting power measurement. Wear 139(2): 303-311.
- Cheneler D, Ward MCL (2018) Active Thermal Sensor for Improved Distributed Temperature Sensing in Haptic Arrays. Journal of Sensors, Article ID 9631236.