2.624

2020影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于事件相机的机器人感知与控制综述

粟傈 杨帆 王向禹 郭川东 童良乐 胡权

粟傈, 杨帆, 王向禹, 郭川东, 童良乐, 胡权. 基于事件相机的机器人感知与控制综述. 自动化学报, 2021, 48(x): 1−21 doi: 10.16383/j.aas.c210263
引用本文: 粟傈, 杨帆, 王向禹, 郭川东, 童良乐, 胡权. 基于事件相机的机器人感知与控制综述. 自动化学报, 2021, 48(x): 1−21 doi: 10.16383/j.aas.c210263
Su Li, Yang Fan, Wang Xiang-Yu, Guo Chuan-Dong, Tong Liang-Le, Hu Quan. A survey of robot perception and control based on event camera. Acta Automatica Sinica, 2021, 48(x): 1−21 doi: 10.16383/j.aas.c210263
Citation: Su Li, Yang Fan, Wang Xiang-Yu, Guo Chuan-Dong, Tong Liang-Le, Hu Quan. A survey of robot perception and control based on event camera. Acta Automatica Sinica, 2021, 48(x): 1−21 doi: 10.16383/j.aas.c210263

基于事件相机的机器人感知与控制综述

doi: 10.16383/j.aas.c210263
基金项目: 国家自然科学基金(11872011), 国防科工局稳定支持科研项目(HTKJ2020KL502013)资助
详细信息
    作者简介:

    粟傈:首都师范大学信息工程学院讲师, 分别于2011年、2017年获得北京航空航天大学学士、博士学位. 2013−2015年博士在法国南巴黎大学联合培养. 研究方向为类脑计算和机器人感知与控制. 本文通信作者. E-mail: li.su@cnu.edu.cn

    杨帆:首都师范大学信息工程学院硕士研究生. 主要研究方向为基于事件相机的目标检测. E-mail: yangfannie@cnu.edu.cn

    王向禹:首都师范大学信息工程学院硕士研究生. 主要研究方向为基于事件相机的目标识别与跟踪. E-mail: wangxiangyu@cnu.edu.cn

    郭川东:北京理工大学宇航学院博士研究生. 主要研究方向为复杂柔性航天器动力学、测量与控制. E-mail: guochuandong@bit.edu.cn

    童良乐:首都师范大学信息工程学院硕士研究生, 主要研究方向为类脑器件与电路设计. E-mail: kytll2021@163.com

    胡权:北京理工大学宇航学院副教授. 分别于2009年、2014年获北京航空航天大学学士、博士学位. 主要研究方向为多体系统动力学、空间机器人感知与控制. E-mail: huquan@bit.edu.cn

A Survey of Robot Perception and Control Based on Event Camera

Funds: Supported by National Natural Science Foundation of China (11872011), Stable support of scientific research projects by the State Administration of science, technology and industry for national defense(HTKJ2020KL502013)
More Information
    Author Bio:

    SU Li Assistant Professor at the College of Infomation Engineering, Capital Normal University. She received the B.S. degree and the Ph.D. degree from Beihang University, Beijing, China, in 2011 and 2017, respectively. From 2013 to 2015, she was a visiting Ph.D. student with the University of Paris-Sud, Orsay, France. Her main research interest includes neuromorphic computing, perception and control of robot. Corresponding author of this paper

    YANG Fan Master student at the College of Infomation Engineering, Capital Normal University. Her main research interest includes object detection based on event camera

    WANG Xiang-Yu Master student at the College of Infomation Engineering, Capital Normal University. His main research interest includes object recognition and tracking based on event camera

    GUO Chuan-Dong Ph. D., candidate at the College of Aerospace Engineering, Beijing Institute of Technology. His main research interest includes attitude dynamics, measurement and control of complex flexible spacecraft

    TONG Liang-Le Master student at the College of Infor-mation Engineering, Capital Normal University. His main research interest includes neuromorphic device and circuit design

    HU Quan Tenured Associate Professor at the College of Aerospace Engineering, Beijing Institute of Technology. He received his B.Sc. degree and Ph.D degree from Beihang University in 2009, 2014, respectively. His main research interest includes multibody dynamics, perception and control of space robot

  • 摘要: 事件相机作为一种新型动态视觉传感器, 通过各个像素点独立检测光照强度变化并异步输出“事件流”信号, 它具有数据量小、延迟低、动态范围高等优秀特性, 给机器人控制带来新的可能. 本文主要介绍了近年来涌现的一系列事件相机与无人机、机械臂和人形机器人等机器人感知与运动控制结合的研究成果, 同时聚焦基于事件相机的控制新方法、新原理以及控制效果, 并指出基于事件相机的机器人控制的应用前景和发展趋势.
  • 图  1  本文结构图

    Fig.  1  Structure diagram of this paper

    图  2  旋转圆盘场景下帧相机与事件相机输出对比[2, 3]. 帧相机在每一帧图像中记录全部像素点上的数据, 即使圆盘中大部分区域的信息是无用的; 事件相机仅记录圆盘中的黑点位置, 因此仅对运动的有效信息输出事件.

    Fig.  2  Comparison of output between frame camera and event camera in rotating disc scene[2, 3]. The frame camera records the data on all pixels in each frame image, even if the information in most areas of the disc is useless; The event camera only records the position of black spots in the disc, so it only outputs events for the effective information of motion.

    图  3  DVS原理图

    Fig.  3  Schematic diagram of DVS

    图  4  实验环境. 左侧: 跟踪器的一次视觉快照.右侧: 四杆运动装置[22]

    Fig.  4  experimental environment. left side: a visual snapshot of tracker. right side: four-bar motion device[22]

    图  5  实验装置. (1)事件相机(2)双旋翼直升机(3)黑白圆盘, 黑白圆盘标记了要跟踪的参考基准线[24].

    Fig.  5  Experimental platform. (1) Event camera (2) Dualcopter (3) Black and white disk, which marks the horizon to be tracked[24].

    图  6  (a)无人机实验平台[26](1)事件相机传感器; (2) Intel Upboard板卡; (3)飞行控制器. (b)真实无人机躲避两个飞行障碍物, 四旋翼无人机使用EVDodgeNet进行障碍物躲避, 其中无人机左右两侧的弧线轨迹为同时掷向无人机的两个障碍物, 中间的轨迹为无人机闪躲轨迹[27]. (c)神经网络架构图[27], 其中EVDeblurNet: 事件帧去噪网络, 在模拟数据集上进行训练就可推广到真实场景, 无需重新训练或者微调; EVHomographyNet: 单应性估计网络, 用于自我运动估计, 计算相机的自我运动. 第一个使用事件相机的单应性估计方案; EVSegFlowNet: 物体分割与光流计算网络, 该网络可分割场景中的运动物体并获取其光流信息.

    Fig.  6  (a) UAV experimental platform[26] (1) event camera sensor; (2) Intel Upboard board board; (3) flight controller. (b) The real UAV avoids two flying obstacles, and the quadrotor UAV uses EVDodgeNet to avoid obstacles, in which the arc trajectory on left and right sides of the UAV is two obstacles thrown at the same time, and the middle trajectory is the UAV dodge trajectory[27]. (c) neural network architecture diagram[27], in which EVDeblurNet: event frame denoising network, which can be extended to real scenes by training on simulated data sets without retraining or fine tuning; EVHomographyNet: homography estimation network, which is used to estimate self-motion and calculate self-motion of camera. The first homography estimation scheme using event camera; EVSegFlowNet: an object segmentation and optical flow computing network, which can segment moving objects in the scene and obtain their optical flow information.

    图  7  无人机垂直降落实验装置[35]

    Fig.  7  Experimental device for vertical landing of UAV[35]

    图  8  处理步骤以及抓取场景[41]

    Fig.  8  Processing steps and capture scene of reference [41]

    图  9  实验装置的俯视图(左)和侧视图(右)[42]

    Fig.  9  Top view (left) and side view (right) of the experimental device[42]

    图  10  夹持器及网络示意图[45]

    Fig.  10  Schematic diagram of gripper and network[45]

    图  11  (a) iCub机器人[54] (b) iCub嵌入的事件视觉系统[52] (c) 数据流[52]

    Fig.  11  (a) ICub robot[54] (b) The event vision system embedded in iCub[52] (c) Data flow[52]

    图  12  前馈分类系统的结构[65]

    Fig.  12  Structure of feedforward classification system[65]

    图  13  篮球得分检测框架[68]

    Fig.  13  Basketball score detection framework[68]

    图  14  两层脉冲神经网络结构示意图[74]

    Fig.  14  Schematic diagram of two-layer pulse neural network structure[74]

    图  15  步态模仿系统结构示意图[76]

    Fig.  15  Schematic diagram of gait simulation system[76]

    图  16  铅笔倒立平衡控制系统: 两台DVS事件相机进行线检测(右上角及左下角), 左上角运动平台由中间两部伺服电机驱动[81]

    Fig.  16  Photo of balancer hardware: 2 DVS (right top and bottom left), the motion table (top left) actuated by two servos (center) [81]

    图  17  Summit XL机器人追踪实验[82]

    Fig.  17  Summit XL robot tracking experiment[82]

    图  18  技术发展脉络图. 横轴为感知技术分类, 负横轴为基于事件帧的图像感知算法, 正横轴为基于异步事件流的感知算法; 纵轴为控制技术分类, 正纵轴为基于目标特征的运动规划与控制的传统方法, 负纵轴为基于异步事件流的新型控制算法, 其中使用相似技术路线的文献处于同一象限, 标注是根据机器人类型进行分类并按时间顺序排列, 虚线内文献只涉及基于事件相机的感知技术.

    Fig.  18  Development venation map of technology. The horizontal axis is the classification of perception technology, the negative horizontal axis is the image perception algorithm based on event frame, and the positive horizontal axis is the perception algorithm based on asynchronous event flow; The vertical axis is the classification of control technology, the positive vertical axis is the traditional method of motion planning and control based on target characteristics, and the negative vertical axis is a new control algorithm based on asynchronous event flow. The documents using similar technical routes are in the same quadrant, and the labels are classified according to robot types and arranged in chronological order. The documents in the dotted line only involve the sensing technology based on event cameras.

    表  1  几种事件相机的性能比较表

    Table  1  Performance comparison table of several event cameras

    文献事件相机型号产品年份分辨率(pixels)最高动态范围(dB)功耗(mw)延迟(μs)像素大小(μm2)
    [10]DVS1282008128×128120132−231<3.6
    [17]DVS128020201280×960120150
    [15]ATIS2011304×24014350−175<430×30
    [20]DAVIS3462013346×240120
    [21]DAVIS2402014240×1801305−14<318.5×18.5
    [19]CeleX-V20191280×800120390−470<0.5
    下载: 导出CSV

    表  2  无人机部分实验场景

    Table  2  Experimental scenarios of UAV

    文献无人机型号事件相机处理器或控制器事件流处理方法备注
    基于事件帧
    [2]Parrot AR.Drone 2.0DVSOdroid U2 onboard computerHough变换跟踪速度高达1200°/s
    [22]DVSStandard PC卡尔曼滤波Lie-EKF公式跟踪速度超过2.5 m/s,
    加速度超过25.8 g
    [24]组装DAVIS 240CIntel Upboard Lumenier F4
    AIO flight controller
    Hough变换卡尔曼滤波跟踪速度高达1600°/s
    [27]Intel Aero Ready to
    Fly Drone
    DAVIS 240C
    DAVIS 240B
    NVIDIA TX2 CPU+GPU浅层神经网络碰撞预测时间60 ms
    [37]定制MavTecDVS128Lisa/MX Odroid XU4聚类, 光流算法
    基于事件流
    [25]组装双目DVS 128Odroid U3 quad-core computer高斯核函数卡尔曼滤波碰撞预测时间250 ms
    [26]基于DJI F330定制Insigthness SEEM1Intel Upboard Lumenier F4
    AIO flight controller
    DBSCAN聚类光流算法碰撞预测时间68 ms (2 m处)
    183 ms (3 m处)
    [28]定制Insightness SEEM1Qualcomm Snapdragon Flight
    NVIDIA Jetson TX2
    DBSCAN聚类光流
    算法卡尔曼滤波
    碰撞预测时间3.5 ms
    [35]基于DJI F450DAVIS 346PixRacer Khadas VIM3Hough算法卡尔曼滤波规划时间2.95 ms
    [39]Parrot Bebop2Dual-core ARM cortex A9 processor.脉冲神经网络光流算法神经形态控制器效果良好
    下载: 导出CSV

    表  3  部分基于事件相机的实验设备

    Table  3  Experimental equipment based on event camera

    文献使用传感器机械臂设备方法
    [41]DAVIS240CUR10事件流
    [43]Prophesee Gen3 NeuTouch7 DOF Franka Emika Panda arm事件流+SNN
    [42]DAVIS240CBaxter robot arm事件帧
    [45]DAVIS240CAX-12A Dynamixel servo motor事件帧+DNN
    [46]DAVIS240C ATI F/T sensor(Nano17)Baxter robot arm事件帧+CNN
    [47]DAVIS240C ATI F/T sensor (Nano17)Baxter robot daul-arm事件帧
    [49]DVS128事件帧
    下载: 导出CSV
  • [1] Posch C, Serrano-Gotarredona T, Linares-Barranco B, et al. Retinomorphic event-based vision sensors: Bioinspired cameras with spiking output. Proceedings of the IEEE, 2014, 102(10): 1470–1484 doi: 10.1109/JPROC.2014.2346153
    [2] Mueggler E, Huber B, Scaramuzza D. Event-based, 6-DOF pose tracking for high-speed maneuvers. In: Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. Chicago, USA: IEEE, 2014. 2761–2768
    [3] Cho D IL, Lee T J. A review of bioinspired vision sensors and their applications. Sensors and Materials, 2015, 27(6): 447–463
    [4] Mahowald M. VLSI analogs of neuronal visual processing : a synthesis of form and function. California Institute of Technology, 1992
    [5] Boahen K. A retinomorphic chip with parallel pathways: Encoding increasing, on, decreasing, and off visual signals. Analog Integrated Circuits and Signal Processing, 2002, 30(2): 121–135 doi: 10.1023/A:1013751627357
    [6] Kramer J. An integrated optical transient sensor. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 2002, 49(9): 612–628 doi: 10.1109/TCSII.2002.807270
    [7] Rüedi P F, Heim P, Kaess F, et al. A 128 × 128 Pixel 120-dB Dynamic-Range Vision-Sensor Chip for Image Contrast and Orientation Extraction. IEEE Journal of Solid-State Circuits, 2003, 38(12): 2325–2333 doi: 10.1109/JSSC.2003.819169
    [8] Luo Q, Harris J G, Chen Z J. A time-to-first spike CMOS image sensor with coarse temporal sampling. Analog Integrated Circuits and Signal Processing, 2006, 47(3): 303–313 doi: 10.1007/s10470-006-5676-5
    [9] Chi Y M, Mallik U, Clapp M A, et al. CMOS camera with in-pixel temporal change detection and ADC. IEEE Journal of Solid-State Circuits, 2007, 42(10): 2187–2196 doi: 10.1109/JSSC.2007.905295
    [10] Lichtsteiner P, Posch C, Delbruck T. A 128 × 128 120 dB 15 μs latency asynchronous temporal contrast vision sensor. IEEE Journal of Solid-State Circuits, 2008, 43(2): 566–576 doi: 10.1109/JSSC.2007.914337
    [11] Gallego G, Delbrück T, Orchard G, et al. Event-based vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020: 1–26
    [12] Amir A, Taba B, Berg D, et al. A low power, fully event-based gesture recognition system. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017. 7243–7252
    [13] Conradt J, Cook M, Berner R, et al. A pencil balancing robot using a pair of AER dynamic vision sensors. In: Proceedings of the 2009 IEEE International Symposium on Circuits and Systems. Taipei, Taiwan, China: IEEE, 2009. 781–784
    [14] Xu H, Gao Y, Yu F, et al. End-to-end learning of driving models from large-scale video datasets. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu, HI, USA: IEEE, 2017. 3530–3538
    [15] Posch C, Matolin D, Wohlgenannt R. A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS. IEEE Journal of Solid-State Circuits, 2010, 46(1): 259–275
    [16] 胡生龙. 城市内动态场景下基于双目视觉的无人车同步定位与建图[D], 哈尔滨工业大学, 2020

    Hu Sheng-Long. Simultaneous localization and mapping of unmanned vehicle based on binocular vision in urban dynamic scene [Master dissertation], Harbin Institute of Technology, 2020
    [17] Suh Y, Choi S, Ito M, et al. A 1280×960 Dynamic Vision Sensor with a 4.95-μm Pixel Pitch and Motion Artifact Minimization. In: Proceedings of the 2020 IEEE international symposium on circuits and systems (ISCAS). Seville, Spain: IEEE, 2020. 1–5
    [18] Posch C, Matolin D, Wohlgenannt R. An asynchronous time-based image sensor. In: Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS). Seattle, USA: IEEE, 2008: 2130–2133
    [19] Chen S, Guo M. Live demonstration: CeleX-V: A 1M pixel multi-mode event-based sensor. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Long Beach, CA, USA: IEEE, 2019. 1682–1683
    [20] Serrano-Gotarredona T, Linares-Barranco B. A 128, × 128 1.5% contrast sensitivity 0.9% FPN 3 μs latency 4 mW asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers. IEEE Journal of Solid-State Circuits, 2013, 48(3): 827–838 doi: 10.1109/JSSC.2012.2230553
    [21] Brandli C, Berner R, Yang M, et al. A 240 × 180 130 dB 3 μs latency global shutter spatiotemporal vision sensor. IEEE Journal of Solid-State Circuits, 2014, 49(10): 2333–2341 doi: 10.1109/JSSC.2014.2342715
    [22] Chamorro W, Andrade-Cetto J, Solà J. High-speed event camera tracking. In: Proceedings of the 2020 British machine vision conference(BMVC). Online Proceedings, UK: Springer, 2020(2). 1–12
    [23] Solà J, Deray J, Atchuthan D. A micro Lie theory for state estimation in robotics. arXiv: 1812.01537, 2018
    [24] Dimitrova R S, Gehrig M, Brescianini D, et al. Towards Low-Latency High-Bandwidth Control of Quadrotors using Event Cameras. In: Proceedings of the IEEE International Conference on Robotics and Automation. Paris, France : IEEE, 2020. 4294–4300
    [25] Mueggler E, Baumli N, Fontana F, et al. Towards evasive maneuvers with quadrotors using dynamic vision sensors. In: Proceedings of the 2015 European Conference on Mobile Robots (ECMR). Lincoln, UK: IEEE, 2015. 1–8
    [26] Falanga D, Kim S, Scaramuzza D. How Fast Is Too Fast? The Role of Perception Latency in High-Speed Sense and Avoid. IEEE Robotics and Automation Letters, 2019, 4(2): 1884–1891 doi: 10.1109/LRA.2019.2898117
    [27] Sanket N J, Parameshwara C M, Singh C D, et al. EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras. In: Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA). Paris, France: IEEE, 2020. 10651–10657
    [28] Falanga D, Kleber K, Scaramuzza D. Dynamic obstacle avoidance for quadrotors with event cameras. Science Robotics, 2020, 5(40)
    [29] Mitrokhin A, Fermuller C, Parameshwara C, et al. Event-Based Moving Object Detection and Tracking. In: Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain: IEEE, 2018. 1–9
    [30] Khatib O. Real-time obstacle avoidance for manipulators and mobile robots. Autonomous robot vehicles. New York: Springer, 1986. 396–404
    [31] Lin Y, Gao F, Qin T, et al. Autonomous aerial navigation using monocular visual-inertial fusion. Journal of Field Robotics, 2018, 35(1): 23–51 doi: 10.1002/rob.21732
    [32] Oleynikova H, Honegger D, Pollefeys M. Reactive avoidance using embedded stereo vision for MAV flight. In: Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA). Seattle, WA, USA: IEEE, 2015. 50–56
    [33] Barry A J, Florence P R, Tedrake R. High-speed autonomous obstacle avoidance with pushbroom stereo. Journal of Field Robotics, 2018, 35(1): 52–68 doi: 10.1002/rob.21741
    [34] Huang A S, Bachrach A, Henry P, et al. Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera. Robotics Research. Switzerland: Springer, Cham, 2017. 235–252.
    [35] Eguíluz A G, Rodríguez-Gómez J P, Martínez-de Dios J R, et al. Asynchronous event-based line tracking for time-to-contact maneuvers in UAS. In: Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas, NV, USA: IEEE, 2020. 5978–5985.
    [36] Lee D. Lee's 1976 paper. Perception, 2009, 38(6): 837–858 doi: 10.1068/pmklee
    [37] Pijnacker Hordijk B J, Scheper K Y W, De Croon G C H E. Vertical landing for micro air vehicles using event-based optical flow. Journal of Field Robotics, 2018, 35(1): 69–90 doi: 10.1002/rob.21764
    [38] Scheper K Y W, De Croon G C H E. Evolution of robust high speed optical-flow-based landing for autonomous MAVs. Robotics and Autonomous Systems, 2020, 124: 103380 doi: 10.1016/j.robot.2019.103380
    [39] Hagenaars J J, Paredes-Valles F, Bohte S M, et al. Evolved Neuromorphic Control for High Speed Divergence-Based Landings of MAVs. IEEE Robotics and Automation Letters, 2020, 5(4): 6239–6246 doi: 10.1109/LRA.2020.3012129
    [40] Tavanaei A, Ghodrati M, Kheradpisheh S R, et al. Deep learning in spiking neural networks. Neural Networks, 2019, 111: 47–63 doi: 10.1016/j.neunet.2018.12.002
    [41] Muthusamy R, Ayyad A, Halwani M, et al. Neuromorphic Eye-in-Hand Visual Servoing. IEEE Access, 2021, 9: 55853-55870 doi: 10.1109/ACCESS.2021.3071261
    [42] Rigi A, Baghaei Naeini F, Makris D, et al. A Novel Event-Based Incipient Slip Detection Using Dynamic Active-Pixel Vision Sensor (DAVIS). Sensors, 2018, 18(2): 333 doi: 10.3390/s18020333
    [43] Taunyazov T, Sng W, Lim B, et al. Event-Driven Visual-Tactile Sensing and Learning for Robots. In: Proceedings of Robotics: Science and Systems 2020. Corvallis, USA: MIT, 2020.
    [44] Davies M, Srinivasa N, Lin T-H, et al. Loihi: A Neuromorphic Manycore Processor with On-Chip Learning. IEEE Micro, 2018, 38(1): 82–99 doi: 10.1109/MM.2018.112130359
    [45] Baghaei Naeini F, AlAli A M, Al-Husari R, et al. A Novel Dynamic-Vision-Based Approach for Tactile Sensing Applications. IEEE Transactions on Instrumentation and Measurement, 2020, 69(5): 1881–1893 doi: 10.1109/TIM.2019.2919354
    [46] Baghaei Naeini F, Makris D, Gan D, et al. Dynamic-Vision-Based Force Measurements Using Convolutional Recurrent Neural Networks. Sensors, 2020, 20(16): 4469 doi: 10.3390/s20164469
    [47] Muthusamy R, Huang X, Zweiri Y, et al. Neuromorphic Event-Based Slip Detection and Suppression in Robotic Grasping and Manipulation. IEEE Access, 2020, 8: 153364–153384 doi: 10.1109/ACCESS.2020.3017738
    [48] Vasco V, Glover A, Bartolozzi C. Fast event-based Harris corner detection exploiting the advantages of event-driven cameras. In: Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, Korea (South): IEEE, 2016. 4144–4149
    [49] Kumagai K, Shimonomura K. Event-based Tactile Image Sensor for Detecting Spatio-Temporal Fast Phenomena in Contacts. In: Proceedings of the 2019 IEEE World Haptics Conference (WHC). Tokyo, Japan: IEEE, 2019. 343–348
    [50] Monforte M, Arriandiaga A, Glover A, et al. Where and When: Event-Based Spatiotemporal Trajectory Prediction from the iCub’s Point-Of-View. In: Proceedings of the IEEE International Conference on Robotics and Automation. Paris, France: IEEE, 2020. 9521–9527
    [51] Iacono M, Weber S, Glover A, et al. Towards Event-driven Object Detection with Off-the-shelf Deep Learning. In: Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain: IEEE, 2018. 1–9.
    [52] Bartolozzi C, Rea F, Clercq C, et al. Embedded neuromorphic vision for humanoid robots. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Colorado Springs, CO, USA: IEEE, 2011. 129–135
    [53] Glover A, Vasco V, Iacono M, et al. The Event-Driven Software Library for YARP—With Algorithms and iCub Applications. Frontiers in Robotics and AI, 2018, 4: 1-7
    [54] ICub robot - iCub - IIT[Online], available: https://icub.iit.it/products/icub-robot, July 20, 2021
    [55] Wiesmann G, Schraml S, Litzenberger M, et al. Event-driven embodied system for feature extraction and object recognition in robotic applications. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. Providence, RI, USA: IEEE, 2012. 76–82
    [56] Glover A, Bartolozzi C. Event-driven ball detection and gaze fixation in clutter. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems(IROS). Daejeon, Korea: IEEE, 2016. 2203–2208
    [57] Vasco V, Glover A, Mueggler E, et al. Independent motion detection with event-driven cameras. In: Proceedings of the 2017 18th International Conference on Advanced Robotics, (ICAR). Hong Kong, China: IEEE, 2017. 530–536
    [58] Rea F, Metta G, Bartolozzi C. Event-driven visual attention for the humanoid robot iCub. Frontiers in Neuroscience, 2013, 7: 1–11
    [59] Iacono M, D’Angelo G, Glover A, et al. Proto-object based saliency for event-driven cameras. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems. Macau, China: IEEE, 2019. 805–812
    [60] Glover A, Bartolozzi C. Robust visual tracking with a freely-moving event camera. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems. Vancouver, BC, Canada: IEEE, 2017. 3769–3776.
    [61] Gibaldi A, Canessa A, Chessa M, et al. A neuromorphic control module for real-time vergence eye movements on the iCub robot head. In: Proceedings of the 2011 11 th IEEE-RAS International Conference on Humanoid Robots. Bled, Slovenia: IEEE, 2011. 543–550
    [62] Vasco V, Glover A, Tirupachuri Y, et al. Vergence control with a neuromorphic iCub. In: Proceedings of the IEEE-RAS International Conference on Humanoid Robots. Cancun, Mexico: IEEE, 2016. 732–738
    [63] Akolkar H, Valeiras D R, Benosman R, et al. Visual-Auditory saliency detection using event-driven visual sensors. In: Proceedings of the 2015 International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP). Krakow, Poland: IEEE, 2015. 1–6
    [64] Zhao B, Ding R, Chen S, et al. Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(9): 1963–1978 doi: 10.1109/TNNLS.2014.2362542
    [65] Zhao B, Yu Q, Yu H, et al. A bio-inspired feedforward system for categorization of AER motion events. In: Proceedings of the 2013 IEEE Biomedical Circuits and Systems Conference (BioCAS). Rotterdam, Netherlands: IEEE, 2013. 9–12
    [66] Zhao B, Chen S, Tang H. Bio-inspired categorization using event-driven feature extraction and spike-based learning. In: Proceedings of the International Joint Conference on Neural Networks(IJCNN). Beijing, China: IEEE, 2014. 3845–3852
    [67] Nan Y, Xiao R, Gao S, et al. An Event-based Hierarchy Model for Object Recognition. In: Proceedings of the 2019 IEEE Symposium Series on Computational Intelligence. Xiamen, China: IEEE, 2019. 2342–2347
    [68] Shen J, Zhao Y, Liu J K, et al. Recognizing Scoring in Basketball Game from AER Sequence by Spiking Neural Networks. In: Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN). Glasgow, UK: IEEE, 2020. 1–8
    [69] Massa R, Marchisio A, Martina M, et al. An Efficient Spiking Neural Network for Recognizing Gestures with a DVS Camera on the Loihi Neuromorphic Processor. In: Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN). Glasgow, UK: IEEE, 2020. 1–9
    [70] Camuñas-Mesa L A, Domínguez-Cordero Y L, Linares-Barranco A, et al. A configurable event-driven convolutional node with rate saturation mechanism for modular ConvNet systems implementation. Frontiers in Neuroscience, 2018, 12: 63 doi: 10.3389/fnins.2018.00063
    [71] Camunas-Mesa L A, Linares-Barranco B, Serrano-Gotarredona T. Low-power hardware implementation of SNN with decision block for recognition tasks. In: Proceedings of the 2019 26th IEEE International Conference on Electronics, Circuits and Systems (ICECS). Genoa, Italy: IEEE, 2019. 73–76
    [72] Pérez-Carrasco J A, Zhao B, Serrano C, et al. Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate coding and coincidence processing - Application to feedforward convnets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(11): 2706–2719 doi: 10.1109/TPAMI.2013.71
    [73] Diehl P U, Cook M. Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in Computational Neuroscience, 2015, 9: 1–9
    [74] Iyer L R, Basu A. Unsupervised learning of event-based image recordings using spike-timing-dependent plasticity. In: Proceedings of the International Joint Conference on Neural Networks. Anchorage, AK, USA: IEEE, 2017: 1840–1846
    [75] Cheng R, Mirza K B, Nikolic K. Neuromorphic robotic platform with visual input, processor and actuator, based on spiking neural networks. Applied System Innovation, 2020, 3(2): 28 doi: 10.3390/asi3020028
    [76] Ting J, Fang Y, Lele A, et al. Bio-inspired Gait Imitation of Hexapod Robot Using Event-Based Vision Sensor and Spiking Neural Network. In: Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN). Glasgow, UK: IEEE, 2020. 1–7.
    [77] Youssef I, Mutlu M, Bayat B, et al. A Neuro-Inspired Computational Model for a Visually Guided Robotic Lamprey Using Frame and Event Based Cameras. IEEE Robotics and Automation Letters, 2020, 5(2): 2395–2402 doi: 10.1109/LRA.2020.2972839
    [78] Blum H, Dietmüller A, Milde M, et al. A neuromorphic controller for a robotic vehicle equipped with a dynamic vision sensor. In: Proceedings of the Robotics Science and Systems 2017 Conference. Massachusetts, USA: Proceedings of Robotics: Science and Systems 2017, 2017. 1–9
    [79] Renner A, Evanusa M, Sandamirskaya Y. Event-Based Attention and Tracking on Neuromorphic Hardware. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Long Beach, CA: IEEE, 2019. 1709–1716
    [80] Gehrig M, Shrestha S B, Mouritzen D, et al. Event-Based Angular Velocity Regression with Spiking Networks. In: Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA). Paris, France: IEEE, 2020. 4195–4202.
    [81] Conradt J, Berner R, Cook M, et al. An embedded AER dynamic vision sensor for low-latency pole balancing. In: Proceedings of the 2009 IEEE 12th International Conference on Computer Vision Workshops. Kyoto, Japan: IEEE, 2009. 780–785
    [82] Moeys D P, Corradi F, Kerr E, et al. Steering a predator robot using a mixed frame/event-driven convolutional neural network. In: Proceedings of the 2016 Second International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP). Krakow, Poland: IEEE, 2016. 1–8
    [83] Delbruck T, Lang M. Robotic goalie with 3 ms reaction time at 4% CPU load using event-based dynamic vision sensor. Frontiers in Neuroscience, 2013, 7: 1–7
    [84] Delbruck T, Lichtsteiner P. Fast sensory motor control based on event-based hybrid neuromorphic-procedural system. In: Proceedings of the 2007 IEEE International Symposium on Circuits and Systems. New Orleans, LA, USA: IEEE, 2007. 845–848
    [85] Mueller E, Censi A, Frazzoli E. Low-latency heading feedback control with neuromorphic vision sensors using efficient approximated incremental inference. In: Proceedings of the 2015 54th IEEE Conference on Decision and Control (CDC). Osaka, Japan: IEEE, 2015. 992–999
    [86] Mueller E, Censi A, Frazzoli E. Efficient high speed signal estimation with neuromorphic vision sensors. In: Proceedings of the 2015 International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP). Krakow, Poland: IEEE, 2015. 1–8
    [87] Delbruck T, Pfeiffer M, Juston R, et al. Human vs. computer slot car racing using an event and frame-based DAVIS vision sensor. In: Proceedings of the 2015 IEEE International Symposium on Circuits and Systems (ISCAS). Lisbon, Portugal: IEEE, 2015. 2409–2412
    [88] Censi A. Efficient neuromorphic optomotor heading regulation. In: Proceedings of the 2015 American Control Conference (ACC). American Automatic Control Council, Chicago, IL, USA: IEEE, 2015. 3854–3861
    [89] Singh P, Yong S Z, Gregoire J, et al. Stabilization of linear continuous-time systems using neuromorphic vision sensors. In: Proceedings of the 2016 IEEE 55th Conference on Decision and Control (CDC). Las Vegas, NV, USA: IEEE, 2016. 3030–3036
    [90] Singh P, Yong S Z, Frazzoli E. Stabilization of stochastic linear continuous-time systems using noisy neuromorphic vision sensors. In: Proceedings of the American Control Conference(ACC). Seattle, WA, USA: IEEE, 2017. 535–541
    [91] Singh P, Yong S Z, Frazzoli E. Regulation of linear systems using event-based detection sensors. IEEE Transactions on Automatic Control, 2019, 64(1): 373–380 doi: 10.1109/TAC.2018.2876997
  • 加载中
计量
  • 文章访问数:  332
  • HTML全文浏览量:  247
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-03-31
  • 修回日期:  2021-11-26
  • 网络出版日期:  2021-12-30

目录

    /

    返回文章
    返回