Computational Optical Sensing and Processing Laboratory



Leader: Ákos Zarándy PhD DsC;
Researchers: Zoltán Nagy PhD , András Kiss PhD;
PhD student: Tamás Zsedrovits and several BSc and MSc students;
Engineer: Máté Német;



    The Unmanned Arial Vehicles (UAVs) went through an intensive development period in the last decade and in many (mostly military) applications they have proved their reason for existence. Nowadays more and more commercial application opportunities are opening for UAVs, however the process is slowed down due to the lack meeting certain safety standards. One of these safety standards is the automatic collision avoidance system, which is required for the autonomous UAVs as well as the remotely piloted ones [1][4].

    Radar based automatic collision avoidance system exists for a long time and all the larger commercial airliners are already equipped with it. Recently the large remotely piloted military UAVs (e.g. Predator, Global Hawk) are also equipped with a multiple sensors (TCAS, and radar) based collision avoidance system. These systems work perfectly on on large aircrafts however, radar based solutions are too expensive and too bulky for small or medium sized UAVs. An affordable alternative of radar based detection of the intruder aircraft is the visual detection. In our laboratory, in cooperation with the SCL lab, we are preparing a visual based collision avoidance system, which is designed for small and medium sized UAVs.

    The sense ad avoid system contains a closed loop with the embedded collision avoidance capability based on visual detection of the approaching aircraft. The organization of the system is as follows. The input images are recorded by the Cameras. The recorded pictures are transmitted by the Image Acquisition block to the Preprocessing block by which the pictures are filtered. The next step of the processing is the Detection. The images are processed by image processing algorithms to detect the approaching objects. Data Association & Tracking is responsible for the combination of the orientation and angle of attack data of the approaching object calculated by Detection and the own position and inertial data measured by onboard INS/GPS (Inertial Navigation System/Global Positioning System).

    We have built a flyable system for performing image capturing, image storing, and image processing (detection of remote aircafts). It contains the cameras, an interface board, and FPGA board, a solid state drive. We have selected 5 pieces of 1.2 megapixel grayscale global shutter cameras as the image sources. The joint image is covers about 240×65 field of view. The entire resolution is 4650×1280. To be able to hold the cameras in the required position, and avoid cross camera vibration, we have designed and manufactured a solid aluminum camera holder.

    In our system we are implementing a multi-adaptive recognition algorithm which detects aircrafts against blue sky or cloudy backgrounds. From the very beginning of the algorithm design, we kept in mind the strict power, volume and other constraints of an air-born UAV application. To be able to fulfill these constraints, we decided to use many-core cellular array processor, implemented in FPGA. Therefore we selected topographic operators, which well fit in this environment.

    Original image with a remote aircraft agains cloudy background

    Detected aircraft (green)


    Small intruder against cloudy background (raw)

    Small intruder detected against cloudy background

    Large intruder against cloudy background (raw)

    Large intruder detected against cloudy background



    • T. Zsedrovits, Á. Zarándy, B. Vanek, T. Péni, J. Bokor, T. Roska, „Collision avoidance for UAV using visual detection” ISCAS-2011, Rio de Janeiro, Brasil, 2011. pdf
    • B. Vanek, T. Péni, Á. Zarándy, J. Bokor,T. Zsedrovits, T. Roska, "Performance analysis of a vision only sense and avoid system for small UAVs.”, AIAA guidance, navigation, and control conference. Portland, 2011. pdf
    • Z. Nagy, A. Kiss, Á. Zarándy, B. Vanek, T. Péni, J. Bokor, T. Roska “Volume and power optimized high-performance system for UAV collision avoidance” ISCAS-2012, Seoul, Korea, 2012. pdf
    • Ákos Zarándy, Tamás Zsedrovits, Zoltán Nagy, András Kiss, Tamás Roska “Visual sense-and-avoid system for UAVs”, CNNA 2012, Turin, Italy, 2012. pdf
    • Tamas Zsedrovits, Akos Zarandy, Balint Vanek, Tamas Peni, Jozsef Bokor, Tamas Roska, “Estimation of Relative Direction Angle of Distant, Approaching Airplane in Sense-and-Avoid”, J Intell Robot Syst, July, 2012. pdf
    • Supporting grants

      The project is supported by ONR Grants N62909-11-1-7039, N62909-10-1-7081.