The challenge of implementing the video processing of such a system in hardware is that present object detection software takes advantage of the sheer power of modern processors, using very complicated and computationally expensive algorithms.  In order to be implemented in hardware, these complex algorithms needed to be simplified without greatly affecting their effectiveness in detecting objects.

In this research project, a Field-programmable Gate Array (FPGA) on an Altera DE2-70 Development Education board was used to develop the custom hardware required to perform this video processing.  The Altera Nios II embedded processor system was used to perform all hardware interaction tasks necessary on the DE2-70 board and the custom hardware was constructed as modules inside the Nios II system.  The theory of object detection was broken up into three portions:  background subtraction, noise filtering, and finally, object detection.  The background subtraction was accomplished in software on the Nios II processor, but in very simple code fast enough for an FPGA to efficiently execute.  The noise filtering and object detection modules were then written in the Verilog hardware description language.

The final product successfully identified single objects in the video feed while occupying only 16% of the FPGA’s logic elements and 14% of memory.  The video processing was accomplished within 99,216 cycles per frame, or 5,952,960 cycles per second at 60 frames per second.  The Nios II/f processor used can complete 51 million instructions per second, making this system require approximately 10% of the computational ability of the FPGA.

Further research should address the process of background subtraction. The current process involves capturing a single frame from the live video feed.  This is prone to noise and requires user input.  In addition, this process is the last portion of the project still done in software.  Also, the project currently only identifies one object.  If multiple objects are in the frame, they are seen as a single object.  Future research should include a method to identify multiple objects.