ECE 5725 Final Project: Autonomous Nerf Aiming System #### by Daniel Sachs & Divya Gupta ############################################################################## import time import cv2 import numpy as np import RPi.GPIO as GPIO import pigpio import sys import subprocess import pygame from pygame.locals import * import os from PIL import Image Change in panning servo's PWM duration depends on # difference between current x position and center type options: "Arduino_bare", "Shield_v4", "Shield_v6", "Shield_v7", "Standalone_v3", "Standalone_v5", "Standalone_v7", "Standalone_v8" The gun is mechanically very similar to the Nerf Rapidstrike rifle, using a motor to drive an air piston that launches each dart. It's a quieter and more reliable firing system than the flywheels used in guns like the Nerf Stryfe, which spins up a pair of wheels into which darts are fed by a lever connected to the trigger. The Terrascout can launch between two and three darts per second, sending them flying at a distance of about 50 feet. As the system is based upon computer vision, we needed a reliable camera that could be attached to the barrel of the gun. We first investigated the 8 Megapixel Pi camera. After plugging and unplugging the camera multiple times during testing, it stopped connecting to the system. As we had access to a Logitech C920 HD Pro Webcam, we decided to integrate it into our system instead of the Pi camera. This camera interfaces over USB to the Raspberry Pi so it was easy to wire and mount to the system since we did not need a long ribbon cable. One problem we ran into when using the USB camera is that the OpenCV capture function has an internal 5 frame buffer. This meant that when we tried to get a image, it was out of date by the time we accessed it. The solution we came up with to get one current frame is to read 5 frames, throwing away the first 4 and only keeping the 5th frame. Another issue we encountered with the camera was that it would occasionally not turn on while our program was running, especially when it had been idle for some time; we did not come up with a permanent solution to this, however unplugging and replugging the camera and restarting the program usually worked. Electrical System

Nerf Gun Cardboard Box Battle! Cole Attacks Ethan with Nerf Nerf Gun Cardboard Box Battle! Cole Attacks Ethan with Nerf

To generate the PWM signal, we need to specify a frequency and duty cycle. The servo expects 20 ms ‘off’ time, so the equations for the frequency and duty cycle are given below, where ‘pulse’ is ‘on’ time. Pulses of 1.5ms turn the motor to 90 degrees; shorter pulses move the servo to 0 degrees and longer pulses move towards 180 degrees. Pulse length is set based on how far off-center the target currently is from the reference point of the aiming system. The camera is mounted on the Nerf gun, and its center is aligned with the barrel. Therefore, we set the change in pulse width to be the difference between the current x and y coordinates of the center of the target face and (160, 120), the center of the frame. We limited the minimum and maximum pulse lengths for each servo to the range we found it could smoothly travel. We also limited the maximum change in pulse width in one time step so that the servo movement would not be too jerky. That said, everything comes together to make the Terracout a really enjoyable toy. It has all the advantages of a surprisingly peppy remote control vehicle and a Nerf gun you can fire from across the room, all with a camera that lets you watch whatever you're aiming at. It's a fantastic spy and artillery device for anyone who wants to wage warfare against their siblings or coworkers.

Computer Vision Design/Testing

While well-known Pi community member David Pride admits that Nerf guns hadn’t been invented when he was a youngster, his interest was sparked when he saw two tables full of Nerf gear at a car boot sale. “I started wondering whether you could operate the trigger mechanism with a servo – turns out you can!” Several important lessons we learned over the course of this project and may be of use for future students are that the Haar cascades classifier and Local Binary Patterns Histogram algorithm were easy to implement in OpenCV and relatively accurate for our purposes. We were stumped at first by the camera lag before we learned about the five frame internal buffer on the capture function; our way of circumventing this issue by throwing away four frames so that we always use a current frame seemed to work well but may not be effective if a higher frame rate is necessary. Additionally, we found that using hardware PWM through the PiGPIO library greatly improved the stability of our servos compared to software PWM. Controlled manually using a wireless PS3 joypad, FRED-209 can fire multiple foam darts at the chosen target(s). Its twin motors are driven using a ZeroBorg board, while the firing servo is connected directly to the GPIO 18 PWM pin on the Raspberry Pi. A tilt mechanism for aiming is controlled by the joypad’s shoulder buttons. The first step in programming our system was to set up the system to capture a picture and detect where the faces are inside of the image. To do this, we used OpenCV, a popular open source library for computer vision which has many pre-trained classifiers for object detection, and referenced an example using face detection from TheCodacus.com (referenced below). We chose to use the Haar cascade classifier for face detection, using the provided haarcascade_frontalface_default.xml file from OpenCV. It only works on gray scale images, so after capturing each image we convert them to grayscale with OpenCV. Then we ran the classifier which returned the location of a bounding box around each face. During testing, we displayed the image with this box on our screen and found that this classifier worked reasonably well to detect faces when the person is looking directly into the camera. Facial Recognition - Collecting images In my opinion, a 5 kg spring works best (the original one is a two kilo spring). I tried a 7 kilo spring in one, and although it was strong, it resulted in a lot of mis-firing.

Nerf – Hasbro Pulse Nerf – Hasbro Pulse

David describes the robot’s public debut at the Cotswold Jam as ‘controlled chaos’. “It went down extremely well. I built some ‘evil alien’ targets to give the participants something to aim at – apart from each other!” define disablePlateDelay 5000 // how long to disable sentry when plate is pressed (in milliseconds)My main aim was for it to look awesome; which was difficult to achieve without removing the front part of the gun. But, I managed to accomplish this without removing the barrel, and I’m really happy with how it turned out! state = 0 elif ( not ( 1 in targets) and ( 1 in civilians) and x >cols_left[ 0] and x rows_up[ 0] and y

Nerf Targeting System - Cornell University ECE 5725 Project - Nerf Targeting System - Cornell University

elif ( not ( 6 in targets) and ( 6 in civilians) and x >cols_left[ 2] and x rows_up[ 1] and y cols_left[ 0] and x rows_up[ 1] and y

Nerf Guns of 2023 | Tested by GearLab The 5 Best Nerf Guns of 2023 | Tested by GearLab

Continuing work on the project, David plans to power it with LiPo batteries – “It currently runs on 14 (!) AA batteries which really don’t last very long as the drive motors and flywheel motors are pretty greedy.”Connecting the two servos and the electronic trigger to the Arduino is relatively simple. The wiring is shown in the first picture. The 1k resistor protects the Ardunio from electric shorts and the diode prevents stray electricity and sparks. You can of course leave your circuit on the breadboard, but I decided to use a perfboard. elif ( not ( 2 in targets) and ( 2 in civilians) and x >cols_left[ 1] and x rows_up[ 0] and y