Nerf automatic sentry gun with face detection and WiFi control

SFG_Electronics

I am a full time student working towards my Bachelor of Science degree in IT: Software (basically, this program recently got broken down into both IT and Software Development, so this program contains the core values of the other two).  I am currently studying both Python and Java.  I recently discovered the Arduino system and some of the amazing things that can be done with these tiny boards.  But for my latest project, I found that they just can't do it all.

So, my son has a very strong interest (he's 11) in everything I do.  When he found out some of the projects I was making with Arduino, he decided that he wanted to learn along with me.  When I asked him what he wanted to build for his first project, he said "an automatic sentry gun that shoots Nerf darts".  I have to applaud him for the ambitiousness!  I of course replied to him with "no problem!  That will be easy!"  I later found I was partially wrong...

I have gone through hundreds of web pages and forums, and I have decided that "face detection and tracking" with a camera is going to be the best route for this.  There is not an affordable (when I say affordable, I mean to a full-time student living on limited student loans) Arduino package available that has a proper camera interface, much less be able to "track".  Although there is a lot of support in the RPi community, I decided to go with something "different".  I enjoy the challenge to find new ways to do things, and, although I am not against asking for help, I find myself spending considerable time trying to figure things out on my own.  

So here is the project details:

• I have already figured out the "gun" and firing mechanism.  It will consist of aluminum and pvc tubing, and will be able to accept a standard (or I am looking into making a high capacity) Nerf magazine, and shoot the Nerf Elite (blue) darts. The only "electronic" part of the "gun" will be a single continuous rotation servo (modified to still be positionable with an optical encoder from an old mouse) to work both the plunger assembly and the bolt assembly.  I have just ordered the servo, and will begin prototyping this part very soon.  I am contemplating adding a switch (I have some end-stop switches with a roller on the end) in the breach so it will never "dry fire".  With this servo and setup, it should be able to reliably fire 3 rounds in approximately 2 seconds.  I want it to then either wait 3 seconds and verify the target is still there, or immediately turn to the next "face" and continue firing in three round bursts.
  

• I haven't decided yet on what I want to do for the camera.  I am thinking either webcam, or a camera module for the Pi.  I have found some interesting chips available that add functionality to just about any MIPI compatible camera, and they INCLUDE hardware face detection (to unload some tasks off of the CPU).
  

• *SAFETY* is a big concern here.  There will be several switches in place to prevent operation of the loading mechanism if the breach is "open" (no magazine).  Also, as I am not looking to hurt anyone with this, the camera will be angled a bit higher than the dart trajectory (so we don't boink someone's eye out!).  The firing power will be set to a safe level (I have seen videos where modified or custom Nerf guns break 400 feet/second at the muzzle, I don't have a chronograph, but I want to set it at 120 feet/sec or lower).
  

The rest of the unit will consist of:

• A base, with either 3 or 4 foldable legs
• A servo and bearing plate system, to rotate the turret
• A folding "leg", powered by two servos, so the entire unit can move up/down, and fold down nicely when not being used.  This "leg" will be made out of carbon fiber (to not only provide strength, but to also give it that "tough, hi-tech" look
• A servo to tilt the gun assembly up/down
• A laser pointer (either blue or green, just for looks)
• A couple different pushbutton switches and LEDs (for various settings and manual control)
• A speaker, to provide audible notifications
• A thermal sensor, so it can go to sleep, and then "wake up" when someone enters the room (possibly, I'm not 100% sure this is doable on our budget considering the range we are looking for)
• I believe I will be building some oil-filled shocks to to help compensate for the shock generated when this thing fires, as well as some springs to provide stability and assistance for the servos
  

• I figure if we are gonna go this far, let's make it a streaming webserver, so we can either log into a web page or build an app to give it "remote control" capabilities with live streaming video and full control.
  

Yes, this is a very complex and ambitious project for my son and I, both being noobs with this stuff, but the way I see it, you gotta start somewhere, and this is MUCH more interesting than learning how to print "Hello, World!" on the screen in yet ANOTHER language...lol!  At this time I am not sure which Orange Pi would be the best for this project, although I am leaning towards the PC Plus model, especially with its built-in flash storage.  I know all of them are capable of the video and servo control, but it would be nice (and easier) to have one that is already set up for audio output, WiFi, and conventional display (for initial setup and any future changes necessary).

This project will be fully documented and recorded, so any and all procedures will be posted on YouTube, here, and some other forums/websites.  I hope that my work will aid in the advancement and improvement of face detection/recognition and servo control, as well as everything that I will learn along the way, as well as what my son can learn as well.

I look forward to reading your comments and any "constructive" criticism.  If you have any questions for me, please don't be afraid to ask!  Thank you for your support, and don't forget to vote!

Bighairyg

This is great!  I know my kids would absolutely love it (they are both Nerf fanatics!).  But, safety would be a big concern of mine also.  How do you plan on 'not' shooting someone in the face, like a toddler, or even an adult for that matter?  Also, how do you plan on powering this setup?

SFG_Electronics

Once the firing mechanism is set up, I will perform a series of measurements regarding trajectory of the darts.  This will be calculated out to the average trajectory and distance when the gun is level.  I plan on implementing an approximate focal distance to the tracked face(s), and will use this information to position the face(s) in the frame based on distance which will correspond to the pre-measured trajectory and the angle of the gun.  Yeah, I guess you could always have a stray shot, but then it's not much different than manually shooting with a real Nerf gun....besides, unless a 'new' face enters the frame, it will only fire 3 shots before stopping and waiting for a new face.  I've also been pondering on whether or not to add a proximity sensor, so if someone were to get too close it would just shut down.

As far as power, I came across a 12 volt 10 amp power supply for an industrial LED application.  I will be using this running straight off of mains power, and will be using regulated DC-DC buck converters to drop the voltage down to 7.4v for the servos (and for whatever else needs to be powered).  I didn't realize it, but the servos I have have a stall current of 2.5 amps at 7.4v...  I know I won't be pushing them that hard ever, but it is nice to know that I possibly could if necessary (five servos @ 2.5A is 12.5A total...I forgot to mention that the power supply is dual channel, meaning it takes main voltage on one side, and has DUAL 12v 10A outputs on the other).  It would be nice to maybe adapt this for batteries later on, but with my current budget I have to do with what I've got on hand (or can get for free).  My son only plays with the Nerf guns inside anyways (I let him out with a couple one day, and they came back either broken or packed full of mud....lol).

SFG_Electronics

On second thought, I do believe I will be omitting the laser.  If anything, I might go with a focused LED projection instead.  It just dawned on me that my 3 year old daughter might just catch a laser in the eye, and that could be very bad....  Maybe there is a way to reduce the output to make it non-dangerous but still visible....I will have to do some research on that.

JaydenHudson

It many unique features, including its location, which is close to major tourist attractions such as Victoria Harbour and Central, and people can hop over to this site to solve their academic queries. The campus is also located within walking distance from MTR stations, which makes it an ideal place for students who like to travel and explore local culture.

charlly

The Nerf automatic sentry gun with face detection and WiFi control combines fun and technology. It uses facial recognition to track  nevadas bill to prohibit firearms  targets and can be controlled remotely via WiFi. This innovative device adds excitement to Nerf battles, providing a unique and interactive experience for enthusiasts of all ages.