Finished!

As of June 1st 2012 I am officially finished with the ArduiPhone Project. I would like to use this final blog post to write a short reflection and summary of this project.

At the beginning of this academic year, I had an idea: to design a project that would be fun and educational while still meeting the requirements for my final thesis. My goal was to merge  the knowledge I had learned in the previous year (while completing my Graduate Diploma in computer engineering) with iOS application development. I had always been interested in programming applications for the iPhone and figured that this would be a perfect time to start.

ArduiPhone started with a serious brainstorming session on one big piece of paper. After a few hours, the list of features was endless. The hardest part was choosing the topics that would be feasible within the one-year timespan of the project. Eventually the decision was made to use the following important design components which have all been successfully implemented in the final prototype of ArduiPhone:

  1. Java application written to control ArduiPhone over Wi-Fi from a host computer.
  2. iOS application written for iPhone to accept control from Java application.
  3. Arduino microcontroller that enables communication between the iPhone and the motors and sensors (with Redpark serial cable).
  4. Real-time video streaming.
  5. Camera control via Java application.
  6. Image capture and save to iPhone or email to specified account.
  7. Option to use PS3 controller for driving and camera operation.
  8. Ready-LED and reverse buzzer on-board.
  9. Motion detection and obstacle avoidance via on-board infrared sensor with intruder alert email image capabilities.

As I previously mentioned, I wanted to be able to combine as much of my previous course work as possible into this project while still keeping it fun and interesting.

Over the course of the year while completing this project I have found that if you put your mind to something, it really is possible. Two years ago I graduated college with a Bachelors Degree in Mechanical Engineering. I decided that I wanted to focus on a different area of study and found the perfect course offered at the University of Limerick. Today, I have not only received exceptional grades, but my knowledge has expanded vastly. Within this project alone, I have successfully put together multiple topics that are included under the Masters of Computer Engineering course syllabus. I have independently learned three new programming languages and researched into a multitude of background topics involved with software application development and implementation. And we can’t forget, I have also successfully designed and implemented the ArduiPhone. This course allowed me the opportunity to complete a project that I could have only ever dreamed of.

Below I have included a video demonstration of the working ArduiPhone.

Please feel free to post questions in the comments below and I will try my best to answer everything. If you are interested in reading my thesis, please do not hesitate to contact me as well.

If you’re just coming across my blog now, I’d like to invite you to take a read through my previous posts to get a feel for how exactly this project was completed. However, if you have been following my process throughout this project implementation, I’d like to thank you for your continued support.

Pictures for Thesis

I just finished taking the pictures that are going to be in the final copy of my thesis.

Front View

Side View

 

Also, I handed up the final rough draft to my supervisor to be looked over yesterday. If all goes well, I will have a final version printed up by Friday evening!

SUCCESS!

Today, after sorting out some technical difficulties, I finally got my application to run on Rachael’s iPhone 4S!

EVERYTHING WORKS PERFECTLY!

I tested each component of the iOS, Java, and Arduino applications and every feature works exactly as intended. I also got Reflection app set up on my computer so that I can properly stream the live video over Wi-Fi too!

I will run some final tests before my presentation, but for now its all working!

THANKS RACHAEL!!!!

Also, for anyone who is interested, I finished exams on Friday and have been working on completing my thesis for the past three days. I finished chapters 1-4 on Tuesday, 5 yesterday, and 6 today. I plan on finishing my 7th and final chapter before I go home today. Then I will write the introduction and conclusion tomorrow and have a complete rough draft by the end of Monday.

I plan on submitting an electronic copy of my thesis to my supervisor on Tuesday the 29th and a hard bound copy by Friday June 1st. Then I plan on giving the presentation the first week in June!

I will post a video of the working demonstration and my presentation as soon as everything is complete!

 

UPDATE: As of 6:00 PM, Chapter 7 is complete!

Video Streaming and Controller Updates

In my last post I had mentioned that I had found a solution for video streaming. Earlier last week, I met with my supervisor to discuss the details of my project. He helped convince me that implementing M-JPEG streaming from scratch could be a Master’s project on its own. Therefore, I have decided to use an application offered by Apple on the iPhone 4s called “AirPlay Mirroring”. The application allows the user to stream all content that is displayed on their iPhone to an Apple TV over WiFi. It basically allows the user to have a second screen for their iPhone.

After doing some research I found a piece of software called “Reflection App” that allows Airplay Mirroring to work on a Mac. With the use of these two applications (and a borrowed iPhone 4s), I will be able to successfully implement my project!

Over the weekend, I had the idea to use my Playstation 3 controller to control the ArduiPhone. I did a bit of research and found an application called “ControllerMate” that allows the user to map the buttons of an external device to certain keys on the keyboard or a combinations of keys. This application is most often used for gamers who want to customize their controllers for use with their Mac. After connecting my PS3 controller to my Mac over bluetooth, I proceeded to map each button on the controller. The mapping is as follows:

– Arrows and the left joystick are mapped to the arrows on the keyboard (up is forward, down is reverse, etc…)

– Right joystick is mapped to control the mouse (left click on click of joystick)

– ‘X': Capture Image

– ‘Square': Email Image

– ‘Triangle': Start Camera

– ‘O': Stop Camera

This week I will be spending time testing the complete working ArduiPhone and fix any bugs in the software that I come across. I am also continuing to write my thesis as I wrap things up with this project.

User Interface Updates

Today I made some updates to the user interface of both the iPhone app and the Java application. I also completely rewrote the Java application to run more efficiently.

Updated iOS User Interface

Updated Java User Interface

Next week I will write up the other advancements I made throughout the course of this week.

(Hint: I have found a solution for video streaming!)

Updated Sensor Algorithm

Last week I had an idea about how to address the accuracy of the IR sensors. I was having issues detecting motion, partially due to the simplicity of my algorithm. I was continually checking the sensor reading and comparing it to a predetermined value that I set after running some test cases. The problem was the fact that the reading from the sensor is dependent on the input voltage to the sensor. The input voltage is not held completely constant due to the other components attached to my Arduino. Therefore setting a constant predetermined value to compare to is NOT sufficient.

I decided I needed to implement some sort of feedback loop to calculate the percentage offset between readings. The idea is to take ten readings and compare the eleventh reading to the average of the first ten. If the difference is greater than 25%, motion is detected.

The algorithm is written as follows:

  • initialize constants and variables
  • setup an array to store ten readings from the sensor; loop through to fill array prior to starting the main run-loop.
  • run loop:
  1. take reading from sensor
  2. calculate average of elements in array
  3. check for motion by comparing the average to the current reading from the sensor
  4. if difference is greater than 25%, motion is detected -> send command to take picture
  5. otherwise, shift the array by deleting the oldest sensor data, and add the newest reading to the array

This loop (1-5) is run when the vehicle is stopped. I have successfully added it to my Arduino code.

More Updates!

Today was another extremely productive day in the push towards finishing my project.

As time goes on, I am continually searching for useful additions to my project. Yesterday I came across something that is used widely within networked security or IP cameras. The idea is to be able to instantly send an image to an email address when motion is detected. I previously had motion detection working between the iPhone and external IR sensor that triggered the iPhone to take a picture when motion was detected.

Now I just needed to determine how to email the photo.

Yesterday afternoon I came across the skpsmtpmessage library created by Skorpiostech, Inc.. This library uses the Simple Mail Transfer Protocol (SMTP) to send an email directly from the iPhone. I needed this library because I wanted to be able to send the email in the background, without the user knowing (This technique is not currently allowed in the Apple App store due to the fact that it can be used maliciously). After some research and testing, I added the library to my application and set it up to create an email and attach an image. An email is now automatically sent to my account when motion is detected. The user also has the option to have a picture emailed by entering a command through the Java program.

Today I also spent some time updating my code as follows:

Java Program:

– The program will now quit when exiting the window (was having problems earlier)

– Messages can’t be sent over the network stream until the iPhone is connected (this ensures the application doesn’t shut down if the user attempts to press a button early)

– Updated display and code to show status messages received from the iPhone

iOS Application:

– As discussed above, I can now email images directly from my phone by motion detection or command

– Updated code to send status messages to the Java program

– Cleaned the code up a bit, rewrote some functions to behave differently

– Added error control (pictures can only be taken if camera is opened, etc…)

Today I learned a lot about SMTP as well as how images are encoded to be sent over the network. This information will help me a lot to finish the video streaming aspect of this project. If I can email an image, I can certainly send a stream of images to the host computer. Hopefully that’s what will be coming next!

Progress Updates

Today and yesterday were both big days in terms of progress.

Yesterday afternoon my order for the IR sensors was delivered and I programmed my Arduino to allow for the analog input of the sensor. After some tests, I got the sensor to react when ArduiPhone gets close to an obstacle. Currently it is programmed to reverse for 2 seconds and then stop and wait for more incoming commands. I am now considering the possibility of using the sensors to determine how to navigate around an object, until a stop command is received.

Today, through the use of jjob’s web-blog tutorial on using the iPhone camera to capture images I have added camera support to my application. I have added an extra button to the application interface that allows the user to open the camera. Once the camera is turned on, there are two overlaying buttons on screen, one to go back to the home screen and one to capture an image. The two images below show the updated UI.

 

Finally, I added support to allow for incoming commands (from the Java program) to start and stop the camera as well as capture an image and save it to the iPhone itself. Though this isn’t video streaming yet, its a huge step in the right direction. Now that I can capture an image, I should be able to focus on sending that image over the network to the Java program. After that I will then implement video streaming in full by sending multiple images after each other.

Some Possible Arduino Additions

Today, after meeting with my supervisor, I’ve decided to think about the possibility of adding some external devices to the ArduiPhone due to the capabilities of the Arduino. The Arduino has a total of 14 digital output pins, and 6 analog input pins. Currently, the motor shield is only using 4 of the digital output pins and the serial communication uses 2. That leaves me with all of the analog input pins and the 8 remaining digital output pins. I was thinking of using the analog input pins to attach some sort of external proximity sensor that would allow the device to stop before hitting a wall. Also, I have a 16×2 LCD screen that I am considering using as a secondary display on the ArduiPhone.

This morning I got the LCD up and running with a separate Arduino board. I configured it so it should be able to work along side the motor shield while still allowing for serial communication. The picture below shows the working setup.

I am going to look into proximity sensors that will work with the Arduino and try to get those working shortly.

Also, I am continuing my research on video streaming from the iPhone to the computer. I will post updates as soon as I figure it all out.

Interim Report and Presentation

I have just completed writing my interim report and presentation. It’s basically a summary of the work I have completed so far, and what I have left to do next semester.

Check out the following links if you’re interested.

Interim Report (PDF)

Interim Presentation (PPT)

 

Happy Holidays!

Follow

Get every new post delivered to your Inbox.