Using the iPhone as a gamer interface

Apple has finally released their new iTunes app store and everyone is racing to release the hot new game for the iPhone. Although I’ve been eagerly watching the results, I have yet to purchase any of these new games. As a matter of fact, I haven’t even upgraded to the 2.0 software yet. As far as I can tell, it’s going to take a few generations of bad games before we really figure out how to design games for the iPhone. To put it simply, you can’t just port Pac-Man to the iPhone and expect it to work. The iPhone is a powerful machine, but it’s lack of tactile buttons is going to make games (particularly platformers) difficult and the accelerometer is just not going to work with every type of game play.

Although developing games for the iPhone may be hindered by it’s lack of controls, the iPhone itself could surprisingly make for an awesome game controller itself. Imagine controlling the avatar on your computer over WiFi with your iPhone. Tilting it to move forwards or backwards, selecting weapons from your touchscreen, controlling your weapon via swipe motions. The iPhone could make a great game controller, hell it’s even got a built in rumble pad!

Below I’m going to explore the “iPhone as a controller” concept slightly further with two fictitious examples. I n the first I’ll describe taking a current first person shooter and creating an iPhone controller for it. After that I’m going to describe the possibility of merging the iPhone with a laptop to create a totally unique game play.

The iPhone as a controller concept

Let’s look at a traditional FPS like World Of Warcraft and see how we could enhance the gaming experience using an iPhone as a controller. The accelerometer would lend itself well to navigating in WOW’s 3D environment. Leaning forward would make your character walk, leaning would turn, etc. The iPhone’s screen could be utilized for various displays. Shown in the picture above is an inventory of spells that could be left open during a fight. Whenever an item is selected, a sound could come from the iPhones speaker. Whenever a player is damaged, the rumble pack will shake in their hands. Another display could display a topographical map with locations of nearby friends. In this screen, the player could drop pins to make it easier to find for later. Different finger motions could be used to trigger different actions. Swiping a finger from left to right on the iPhone’s screen could wield a weapon being held in your inventory. Swiping forward could jump, etc. All of the swipe motions, inventory displays, and maps could be customized by the player to however they feel.


Tank Buggy is a game I made up to demonstrate another type of game play using the iPhone. T.B. is a cooperative 2 player game in which one player navigates a fast moving tank (using a laptop or desktop computer), while another player mans the tank’s turret (using only an iPhone). The 2 players will be synced up by using a WiFi network. To encourage interactivity, the driver will have a radar screen that shows the location of targets and the gunner will have a map to navigate the course with. This will require communication and teamwork. For the driver, the goal will be to navigate the course quickly, while working with the gunner to help get good clean shots. For the gunner, the goal will be destroy any other tanks in the drivers path and help navigate the course. The gunner will have access to 3 weapons; long range, close range, and remote control missile (which will give the player a first person POV while heading towards it’s target).

So there you have it, the possibilities of the iPhone are not just limited to puzzle games or casual games, but can be adapted to all sorts of new game interfaces. The screenshots used  in the WOW photo are from This idea was heavily inspired by the awesome Remote app by Apple for wireless iTunes control. If you are a game developer and are interested in talking more about iPhone UI, leave a comment below or contact me directly if it’s for professional purposes. The controller ideas listed above are free for anyone to use, but if you do use any of them, I would appreciate a mention in the game credits and link to my site (  In case you are wondering, working on or planning on creating any iPhone games or controllers using any of these ideas because I’m too caught up with IgorLabs working on our new X-Box title. The goal of this post was just to give developers some new ideas (with examples) of new ways to use the iPhone with the way we interact with our computers. Again, if you have any constructive comments, please post them below.


18 responses to “Using the iPhone as a gamer interface

  1. jon Avatar

    Some interesting thoughts doc, i would like to see this happen.

    But since when is WoW an FPS?

  2. Doctor Popular Avatar
    Doctor Popular

    Jon, what is the POV reffered to in WOW? I’ll correct it.

  3. […] Francisco game theorist, crafter, musician, artist and yo-yo performer extraordinaire – has posted an article in which he envisions the potential applications of the iPhone as an alternate interface …. Rather than merely using the iPhone as a stand-alone gaming platform, Doc suggests using it as a […]

  4. brian Avatar

    Wow, that is pretty cool Doc.

  5. qDot Avatar

    The controller ideas listed above are free for anyone to use, but if you do use any of them, I would appreciate a mention in the game credits and link to my site (

    And Lionhead might want to some credit for gesturing (though Black and White was most likely not first, just the earliest I can remember right now, and Aquaria is getting a lot of attention and also uses it)… And Nintendo might want some credit for the gba usage in Crystal Chronicles… And Sega for the VMU in the dreamcast…

    Damn, gonna be a long credit list if we keep this up.

  6. doc Avatar

    “Damn, gonna be a long credit list if we keep this up.”

    Sure it would be a long list if we listed any game with a unique interface or any console with a new controller, but I don’t see what those have to do with my suggestions here.

    In terms of iPhone apps being used as an alternative user interface, I believe the list would be fairly small.

  7. […] came across a great post by Doctor Popular (via Laughing Squid) about Using the iPhone as a gamer interface.  Very cool stuff.  And it got me […]

  8. Mark Avatar

    Hey Doc, that’s cool! It would be interesting to “see” how it feels. I dabble with these sorts of interfaces – I once built a virtual reality system where I could fly around (analogous to walking/running) by tilting a motion sensor forward or backwards – it kind of felt like a virtual motorcycle throttle without the spring-loaded tension. A couple of problems I encountered:

    1. Tilting forward and back to go and stop is a bit harder without an opposing force (i.e., spring-loaded tension). With an isometric joystick you can just relax to stop, in a relatively unconscious way. With the motion sensor interface its relatively laborious to stop since you have to deliberately reverse course.

    2. If you want the forward-backward tilting to include backing up then stopping becomes even harder since its a position in between moving forward and going backwards. [Probably not hard for a yoyo champion]

    3. The setup caused my wrist to hurt after a while. But the setup was not exactly like the one you describe since I was holding the motion sensor in one hand.

    Your idea for gestures sounds very good – much better than touchscreen buttons which are typically used. I played a couple of games on the Nintendo DS that had buttons or gizmos on the touchscreen that you were supposed to use while looking at the other screen – they were difficult to use because lacking tangible shapes and haptic feedback you needed to look down to make sure your fingers were in the right place. Using muscle memory and gestures will probably work much better.

    Regarding player PoV terminology: I haven’t played WoW but I believe its a Third-Person view, where you can see your avatar’s body. If your browser has Java installed you can see a 3rd-person demo on a web page here:
    However, if you hide the avatar (uncheck “character” checkbox) then its essentially a First-Person view.

  9. Jon Avatar

    Yeah, its third person.

    But there is no shoostings going on so it is a first person view but not an FPS.

  10. Cory Avatar

    Any luck with this yet? I was searching for an iPhone-as-controller solution when your site popped up. I would love to be able to control a game on my Mac Pro with my iphone. I am a fan of driving games – if I could use the iphone with the same actions I use with Asphalt 4 with a Mac game, that would be great. An app that could adapt to ANY game using the accelerometer and touch interface would be even better.

    There is an app called QwasiPad that makes the iPhone into a multi-touch trackpad or keyboard to remotely interface with PC machines. This was a small step in a similar direction. I have nothing more to contribute because I know absolutely nothing about software. Just wanted to voice my support. Cheers

  11. David Johnston Avatar
    David Johnston

    It is an interesting gadget due to its nice features.

    Innovation in technology has great achievement.

  12. ffxi gil Avatar

    If the iPhone just had flash I'd be happy. I just want to be able to use sites that have flash on them, then it would be just about the perfect phone. 2.2 looks like it will not deliver on some long awaited additions. I can't say I really care about getting podcasts over the air, that's why I have iTunes open 24-7 to get the podcasts as soon as they are released and when I plug in my iPhone for it's daily charge, they get uploaded.

  13. DocPop Avatar

    At least with the release of Google's G1, Apple has really stepped up their game. Removing the NDAs from developers, creating dev forums, tethering apps to come, who knows what else.

  14. Dalam Avatar

    hah..thats amazing!

  15. williamjohn Avatar

    Hi,Very exciting to see the offers.These coupons helped me
    save a lot similar to

  16. Club Penguin Cheats Avatar

    I am a fan of driving games, if I could use the iphone with the same actions I use with Asphalt 4 with a Mac game, that would be great. An app that could adapt to ANY game using the accelerometer and touch interface would be even better.

  17. registrycleaner Avatar

    The iPhone is a powerful machine.

Leave a Reply

Your email address will not be published. Required fields are marked *