Dorkbot CPH

shiftcontrol had the honor of starting the show at the recent "dorkbot" event here in copenhagen. Arranged by the amazing CIID at kunsthallen nikolaj, about 50 people experienced our project for last years Public Service Festival and a small demo using the iPhone to control Unity. CIIDs Oren Horev and Vinay Venkatraman also presented - more about the event at the CIID and dorkbotCPH sites.

Now, the iPhone -> Unity experiment attracted some attention (partly due to the video we put up at youtube), so we thought it would make sense to describe how it was done a bit more in detail:

Recipe

Jailbreaked iPhone (mine is running 1.1.2) so we can run third-party applications. Follow instructions at http://iphone.unlock.no/ (among others) for how to do this.

Install the akaRemote application by Masayuki Akamatsu on the iPhone. akaRemote provides you with a selection of typical control surfaces (sliders, XYpad, Trigger buttons etc.) and transmits the control data over WiFi using the OSC (Open Sound Control) network protocol.

Install FLOSC by Ben Chun on your computer. This is a wonderful (and old) java based gateway server intended to translate OSC Data arriving on a UDP connection into XML. Start the flosc server with

$ cd flosc-0.3.1/
$ java Gateway 6000 6001
Attempting to start OSC / Flash Gateway server
OscServer created...
TcpServer created...

Connect akaRemote (running on the iPhone) to %your_ip% port 6000 Check that data is coming through by

$ telnet localhost 6001

If the above works, you will see the Terminal fill up with XML data like this:

<OSCPACKET ADDRESS="192.168.0.5" PORT="49170" TIME="0">
  <MESSAGE NAME="/event">
    <ARGUMENT TYPE="i" VALUE="0" />
    <ARGUMENT TYPE="i" VALUE="2" />
    <ARGUMENT TYPE="i" VALUE="10" />
    <ARGUMENT TYPE="i" VALUE="3" />
    <ARGUMENT TYPE="i" VALUE="2" />
    <ARGUMENT TYPE="f" VALUE="0.4653846" />
    <ARGUMENT TYPE="f" VALUE="0.5884615" />
  </MESSAGE>
</OSCPACKET>

The akaRemote app has an option that broadcasts the iPhones accelerometer values too...

Now, lets do something with this data in Unity. Create a simple socket client in Unity using C# (Mono) a good starting point is here and here. And, since we're getting the data in as strings - construct a string parser that re-maps the incoming values to a data structure we can use.

float[]  res = new float[8];
string[] split = Regex.Split(str, "VALUE=");
int i = 0;
foreach (string s in split) {
  if( !s.StartsWith("<") ){
    char[] c = Regex.Split(s, " /")[0].ToCharArray();
    res[i] = System.Convert.ToSingle( new String(c, 1, c.Length-2) );
    i++;
  }
}
float f = 1.57f * Mathf.Rad2Deg;
nx = (res[5] * f) * 2;
ny = (res[6] * f) * 2;

Now, creating small scripts that hook this up to say the rotation of Plane is trivial.

Note: This approach is clearly hackish and inefficient (but hey, it works!) If you find a good way of eliminating the flosc gateway, data translation and string parsing - I hope you'll let me know :-)