ARDroneForP5: Processing, You can control AR.Drone frome Processing -

Shigeo Yoshida (Japanese ver:2011/02/26, English ver:2011/03/29, Last update 2011/06/21)

AR.Drone library for Processing, ARDroneForP5
Application
API
Thanks

ARDroneForP5: AR.Drone library for Processing

Working Environment

Any environment that Processing works

Mac(OSX)
Windows
Linux

AR.Drone firmware version

1.6.6 or older version. (2011/06/21)

To downgrade the firmware of your AR.Drone,you can use this iPhone application. Drone Control

Download

Please download from the following link.

　https://github.com/shigeodayo/ARDroneForP5

How to connect to AR.Drone from PC

Select a WiFi device named "ardrone_******" on your PC, after you connected the battery to the AR.Drone.

e.g. : MacOSX

How to run

Extract the download file, and move it to a suitable folder (e.g. Desktop). There are some sketch file(Processing source code) under the "sketches" folder. When you open the file ARDroneForP5_Sample with Processing, you will get the following window.

Open sketch file with Processing

Run(display)

Run	fuwa
fuwa	Operate like this

In this sample sketch, keyboard operations are as follows. The window shows, altitude, pitch, roll, yaw angle, x y velocity speed, battery power of AR.Drone and the camera view from AR.Drone perspective.

shift key	Takeoff
ctrl key	Landing
up arrow key	Move forward
down arrow key	Move backward
right arrow key	Move right
left arrow key	Move left
S key	Hovering
U key	Move up
D key	Move down
L key	Turn left(CCW)
R key	Turn right(CW)
1 key	Horizontal camera view
2 key	Horizontal camera view(main) + Vertical camera view(sub)
3 key	Vertical camera view
4 key	Vertical camera view(main) + Horizontal camera view(sub)
5 key	Toggle camera

Sample program

	import com.shigeodayo.ardrone.manager.*;
	import com.shigeodayo.ardrone.navdata.*;
	import com.shigeodayo.ardrone.utils.*;
	import com.shigeodayo.ardrone.processing.*;
	import com.shigeodayo.ardrone.command.*;
	import com.shigeodayo.ardrone.*;

	ARDroneForP5 ardrone;
	
	void setup(){
	  size(320, 240);
	  
	  ardrone = new ARDroneForP5("192.168.1.1");
	  // Connect to AR.Drone
	  ardrone.connect();
	  // For sensor data streaming
	  ardrone.connectNav();
	  // For video data streaming
	  ardrone.connectVideo();
	  // Start controlling AR.Drone and getting sensor/video data
	  ardrone.start();
	}
	
	void draw(){
	  background(204);  
	
	  // Get PImage frome AR.Drone
	  PImage img=ardrone.getVideoImage(false);
	  if (img==null)
	    return;
	  image(img, 0, 0);
	
	  // Standard output AR.Drone sensor information
	  // ardrone.printARDroneInfo();
	  // Get each sensor information
	  float pitch=ardrone.getPitch();
	  float roll=ardrone.getRoll();
	  float yaw=ardrone.getYaw();
	  float altitude=ardrone.getAltitude();
	  float[] velocity=ardrone.getVelocity();
	  int battery=ardrone.getBatteryPercentage();
	  
	  String attitude="pitch:"+pitch+"\nroll:"+roll+"\nyaw:"+yaw+"\naltitude:"+altitude;
	  text(attitude, 20, 85);
	  String vel="vx:"+velocity[0]+"\nvy:"+velocity[1];
	  text(vel, 20, 140);
	  String bat="battery:"+battery+" %";
	  text(bat, 20, 170);
	}
	
	// Key binding
	void keyPressed(){
	  if(key==CODED){
	    if(keyCode==UP){
	      ardrone.forward(); // Move forward
	    } else if (keyCode==DOWN){
	      ardrone.backward(); // Move backward
	    } else if (keyCode==LEFT){
	      ardrone.goLeft(); // Move left
	    } else if (keyCode==RIGHT){
	      ardrone.goRight(); // Move right
	    } else if (keyCode==SHIFT){
	      ardrone.takeOff(); // Takeoff
	    } else if (keyCode==CONTROL){
	      ardrone.landing(); // Landing
	    }
	  }else{
	    if (key=='s'){
	      ardrone.stop(); // Hovering
	    } else if (key=='r'){
	     ardrone.spinRight();  // Turn right
	    } else if (key=='l'){
	      ardrone.spinLeft(); // Turn left
	    } else if (key=='u'){
	      ardrone.up(); // Move up
	    } else if (key=='d'){
	      ardrone.down(); // Move down
	    } else if (key=='1'){
	      ardrone.setHorizontalCamera(); // Set horizontal camera view
	    } else if (key=='2'){
	      ardrone.setHorizontalCameraWithVertical(); // Set horizontal camera view with vertical camera view
	    } else if (key=='3'){
	      ardrone.setHorizontalCameraStreaming(); // Set horizontal camera view
	    } else if (key=='4'){
	      ardrone.setVerticalCameraWithHorizontal(); // Set vertical camera view with horizontal camera view
	    } else if (key=='5'){
	      ardrone.toggleCamera(); // Toggle camera
	    } 
	  }
	}

New code

Launch Processing, and select "Sketch->Add file...". Add ARDroneForP5/lib/ARDroneLForP5.jar.

Then, "ARDroneForP5.jar" will be added to "code" directory, automatically.

Now, you can control AR.Drone from Processing!

Notice

Hovering is not working properly.
Problem of inertia of AR.Drone? May need to implement a feedback control?

Application

Sample applications with other Processing libraries. You can easily make these programs with my AR.Drone library.

with ARToolKit

With NyAR4psg (Prpcessing ARToolKit library). Detecting AR marker and overlaying a cube.

Horizontal camera view	Vertical camera view
Detect AR marker	flying situation

To make those programs, download "NyAR4psg" and add File "NyAR2.jar" and "NyARToolkit.jar" to your Processing sketch. Then, put camera_para.dat and patt.hiro into "data" directory.

with Kinect

With Kinect library for Processing.

AR.Drone front camera view. AR.Drone moves according to the hand movement (round dot on the screen).

Distance image of kinect

You can control AR.Drone like this.

To make this program, download Kinect library for Processing and add File "openkinect.jar" and "libKinect.jnilib" to your sketch. And, put KinectTracker.pde (this sketch is in sample program "AveragePointTracking") into your working directory.

Sample sketches

In this library, there are 4 examples.

ARDroneForP5_Sample.pde
ARDroneForP5_SimpleLite.pde (combination with NyAR4psg, AR)
ARDroneForP5_Picking.pde (combination with NyAR4psg and Picking, AR and picking virtual object)
ARDroneForP5_PPopupMenu.pde (combination with NyAR4psg, Picking and PPopupMenu, AR+picking+popupmenu)

You can download libraries used in examples from following links.

API

//constructor
ARDroneForP5()
-You have to call this first.

ARDroneForP5(String)
-You have to call this first with specific IP address.

//public method
boolean connect()
-Call this method to connect and control AR.Drone.
return value
true:connect succeeded
false:connect failure

boolean connectVideo()
-Call this method to get video streaming from AR.Drone.
return value
true:connect succeeded
false:connect failure

boolean void connectNav()
-Call this method to get sensor information from AR.Drone.
return value
true:connect succeeded
false:connect failure

float getPitch()
-Return pitch angle of AR.Drone. You have to call connectNav before calling this method.

float getRoll()
-Return roll angle of AR.Drone. You have to call connectNav before calling this method.

float getYaw()
-Return yaw angle of AR.Drone. You have to call connectNav before calling this method.

float getAltitude()
-Return altitude of AR.Drone. You have to call connectNav before calling this method.

float[] getVelocity()
-Return velocity(0:vx, 1:vy) of AR.Drone. You have to call connectNav before calling this method.

int getBatteryPercentage()
-Return battery percentage of AR.Drone. 0-100%.

void printARDroneInfo()
-Standard out AR.Drone sensor information(attitude, pitch, roll, yaw, battery percentage and velocity).

PImage getvideoImage(boolean)
-Return AR.Drone view image as PImage(Processing).
if args is true, automatically resizeing the image (320x240)

void setHorizontalCamera()
-Set AR.Drone camera view as horizontal camera view (320x240).

void setVerticalCamera()
-Set AR.Drone camera view as vertical camera view (176x144).

void setHorizontalCameraWithVertical()
-Set AR.Drone camera view as large horizontal camera view and small vertical camera view (320x240).

void setVerticalCameraWithHorizontal()
-Set AR.Drone camera view as large vertical camera view and small horizontal camera view (176x144).

void toggleCamera()
-Toggle camera.

void disconnect()
-Disconnect AR.Drone.

void start()
-Start controlling AR.Drone. You have to call connect and connectNav(if you needed) and connectVideo(if you needed) before calling this method.

void landing()
-Let AR.Drone landing.

void takeOff()
-Let AR.Drone takeoff.

void backward()
-Let AR.Drone move backward.

void backward(int)
-Let AR.Drone move backward with specific speed.

void forward()
-Let AR.Drone move forward.

void forawd(int)
-Let AR.Drone move forward with specific speed.

void spinLeft()
-Let AR.Drone spin left (CCW).

void spinLeft(int)
-Let AR.Drone spin left (CCW) with specific speed.

void spinRight()
-Let AR.Drone spin Right (CW).

void spinRight(int)
-Let AR.Drone spin Right (CW) with specific speed.

void stop()
-Let AR.Drone hover (stop).

int getSpeed()
-Return current AR.Drone speed (0-100%).

void setSpeed(int)
-Set AR.Drone speed. (0-100%)

void down()
-Let AR.Drone down.

void down(int)
-Let AR.Drone down with specific speed.

void up()
-Let AR.Drone up.

void up(int)
-Let AR.Drone up with specific speed.

void goLeft()
-Let AR.Drone go left.

void goLeft(int)
-Let AR.Drone go left with specific speed.

void goRight()
-Let AR.Drone go right.

void goRight(int)
-Let AR.Drone go right with specific speed.

void reset()
-Let AR.Drone take and emergency stop.

void setMaxAltitude(int)
-Altitude limitaion of AR.Drone.

void setMinAltitude(int)
-Altitude limitation of AR.Drone.

Thanks

Hashimoto Sunao, JST ERATO Igarashi Design UI Project
Hirose Tanikawa Laboratory, The University of Tokyo
Munehiko Sato, my senior at laboratory, helped us to translate from Japanese to English.
ARDrone API forum
MAPGPS
daniel schmidt and Ian Hartney
Cliff Biffle

Profile

Shigeo Yoshida, shigeo.yoshida.shigeo [at] gmail.com
4th-year student, The University of Tokyo (2011/4/10 currently)
Researcher Assistant, JST ERATO Igarashi Design UI Project
Hirose Tanikawa Laboratory

Contents