Head tracking for Spring is here! Try it :)

[Beherith]

Real time head tracking for Spring is here!

The application uses a webcam which is placed above the monitor to recognize the players head. It then uses this information to track the camera in relation to the players head.
As of yet, the recognition is done by a Haar classifier, and it uses the OpenCV framework.
It adds the recognition information to a file, headdata.txt, in your Headtracker directory, which is then read by a lua script in real time. The widget sets the camera to the corresponding position.

Moving your head left and right moves the camera left and right (just like scrolling in OTA mode), and up and down motion is mapped to tilting the camera (ctrl+scrollwheel in OTA view)

It will display a window, titled "result", this window can be closed if desired, it just shows the output of the Haar classification.

The app adds a great deal of realism to Spring.

Most webcams frame rate drops to about 5 fps when used without any ambient lighting, the ideal frame rate, which is about 10 minimum, needs stronger ambient lighting.
For maximum effect, try the application during daylight hours, as ambient lighting is higher then than what most people can achieve with electric lights. High frame rates for the camera during this time will make the effect look really spectacular!

Its quite CPU intensive as of now, but I don't see this to be a problem on dual core machines. Most dual core pc's should be able to handle it without a drop in Spring frame rate. For my computer, there is absolutely no drop in Spring preformance.

The Lua widget has a gui, but can be hidden by pressing show. The camera positioning can be stopped by pressing stop. The window is draggable, so as not to be in the way.

Enjoy the full three dimensions of Spring!

System requirements:
- A webcam, just about any type will do.
- A dual core processor, preferably better than 2ghz, so as not to impact Spring performance.

Download links:

HeadTracker v0.4 (2.2MB)

Installation instructions:
1. Download the package.
2. Unzip the contents of the zip file to you Spring directiory. That should create the HeadTracker dir in your spring folder.
3. From your headtrack directory, copy the file headtrak.lua to spring/LuaUI/Widgets.
4. Plug in your webcam, and place it on top of your monitor, facing straight at you. Start Headtracker.exe in the Headtracker directory.
5. Start Spring and enable the Hedtrak widget by pressing F11 and selecting it from the list.



Troubleshooting:
Headtrak doesn't recognize my webcam:
Close all applications that may be using it, then unplug and plug in the device again. If this doesnt work, try updating the drivers for it.
The widget displays the message: "NO FILE"
The widget can't find headdata.txt. Make sure you unzipped into the correct folder, and that the program is actually recognizing your face
The tracking works, but it updates very slowly/moves too jerkily:
Try using a bit more ambient lighting, eg, turn on the lights in your room or use it during daylight hours. The widget displays the current camera frame rate on its GUI. For best results, frame rate should be above 10.
As of yet, the jerky movement is also caused by Spring's SetCameraState function, which uses an exponential decay in moving the camera.
Spring is unable to load widget because of missing spring.h.lua or opengl.h.lua files:
Copy the corresponding files from headtracker to Spring/LuaUI/Headers.

Contact:
Here, or in the lobby: [teh]Beherith

Tell me what you think it, any suggestions or opinions are welcome!

[Masure]

Very interesting. I'm not sure it would be useful to me against a standard mouse camera control but I congratulate the devs for this kind of things.

Therefore i'll try this week end.

[MelTraX]

Works, though I only get 3fps atm..

You don't need those header files (spring.h.lua, opengl.h.lua) though.. Just delete them and the includes in the widget..

And since I just looked into your file:

Code: Select all

if hide==true then
  hide=false
else
  hide=true
end

is the same as

Code: Select all

hide = not hide

[Nemo]

Cool stuff. I had 8-20 FPS, and it worked pretty well and smoothly (after deleting those includes). My only complaints are 1) that it couldn't let me move more than a tiny bit in any direction, and 2) that after a little bit the computer I was using overheated and locked up. But hey, that's what school computer labs are for.

[Beherith]

Got a few people saying left/right should be switchable, and that movement speed should be user settable.
Will add these features on tomorrows release :)
Also removing header references and tidying up code.

[Forboding Angel]

Is there any chance that you would be willing to set this up using a wiimote as well?

Many people including myself who don't have a wii would probably gladly get a wiimote just to have this ability. Hell I'm going to buy a webcam tomorrow just so I can check this out.

Anyway, Good work!

[SwiftSpear]

The wiimote runs though the wii IIRC...

[Peet]

The wiimote runs though the wii IIRC...

It uses a bluetooth connection; it can connect to any computer with a bluetooth reciever.

[Beherith]

Is there any chance that you would be willing to set this up using a wiimote as well?

Many people including myself who don't have a wii would probably gladly get a wiimote just to have this ability. Hell I'm going to buy a webcam tomorrow just so I can check this out.

Anyway, Good work!

Unfortunately i dont have a wii either, but the demo code should be easy to port, cause the widget part only wants an input file of very simple format.

[Forboding Angel]

Me being the non-coder that I am... Would anyone be interested in picking this up?

The reason I like the idea of a wiimote thingy, is the fact that it is much more precise (and won't mess up if someone behind you walks to the ------- :).

Additionally, I would really like to her some reviews and reports of how this works and whatnot. I am at work (I work nights) for the next 10 hours, so atm all I can do is drool :)

[danuker]

Yay! It's working!

[Masure]

I gave it a look. Good job for this cam widget.

I suggest a different moving system. At present it works with absolute moves. It should be more useful to have a relative movement just like if your head was an analog pad.

Define a big zero aera to avoid moving for a breath.
When you go out of this zone, the cam has a speed relative to the distance from the zero aera and your head.
You also can define a command to redefine the center ingame.

I think this could improve the whole thing.

[Dragon45]

Could someone put up a youtube video of this working? Video their head and the subsequent motion fo the screen, i mean.

[Relative]

Gah, now I have to boot up windows....

[SinbadEV]

Thought Number 1: Awesome, can't wait to try it
Thought Number 2: Man I wish I could find my blue tooth adapter so I could try to do it with the Wiimote

[LordMatt]

Any chance of a Linux version someday?

[BrainDamage]

Any chance of a Linux version someday?

looking at it, i obtained the sources from beherit, i'm trying to solve some compiling error & platform diversity, then it should work for linux as well

[LordMatt]

Sweet! :D

[jcnossen]

Headtracker.exe lets me select my webcam, and then just does nothing (gives me an empty console window). No headdata.txt is written in my headtracker directory

EDIT: Nevermind, windows explorer was stealing control of my camera

[jcnossen]

I occasionally see negative detection times in the console window.. That would be a bug right?