Head tracking for Spring

[Beherith]

I am working on a head tracking application that uses any webcam, which is placed above the monitor to recognize the players head. It then uses this information to track the camera in relation to the players head.
As of yet, the recognition is done by a Haar classifier, and it uses the OpenCV framework.
It adds the recognition information to a file, which is then read by a lua script in real time. The widget sets the camera to the corresponding position.
Moving your head left and right moves the camera left and right (just like scrolling in OTA mode), and up and down motion is mapped to tilting the camera (ctrl+scrollwheel in OTA view)

The app adds a great deal of realism to spring, but so far i have quite a few issues with it:
The Haar classification isn't too accurate, so that still needs a lot of tweaking.
Most webcams frame rate drops to about 5 fps when used with ambient lighting, the ideal frame rate, which is about 15 minimum, needs stronger ambient lighting.
Its quite CPU intensive, but I don't see this to be a problem on dual core machines. Any dual core pc should be able to handle it without a drop in spring frame rate.

Though the biggest issue so far is with Lua's Spring.SetCameraState function. It uses an exponential decay when moving the camera to a position. I would need a linear motion to make the movement less jerky.

So far there is only support for windows, as I have no experience with other platforms.
I'm wondering wether the community would like to try this app? If so, I can probably upload it.
Is there any interest in adding this feature to spring?

Thanks for your attention,
Beherith

[aegis]

would go perfectly with the 3d glasses i just made

[DemO]

http://uk.youtube.com/watch?v=Jd3-eiid-Uw

[Beherith]

http://uk.youtube.com/watch?v=Jd3-eiid-Uw

Yes, it was inspired by that video With the difference, that almost everyone has a webcam, its much cheaper than a wiimote, and I dont own a wii

[AF]

The logitech software does this too for the 3D avatar heads, doesnt seem to give too much of a performance hit.

[BrainDamage]

what language do you plan to use to write it? will it be open source?

[Beherith]

what language do you plan to use to write it? will it be open source?

Its already written, and its C for the app, and lua naturally for the intrface widget. I'm not sure about the open source yet, but most likely it will be, as the community here strongly opposes anything thats not (remember springmark...)

[BrainDamage]

what language do you plan to use to write it? will it be open source?

Its already written, and its C for the app

if you don't use too weird libs, it should run under Linux without too many probs

if you need an hand to port it (and it doesn't take too long), gimme a shout in #springlobby in the lobby server or #springlobby@freenode irc channel

[HeavyLancer]

We need a driver/patch written so that Spring has TrackIR support. That would be awesome. You can zoom in/out by moving your head backwards and forwards, and easily move around the map with head movements.

[Beherith]

if you don't use too weird libs, it should run under Linux without too many probs

if you need an hand to port it (and it doesn't take too long), gimme a shout in #springlobby in the lobby server or #springlobby@freenode irc channel

Thanks for the offer, it only uses standard OpenCV libs, those all work on linux. When I shape it for a final release, I will contact you.

[LordMatt]

Well springmark was opposed so much mainly because jj was forcing it down people's throats.

[Beherith]

By the way, is there any way to make MS Visual Studio 2003 package all the nessecery dlls into, or next to the executable?

[Neddie]

I like this idea, keep us informed.

[Gota]

Demo that technology is dangerous.I quote:
"i just did some "head tracking" on my girlfriend"
People are already misusing it...

[DemO]

http://uk.youtube.com/watch?v=pxT_f_F32EU

PS3 does it too! Using the playstation eye.

[Beherith]

The time has come, release will be out shortly...