Mesh Deformation, Skeletal Animation, Bypass of UpSpring

[MidKnight]

Lots of people want these features, and they are inherently linked.

Let us discuss how to best achieve this!

I talked to Tobi, and he told me the hardest part of implementing a new model format would be getting it to interface with Spring's aiming/firing code.

[thesleepless]

i don't see that as a huge issue, it can still use pieces which would act exactly as they currently do, but they can be bones that influence mesh verts as well as ridgid mesh pieces like we currently have, i've been trying to come up with a new model format that could cater for all these things.
i'll post my ideas in awhile.

[Master-Athmos]

Yeah - I also don't understand the big problem about integrating bones with the current scripting system. As thesleepless said you'd control the joints via invisible pieces you move and turn just as you do it today...

I don't really see the point in disliking UpSpring though. It's a nice little programm that does what it should without too much hassle. It might have some room for improvements but those "shortcomings" in the end just are the because it hasn't been touched in years afaik. The only real shortcoming might be that it probably won't run on Linux. In the end the amount of guys actually being on Linux only is extremely small though...

Besides the animation level there imo also is a 2nd big need for advanced unit rendering - i.e. shaders, an 8bit transparency channel or even dynamic lights (which probably is more a map / unit rendering combo topic though). You already can use shaders and as I see it even the transparency thing via Lua but to my mind it's a very bad idea to actually do the rendering via your script language...

How to achieve all this? Well someone has do to the work which is where things get problematic in terms of who would be that someone...

[smoth]

I don't really see the point in disliking UpSpring though. It's a nice little program that does what it should without too much hassle.

this.

[Tobi]

i don't see that as a huge issue, it can still use pieces which would act exactly as they currently do, but they can be bones that influence mesh verts as well as ridgid mesh pieces like we currently have, i've been trying to come up with a new model format that could cater for all these things.
i'll post my ideas in awhile.

The issue is that you need animations parameterized to arbitrary angles that come as input from Spring.

With any existing animation format that's just based on sequences of joint movements it isn't clear to me how you'd integrate that.

I.e. how do you make a parameterized md5 animation that can move a soldier's weapon naturally to any heading/pitch?

(The only way I can think of is inverse kinematics, or some animation format that actually supports parameterized animations, but I don't know any such format besides some commercial thing in Pro/Engineer.)

Yeah - I also don't understand the big problem about integrating bones with the current scripting system. As thesleepless said you'd control the joints via invisible pieces you move and turn just as you do it today...

"Also"?

Personally I don't really see the advantage of just adding bones and not a proper animation format; the current piece hierarchy is pretty much already a variant on bone based animation (pieces are joints and the bones are implicitly defined by the relations between pieces), possibly with the exception that you can not have vertices linked to multiple bones (i.e. interpolated) and that the 'skeleton' is always a tree (not sure any bone based things support (arbitrary) graphs though...).

As I said, I think the hard part is parameterizing the animations you make in <insert favorite modeling program> so your soldier/bot/whatever can move his weapons in any allowed direction.

I heard Argh suggest in some thread tho that he didn't really mind if he just had, say, 8 orientations animated, and then bullets coming in the wrong direction out of the gun. That may be a workaround for this problem.

I don't really see the point in disliking UpSpring though. It's a nice little programm that does what it should without too much hassle. It might have some room for improvements but those "shortcomings" in the end just are the because it hasn't been touched in years afaik. The only real shortcoming might be that it probably won't run on Linux. In the end the amount of guys actually being on Linux only is extremely small though...

It runs on Linux, as Kloot has said before already.

The major shortcoming AFAIK is that coordinate systems don't match with Spring (X axis flipped). But as I don't really use it I'm sure actual content developers can point out more things.


EDIT: and another hard part is of course that whatever chain of calculation leads to the firing point Spring should use should run as synchronized code. That makes it a bit dangerous to simply plug in some external library without looking at it's source.

[Master-Athmos]

Why do you think you'd need more than just doing the mesh, creating a skeleton you apply to it and then control the joint orientation and movement via unit pieces? With like the Blender BOS export script we have you'd even be able to do pretty much the same as you'd do in your favorite 3D software suite...

Mixing animations pretty much can be done already thanks to the hierarchy system we currently have anyway for animations. To my mind we need no fancy stuff like e.g. doing a running and a walking animation an then have a slider to mix the two of them together. I mean we're not doing a shooter or something like that which might need that (with RTS model counts we probably wouldn't get any good performance when using this anyway). We imo also don't need like a setting for to what angles a joint can turn at max - again as we're doing no shooter we don't need stuff like that to do some fancy ragdoll stuff or people looking at the player and turning when the head reaches its max rotation...

We just need a skeleton for animation with in the best case multiple joints per vertex (as otherwise we'd be back at introducing 1998 tech as Half-Life probably was the first title taking its step leaving simple vertex animations) or in other words weighed vertices so you even can do things like pulsating stuff (like the heart in The Cursed) without the need of doing a pure vertex animation...

[Tobi]

Why do you think you'd need more than just doing the mesh, creating a skeleton you apply to it and then control the joint orientation and movement via unit pieces?

Uhm because that's exactly what we have now (except possibly linking vertices to multiple joints and interpolating them), and everyone is asking for something better?

EDIT: maybe point out the exact differences you see between them?

I can see for now:

• Vertices linked to multiple joints (pieces), position interpolated.
• Faces defined with vertices which are assigned to different joints (pieces).

[jK]

People don't even create LODs, yeah not even wreckages of their models, not speaking of true normal mapping via highpoly -> lowpoly reducers, how should they take usage of something like vertex animation?

[Kloot]

Let us discuss how to best achieve this!

Read. The. Older. Threads. First.

[Master-Athmos]

Uhm because that's exactly what we have now (except possibly linking vertices to multiple joints and interpolating them), and everyone is asking for something better?

Well either I totally missed some news or we're talking at cross-purposes...

I'm talking of doing like let's say one single big mesh and then putting joints / bones into it for animation. Think of a snake crawling or a thing with some tentacles or like a cape or long hair blowing around. You cannot really cut things like that into pieces and animate them like we do it right now as this system only works for mechanical things but for nothing "organic"...

People don't even create LODs, yeah not even wreckages of their models, not speaking of true normal mapping via highpoly -> lowpoly reducers, how should they take usage of something like vertex animation?

First the thing about normalmapping is unfair as it really wasn't available for long and afaik Argh is the only one extensively using it at all. So moaning about people not using a feature that is extremely new and hardly used isn't a good argument...

When it comes to pure vertex animation - you're right that it'd be rarely useful once there are bones & stuff around. Still some things like pulsating stuff or even things like breathing still would be extremely difficult to do. I don't give it any high importance though - I just mentioned the weighing as it pretty much makes all of these things possible with the bones system...

[Tobi]

People don't even create LODs, yeah not even wreckages of their models, not speaking of true normal mapping via highpoly -> lowpoly reducers, how should they take usage of something like vertex animation?

I would guess (correct me if I'm wrong) that is all because it either:

• isn't implemented,
• isn't documented,
• or tool support is too crappy / non-existing.

Either way it's off topic as wrt animation this thread is about skeletal animation, and not vertex animation.

I'm talking of doing like let's say one single big mesh and then putting joints / bones into it for animation. Think of a snake crawling or a thing with some tentacles or like a cape or long hair blowing around. You cannot really cut things like that into pieces and animate them like we do it right now as this system only works for mechanical things but for nothing "organic"...

Yeah I understand.

However what I don't see (apart from the points above) is what you think are the differences between the two. (see also all the other threads Kloot linked to.)

A joint = a piece.
A bone is the (implicit) connection between two joints (pieces).

As far as I see the only fundamental difference is:

• Faces defined with vertices which are assigned to different joints (pieces).

Which is because (what jK already said in one of the other threads) pieces in Spring actually store their geometry.

(The current rendering implementation using a display list per piece would break down if one wanted to add the possibility to make multi-piece faces.)

Read. The. Older. Threads. First.

Any links to source code, or was it really not more then 'make some md5 animation player use Spring's OpenGL context', as you seem to suggest in these threads?

EDIT:
What about this problem btw?

Joints actually, the bones are implicit. But each joint only knows its parent joint (not vice versa), so the transformations cannot be applied the same way as in 3DO/S3O.

Could you elaborate a bit more? Or did you already solve this?

(It's unclear to me whether this is an unsolved issue and if so, why.)

[Kloot]

My work basically amounted to a player that ran the walking animation (of some Doom III models) in a loop if the unit that it was attached to moved, with the framerate scaled to the unit's speed.

The other thing you quoted is just related to a practical issue in that Spring applies forward kinematics only, so the skeletal tree would need to be constructed explicitly (bottom-up) to integrate it with the existing S3DModel design if you were to treat MD5 joints as pieces. But IMO, all of this is secondary to the problem of parameterizing the animations themselves, which would in effect require real-time (CPU-only due to sync constraints) IK.

[Master-Athmos]

@Tobi:
I guess I know now what you didn't get right - maybe I didn't explain myself very well...

My idea is that gameplay-wise everything stays as it is. The actual units consists of (hidden) pieces that are controlled via script just as things are right now. The actual look of the unit then just is like a "visual dummy" that gets put at the unit's spot. Maybe you remembet zpock's iirc normal map experiments where he put an alien model or whatever it was "on top" of the Core Commander. That example sort of goes into the direction I wanted to describe...

To stay with that example we would make the entire commander model invisible and make the mesh we want it to have fit to that invisible structure. You then would need all the fancy tech to apply a skeleton to that mesh and be able to deform it that way. So the unit rendering and actual animation / deformation runs totally independent from that actual gameplay relevant part (like aiming and so on) as this still gets done the "traditional" i.e. recent way of doing it. As you now don't want a static mesh wrapped around your invisible unit you need to apply every piece of your invisible model to a single bone / joint of the skeleton. You then catch all of the movements you do to your invisible unit and apply those to your skeleton which then deforms the mesh. In the end you sort of have the actual look of the unit being totally seperate from the game-relevant side of it. You'd keep every logic that the units need to work just the way it is now with the invisible unit and sort of let it play the "puppetmaster" for a simple visual representative for the unit which doesn't touch anything of the gameplay but is just some "fancy visual fx"...

Doing it this way imo would be a good method as you don't have to touch the problematic area of how to script the animations and even more how to do the aiming logic and things like that. "Remote controlling" the animation via the current hierarchy system might be a good option to solve this problem...

[smoth]

People don't even create LODs, yeah not even wreckages of their models, not speaking of true normal mapping via highpoly -> lowpoly reducers, how should they take usage of something like vertex animation?

thank you. they are very "cart before the horse"

[Argh]

I heard Argh suggest in some thread tho that he didn't really mind if he just had, say, 8 orientations animated, and then bullets coming in the wrong direction out of the gun. That may be a workaround for this problem.

Yes, I'd use it. And it would be fine. I "just" need the tech and probably a bit o' patience while I climb the learning curve. I have done a fair amount of IK animation stuff, so it'd mainly be about dealing with the export process and hooking it into some BOS / Lua.

As for normalmaps... I don't have any examples of using higher-poly meshes mainly because the only serious use for them is organic stuff, unless you're talking about distortion maps or FPS-levels of detail, the former's out of the ballpark for me on this hardware atm, the latter's not relevant to RTS design.

It's a tech issue, basically, and I confess that I simply don't have a solution atm. I've tried various hacks to get the required values from the vertex shader into the fragment shader, but I run into the "too many floats" errors every time it compiles. I've explained why, in considerable detail... and nobody seemed to be interested in seeing if there were other solutions, so that's where that stands atm, pretty much where it stood before I argued for depth-test support. I will take a final look at this when I do the final set of tweaks to the current shaders I'm going to be releasing when P.U.R.E. 1.3 is complete.

But mainly, I want to use organic modeling techniques, which I am conversant in but have had no use for in this engine. And that's exactly the stuff I can't make because of a lack of the animation support.

[Neddie]

People don't even create LODs, yeah not even wreckages of their models, not speaking of true normal mapping via highpoly -> lowpoly reducers, how should they take usage of something like vertex animation?

thank you. they are very "cart before the horse"

Except the people who are responsible developers, the people who the engine should support the most, such as yourself, do use some if not all of the options already here, and improved animation is something many - particularly those who work with organics - have desired for some time with the intent of using it as it appears in an acceptable form.

Making the models to take advantage of functionality without it being available is cart before the horse. The present situation is merely no horse, no cart.

[KDR_11k]

I.e. how do you make a parameterized md5 animation that can move a soldier's weapon naturally to any heading/pitch?

(The only way I can think of is inverse kinematics, or some animation format that actually supports parameterized animations, but I don't know any such format besides some commercial thing in Pro/Engineer.)

Games have been doing that since Half-Life, you just mark some bones as controller bones and rotate them. Usually you mark the whole spine as controllers for the aim and rotate each joint in it a bit to get a total twist that matches the desired angle.

[Tobi]

Hmm right, so basically some cheap-ass inverse kinematics ripoff, if I may call it that

[KDR_11k]

Naah, it's all forward kinematics and not even trying to be IK (Blender has something similar in its constraints functionality). I'm not really sure how IK even makes sense for aiming anyway since you only want a direction, not a specific position.

There's also the option to make a centered pose and four poses for the up/down/left/right extremes and interpolating but that's a lot of work and AFAIK not worth it.

[Tobi]

Hmm right, I assume you know the jargon better.

What I basically mean is it's also a set of equations (constraints) that can be solved (trivially, in this case) to give the orientations of the spine and all the joints on the path from spine to gun.

I assume you can still use IK for aiming, you just have 3 free variables (the position of the gun endpoint). Could force this on some sphere possibly, or minimize the amount of rotation that is needed to get the gun in a particular direction, without caring about the position. (i.e. so it finds the most "lazy" pose with gun in right direction.)

Either way, the forward kinematics thing with a way to select the joints + rotation axes sounds like a decent solution.