Lua unit scripting

[Tobi]

Since people are already finding the wiki page I've been working on by themselves, I hereby announce they exists. (Ok, I haven't been very good at not mentioning the link either )

It's still work in progress so not yet really polished or anything.

This are the pages:

http://springrts.com/wiki/Animation-LuaScripting
http://springrts.com/wiki/Animation-LuaCallins
http://springrts.com/wiki/Animation-LuaCallouts


I've also been working on porting some Spring: 1944 units to Lua unit scripts. This can serve as an example. I plan to port more later on, this is also an ongoing project. These unit (and any future progress) can be found in a mutator for the SVN version of Spring: 1944.


Note that there will be some breaking changes in the near future. In particular I've been looking at making the coordinate system more consistent with the global coordinate system (This will probably mean in next releases rotations around the z-axis will be right-handed instead of left-handed.) Also I may deprecate some silly call-ins that are pretty easy to replicate using other existing functionality.

If you have any questions/comments I'd like to hear them; this will also help me to know what I should focus on in the documentation.

[CarRepairer]

No conversion tool yet?

[Argh]

1. Are there any clear-cut performance advantages / disadvantages?

I can see the flexibility of keeping logic all in one language, so I'm wondering if there are any serious tradeoffs. But I'd happily trade BOS for Lua if there aren't any serious penalties, conversion looks like it's mainly going to be some grep and minor alterations.

2. I see that Killed() has a number of areas which are still WIP (explode, etc.). Is it possible that this will entail greater flexibility in this area in the future?

3. If the coordinate system is made to match, just keep in mind that that will make 1:1 quick conversions considerably harder. Perhaps make that a flag set on Create?

4. The equivalents to the shorthand [] and <> would probably be a good idea.

5. How do we do #includes? Just VFS.Include(ANIMATION_LOCATION .. 'standard_commands.lua',nil,VFSMODE)?

6. Can we send animation commands to Features with this method?

[imbaczek]

4. The equivalents to the shorthand [] and <> would probably be a good idea.

ideally, [] and <> won't even make sense. all speeds and angles should be given in their respective units depending on the function they're passed to. [] and <> were needed because BOS can't handle floats; lua, OTOH, can't really handle integers (everything is a double precision float).

note that i don't know how Tobi decided to implement all this.

[Argh]

True, true, and I'd certainly approve if "Move(unitID, Piece, Axis, Distance)" was just automatically converted into linear units with the proper multiplier.

Gotta remember though: UpSpring is our primary tool for posing everything, as it already uses BOS units of linear / rotational coordinates and measurement. This is unlikely to change.

That's why I'm fairly dubious about "correcting" the rotation system (unless UpSpring is also updated), and why a need for straightforward conversion of [1] to "1" is necessary.

The conversion issue is very straightforward- Move(unitID,myPiece,x_axis,1,1) can be the equivalent of move myPiece to x-axis [1] speed [1]; without a lot of pain.

Just thinking through the details here, before I spend some time getting my feet wet. Converting the average P.U.R.E. unit script will be fairly complicated at first, they aren't exactly simple tank scripts, and if there aren't some major advantages then it's hard to justify the time spent.

Another question: if a Unit has a Lua script, does it still run a BOS thread?

And if we want to terminate a Lua animation thread, but not kill the Unit, can we do that, without hosing the entire system? If not, that would be a good idea- put a way to do a break in, so that we can do operations during Create(), halt the thread and not touch its other elements, and only run an animation script through Lua Callins elsewhere.

That would be a major step forward, if I could quit running so much overhead "Units" that aren't actually doing anything unless they die. As most of you Dev people know, this would probably have a pretty big performance savings in World Builder, where "empty threads" for hundreds of objects and their callins still have a non-trivial CPU cost every frame. If I can exit the Lua animation code, and BOS no longer imposes any overhead (i.e., by using this I can "opt out" of BOS, finally) then I can see some very significant cost savings- Units without sight radii would perform almost as well as Features.

[smoth]

No conversion tool yet?

You are making me *FRUMPLE*

[Argh]

Yeah, that makes me lol, frankly. Use a grep parser, d00d. Seriously, that's the least of the issues... and it's a one-time cost only for a project, timewise...

Oh, and last question:

Which BOS constants do *not* have 1:1 equivalents in Lua? I.E., are the mathematical / operation constants, such as get PIECE_XZ, going to be replicated?

Kinda important question, actually. I don't use a lot of these, but not all of them are readily equivalent in Lua callouts atm, iirc.

[smoth]

Use a grep parser

I don't know if it is really that simple.

[Argh]

Should be, for most of the code, just looking at Tobi's example.

Dunno for sure yet, just based on quick comparison with his example script, though, it looks like most of it. We'll see, when it comes to loops and sub-loops. Using a tank script wasn't exactly a good way to evaluate how much pain walk-cycles are going to be to convert.

I guess I'll sit down with this in another couple of hours, start to see. I'm really curious about the performance advantages, if any.

[Tobi]

No conversion tool yet?

Not a 'run and forget about' one. Lurker has written some prototype thing, which, combined with some common sense, copy paste and search and replace can speed up manual conversion a bit.

I don't know whether he has any plans to continue on this.

1. Are there any clear-cut performance advantages / disadvantages?

I haven't done accurate realistic measurements yet (well I did some but I forgot the results <_<), but some preliminary testing suggested Lua in general is a bit faster then COB.

I can see the flexibility of keeping logic all in one language, so I'm wondering if there are any serious tradeoffs. But I'd happily trade BOS for Lua if there aren't any serious penalties, conversion looks like it's mainly going to be some grep and minor alterations.

That's what the Lua script lurker has worked on does.

The main advantages over BOS/COB are that Lua is a more mature language with more data types then just integers, and that you can use all normal (synced) Lua call-outs easily from within Lua unit scripts now (less need for ugly hacks to interface synced Lua code with unit scripts.)

2. I see that Killed() has a number of areas which are still WIP (explode, etc.). Is it possible that this will entail greater flexibility in this area in the future?

If I recall correctly the only thing missing is symbolic constants for the flags to explode.

3. If the coordinate system is made to match, just keep in mind that that will make 1:1 quick conversions considerably harder. Perhaps make that a flag set on Create?

No, making it configurable would just complicate code reuse between different scripts (and games).

The conversion isn't that much harder; you'll just have to invert the destination for moves along the X axis, and starting at the next release invert the destination angle/speed for turns/spins around the Z axis.

4. The equivalents to the shorthand [] and <> would probably be a good idea.

Special operators are a no go IMO since this requires quite some modification of the Lua parser.

It's easy however to define a function on the top of the script, or in the 'header' file that is automatically prepended to each script. (see gamedata/unit_script_header.lua in springcontent.sdz)

5. How do we do #includes? Just VFS.Include(ANIMATION_LOCATION .. 'standard_commands.lua',nil,VFSMODE)?

That is one way, however this means everytime a unit which uses this script is created the file has to be decompressed and parsed.

In the Spring: 1944 mutator I've included another gadget that simply runs all *.lua files in 'script/library' (default directory prone to change, because it conflicts with adding recursively scanning for unit scripts in 'scripts' [like what was added for unitDefs in 0.80])

I've started the convention these *.lua scripts put their unit script library function into the 'GG.UnitScript' table.

Since unit scripts have access to the GG table, they can then easily refer to this using e.g. GG.UnitScript.MyUtilityFunction()

6. Can we send animation commands to Features with this method?

Nope, that's unrelated to this.

4. The equivalents to the shorthand [] and <> would probably be a good idea.

ideally, [] and <> won't even make sense. all speeds and angles should be given in their respective units depending on the function they're passed to. [] and <> were needed because BOS can't handle floats; lua, OTOH, can't really handle integers (everything is a double precision float).

note that i don't know how Tobi decided to implement all this.

True, true, and I'd certainly approve if "Move(unitID, Piece, Axis, Distance)" was just automatically converted into linear units with the proper multiplier.

Move takes coordinates simply in elmos; Turn and Spin take angles in radians, angular speeds in radians per second, and angular acceleration in radians per second per frame. (Should I change this to radians per square second?)

Gotta remember though: UpSpring is our primary tool for posing everything, as it already uses BOS units of linear / rotational coordinates and measurement. This is unlikely to change.

That's why I'm fairly dubious about "correcting" the rotation system (unless UpSpring is also updated), and why a need for straightforward conversion of [1] to "1" is necessary.

Good point about UpSpring, however it doesn't convince me yet to keep using inconsistent coordinate systems forever. (It is inconsistent because now, in COB, rotations around the x-axis and z-axis are left-handed and rotations around the y-axis are right handed, and the x-axis is mirrored compared to Spring's world csys.)

Another question: if a Unit has a Lua script, does it still run a BOS thread?

No.

And if we want to terminate a Lua animation thread, but not kill the Unit, can we do that, without hosing the entire system? If not, that would be a good idea- put a way to do a break in, so that we can do operations during Create(), halt the thread and not touch its other elements, and only run an animation script through Lua Callins elsewhere.

Not sure what you are asking here. In the framework I implemented Signal and SetSignalMask which you can use to 'kill' an animation thread without killing the unit, just like in COB.

That would be a major step forward, if I could quit running so much overhead "Units" that aren't actually doing anything unless they die. As most of you Dev people know, this would probably have a pretty big performance savings in World Builder, where "empty threads" for hundreds of objects and their callins still have a non-trivial CPU cost every frame. If I can exit the Lua animation code, and BOS no longer imposes any overhead (i.e., by using this I can "opt out" of BOS, finally) then I can see some very significant cost savings- Units without sight radii would perform almost as well as Features.

If you don't need to do any animation, you don't have to run any animation threads. This is the same as in COB.

Similarly a sleeping or waiting thread doesn't use any CPU, only a bit of memory. This is also the same as in COB.

Also just like in COB functions that aren't implemented aren't called, and as such do not use any CPU. This also didn't change.

Which BOS constants do *not* have 1:1 equivalents in Lua? I.E., are the mathematical / operation constants, such as get PIECE_XZ, going to be replicated?

Spring.UnitScript.GetUnitValue and Spring.UnitScript.SetUnitValue (same as Spring.GetUnitCOBValue and Spring.SetUnitCOBValue) are available..

I also added a global COB table that contains the COB '#defines'. (e.g. COB.ACTIVATION, COB.BUILD_PERCENT_LEFT, etc.)

[FLOZi]

The tank script isn't ideal because it has some funky S44-specific stuff like HE/AP rounds.

[Argh]

Good point about UpSpring, however it doesn't convince me yet to keep using inconsistent coordinate systems forever.

Just keep in mind that if you want people to adopt this standard, then you really should provide either an update for UpSpring, or a animation viewing utility. Providing an updated UpSpring should be trivial, though- just change the signs and recompile it, and voila, we can use it for this.

It's easy however to define a function on the top of the script, or in the 'header' file that is automatically prepended to each script. (see gamedata/unit_script_header.lua in springcontent.sdz)

If that's how it's supposed to work, then 1 should be equal either 1 elmo, if linear or 1 degree if angular, not 1 radian, imo.

I mean... you're doing conversion in there anyhow, so what that conversion means is arbitrary, and we're all used to <1> == 1 degree, not 1 radian.

I am a bit concerned about the performance implications, though- you're effectively adding a multiplication step to every single move / turn / spin command, whereas Scriptor converts this when compiled. Moreover, doesn't this kinda assume that we want interpolation, when in fact most of the time we don't, for performance reasons, and instead want to move from one state to the other instantly?

That is one way, however this means everytime a unit which uses this script is created the file has to be decompressed and parsed.

Ah, I see. So we're creating a thread, basically, not a script that is treated as the basis of threads?

That could be useful.

Code: Select all

Not sure what you are asking here. In the framework I implemented Signal and SetSignalMask which you can use to 'kill' an animation thread without killing the unit, just like in COB.

What I'm getting at is that that in COB, certain things like Activate() callins seem to run, even if the Unit doesn't ever use them. Is that the case here? Or does the handler not operate that thread if there are no new instructions, saving that CPU cost?

Spring.UnitScript.GetUnitValue and Spring.UnitScript.SetUnitValue (same as Spring.GetUnitCOBValue and Spring.SetUnitCOBValue) are available..

Oops, I guess I wasn't clear.

There is no real equivalent (in Spring's Lua callouts atm) of get PIECE_XZ (which is why I picked that example). We can do Spring.GetUnitPiecePosDir, but it's not really the same (btw, is Spring.GetUnitPiecePosition still buggy as hell? It didn't work for flying things, etc. the last time I checked, but that was months ago).

So... I assume that the primary benefit is we get better precision and better raw math access (can finally do certain things that were, if not impossible, really fricking annoying with BOS)... but I'm a bit concerned about these special functions. If they don't exist, that presents a fairly major barrier to conversion for certain types of scripts.

[Tobi]

Good point about UpSpring, however it doesn't convince me yet to keep using inconsistent coordinate systems forever.

Just keep in mind that if you want people to adopt this standard, then you really should provide either an update for UpSpring, or a animation viewing utility. Providing an updated UpSpring should be trivial, though- just change the signs and recompile it, and voila, we can use it for this.

Upspring can play/import/export BOS/COB animations?

It's easy however to define a function on the top of the script, or in the 'header' file that is automatically prepended to each script. (see gamedata/unit_script_header.lua in springcontent.sdz)

If that's how it's supposed to work, then 1 should be equal either 1 elmo, if linear or 1 degree if angular, not 1 radian, imo.

I mean... you're doing conversion in there anyhow, so what that conversion means is arbitrary, and we're all used to <1> == 1 degree, not 1 radian.

No, internally radians are used (also it's the standard in math), so I chose the only one that doesn't need any conversion

I am a bit concerned about the performance implications, though- you're effectively adding a multiplication step to every single move / turn / spin command, whereas Scriptor converts this when compiled. Moreover, doesn't this kinda assume that we want interpolation, when in fact most of the time we don't, for performance reasons?

Why? I don't see how units of measurement have anything to do with wanting or not wanting interpolation. (Remember that in Lua all numbers are floats per definition.)

That is one way, however this means everytime a unit which uses this script is created the file has to be decompressed and parsed.

Ah, I see. So we're creating a thread, basically, not a script that is treated as the basis of threads?

That could be useful.

In Lua a file is also a function.

This function is remembered in the framework and called each time a unit is created. It sets up the call-ins, which are then registered to Spring by the framework.

Code: Select all

Not sure what you are asking here. In the framework I implemented Signal and SetSignalMask which you can use to 'kill' an animation thread without killing the unit, just like in COB.

What I'm getting at is that that in COB, certain things like Activate() callins seem to run, even if the Unit doesn't ever use them. Is that the case here? Or does the handler not operate that thread if there are no new instructions, saving that CPU cost?

That is mistake of the scripter then. If you don't use a call-in and don't want any CPU used by it, remove it.

(I don't know whether scriptor requires certain call-ins to be present, but Spring definitely doesn't require any.)

There is no real equivalent (in Spring's Lua callouts atm) of get PIECE_XZ (which is why I picked that example). We can do Spring.GetUnitPiecePosDir, but it's not really the same (btw, is Spring.GetUnitPiecePosition still buggy as hell? It didn't work for flying things, etc. the last time I checked, but that was months ago).

So... I assume that the primary benefit is we get better precision and better raw math access (can finally do certain things that were, if not impossible, really fricking annoying with BOS)... but I'm a bit concerned about these special functions. If they don't exist, that presents a fairly major barrier to conversion for certain types of scripts.

What about GetUnitValue(COB.PIECE_XZ, ..) ?

GetUnitValue and 'get blablah' in COB arrive at the same code in the engine.

I'm pretty sure one of the reasons GetUnitPiecePosDir/GetUnitPiecePosition is buggy as hell is the inconsistent coordinate systems that are being used.

A bigger benefit of Lua IMO is having tables and having direct access to the entire Lua API

[zwzsg]

I don't know whether scriptor requires certain call-ins to be present, but Spring definitely doesn't require any.

I once made made a TA unit with a empty BOS. (However empty COB didn't work.)

[Nemo]

A bigger benefit of Lua IMO is having tables and having direct access to the entire Lua API

This this this a thousand times this. Not to pull any kind of authority card, but as someone who has spent an absurd amount of time writing infantry animations...good lord I'm excited to move over to rapid development (ie script open in one window, spring open in another).

[zwzsg]

I am a bit concerned about the performance implications, though- you're effectively adding a multiplication step to every single move / turn / spin command, whereas Scriptor converts this when compiled.

Except it's the other way around, of course. But why bother discussing, you're Argh.

[Argh]

Upspring can play/import/export BOS/COB animations?

Technically, it can export BOS, yes (very poorly).

But mainly, it's used to do all of the poses for "tween" stages in Unit animation. Without it, or some other visualization tool, you're basically hamstringing artists. The more visually-friendly a content system is, the more artists can do with it. Spring's already very artist-unfriendly... why make it worse?

No, internally radians are used (also it's the standard in math), so I chose the only one that doesn't need any conversion

I know the engine uses that internally. I'm just saying that we're used to expressing stuff in degrees. It's simply a lot more natural to us.

Why? I don't see how units of measurement have anything to do with wanting or not wanting interpolation. (Remember that in Lua all numbers are floats per definition.)

I'm basically saying that what you're doing is the equivalent of SmothAnim=1, whereas the performance advantages of SmoothAnim=0 are pretty huge.

If you don't use a call-in and don't want any CPU used by it, remove it.

(I don't know whether scriptor requires certain call-ins to be present, but Spring definitely doesn't require any.)

I know that Create() and Killed() are required.

I'm also certain that there are COB loads associated with other callins whether or not the Unit is actually making use of them (i.e., it returns nil and continues). I'll go look at the source again, but the last time I checked, Spring calls COB threads in lots of places, and has no check whether that's appropriate or not.

Except it's the other way around, of course. But why bother discussing, you're Argh.

Eh? When you write:

move Piece to x-axis [1] speed [1];

It's converted to very linear hex in COB, since it all must be integers in the end. That [1] is 1 * 65536.000000.

So, for each turn step, a Unit with SmoothAnim=0 is doing straightforward integer math. Step --> step, distance or angle vs. speed. If distance = 65536, speed = 1024, then it will take 64 frames to complete the distance. But it's simple integer addition / subtraction for each step.

What I'm saying is that Tobi's code means that it requires... step * some new constant --> step. Get it? That's basically all of the benefits of SmoothAnim=0 out the window, so we might as well just have interpolation on and just use turn-now and sleep.

<shrugs> maybe I'm the only person who uses SmoothAnim=0, but I thought most of us were, because interpolation is so costly.

[Tobi]

Please Argh, do not introduce so much nonsense in this thread.

I've read like 100 times Spring's COB code while adding Lua unit script support, and not a single call-in is required by the COB code.

Of course if you want a proper weapon you need AimWeapon and stuff but Create and Killed really aren't necessary.


SmoothAnim=0 is not doing integer math. The only thing that is integer is COB, immediately in move/turn/spin command these integers are converted to floating point elmos/radians and are passed into exactly the same code that handles Spring.UnitScript.{Move,Turn,Spin}.

Actually, smoothAnim=1 is not even supported at the moment in Lua unit scripts (it's a hack for broken scripts, nothing more).

I know the engine uses that internally. I'm just saying that we're used to expressing stuff in degrees. It's simply a lot more natural to us.

but math.sin/cos/tan/atan/asin/atan all use radians, like the rest of the world... If you insist on degrees, you can always make wrappers like this though:

Code: Select all

local spTurn = Spring.UnitScript.Turn
function Spring.UnitScript.Turn(unitID, piece, axis, destination, speed)
  spTurn(unitID, piece, axis, destination*3.141592/180, (speed or 0)*3.141592/180)
end

[zwzsg]

I know that Create() and Killed() are required.

Last time I tried, Spring requried a Create(), yeah. Though that was years ago, and has probably been changed forth and back a ton since.

Eh? When you write:

move Piece to x-axis [1] speed [1];

The multiplication done by [ ] and < > are done at compile time. However, if you do: x=1*1*1*1*1*1*1*1*1*1; then Scriptor does write the whole formula in cob and recalculate it each time, while any sane compiler would see they're constant and optimise into x=1;


I don't know why you keep talking about SmoothAnim and interpolations, but then you're Argh, so I'm not suprised you would confuse a multiplication and an interpolation and start writing essays based on that.


As for keeping the [ ], I remind you that while TA scrips traditionnaly used a contant of linear 163840 for it, most Spring coder moved to having 65536, mostly because that's what UpSpring used, and it apparently wasn't such a big deal.

[Argh]

if you do: x=1*1*1*1*1*1*1*1*1*1; then Scriptor does write the whole formula in cob and recalculate it each time, while any sane compiler would see they're constant and optimise into x=1;

What does that have to do with anything? Talk about straw men...



Interpolation == averaging out Piece positions / rotations per render frame vs. final position on next game frame.

If piece position now == piece position linear + / - some integer, that's straightforward, integer math.

If piece position now == piece position linear * constant + / - some value * constant, that's fairly lengthy float math in Lua.

If you don't get why that has some fairly serious performance ramifications, that's not my problem.


As for why this relates to interpolation...

Each render frame, with SmoothAnim=0, the Piece should only be changing relative position / orientation of the Pieces and updating the display lists, with all the math that entails, on the next gameframe.

IOW, if you slow the gameframes down, you should eventually see jerky animation, where it's obvious that discrete steps are occurring. This is optimal, in terms of speed, and will actually raise effective framerate a lot on most hardware. IOW, it will be invisible, and when it's not invisible, it's actually helping the end-user's computer run the frames a lot faster.

What I see over here is that it's apparently not working like that, and it's more expensive per render frame as well. It should not update the Piece display lists until the next gameframe, imo.