Spring RTS Engine

http://www.darkstars.co.uk/2008/why-the-new-interface/

A blog post clearing up and generalizing the reasoning behind the new interface and the problems around the old interface.

nice!

(you are the pope, and i am the car with wings, right? :D )

No I'm the god on the cloud ;D

Now don't you FUD too. A lua AI doesn't need to run synced. Still going to be slow, though.

lurker wrote:Now don't you FUD too. A lua AI doesn't need to run synced. Still going to be slow, though.

Heh I didn't know that, I was going off of something trepan said on the forums when he implemented lua AIs, Ill issue a correction!

Can you still use the C++ language for AI making?

yes, also with the new interface.

I don't understand why the C interface can bring multicompiler support when the C++ couldn't.

well.. keep in mind, it is only for AIs, that you will be able to use different compilers. spring its self still has to be compiled with MinGW for windows and GCC for Unix.
As AIs are not synced, compiler specific floating point calculation optimizations are no problem. The only restriction that stopped AIs to be compiled with different compilers so far, was really C++.

(sorry if you knew everything up till here already, just wanted to make this clear)

i do not know about the details, but C++ compilers use different internal formats to store things (eg classes), and therefore, if we create and fill up a UnitDef in the engine (compiled with MinGW), and send it over to an AI compiled in VS, it would not work.
the engine sends a class with the binary name "_UnitDef", and the AI awaits a class "@UnitDef" eg (it is not like this, but i don't know how it is exactly, and i do not want to know )
if you want to know more, search the net for:

• ABI
• name mangling

or ask in a C++ forum or chat, why C++ compiler are not binary compatible.

renrutal wrote:I don't understand why the C interface can bring multicompiler support when the C++ couldn't.

In short: compiler differences. C is relatively more standardized than C++.

There are no standards and conventions for how C++ objects and structures are implemented in compilers ( and if there are, theyre not strictly adhered to), but there are standards with C calling conventions.

As a result if we create an object in different compilers using the same class type and value, they may represent the same data but they arent the same internally. As a result when spring data is sent to the AI in a C++ interface, there are issues with differing compilers because spring assumes the other binary can use the data its sending, whereas it may have different padding or encoding or arrangement, it may use a different structure, it may be in multiple pieces when the target binary expects a continuous block, lots and lots of differing things.

You cant just put a cog into a machine, it may have round teeth or square teeth or jagged teeth, it may be a big cog a small cog, it might be steel, aluminium, wood etc

Its a common conception that AI developers have abandoned the content, and are embarking upon an orgy of development by reinventing the wheel and cavorting with new and exotic programming languages nobody needs. This is nonsense, ignorant speculation by people who didn├óÔé¼Ôäót read the forum threads and made wild assumptions without any effort to understand the problem.

1. It's not "ignorant speculation". Nothing new has been done in this arena, that hasn't been done in Lua. Now, I'm not saying that what Hoijui did wasn't entirely necessary, and I am glad he did so. But now it's done, so you're out of excuses either way

The Chicken AI represented a major step forward, yet there's been hardly any real discussion here about its obvious implications, and you have been guilty of saying, "waaah, I don't know the language", which is no excuse at all.

2. It's been obvious that the content has been abandoned. Nobody's actually doing anything about the serious problems that exist for Lua-based games, which became the mainstream almost a year ago.

3. The notes about "new and exotic programming languages" are right on- C#, Python, and others have all been mentioned, but you guys seem to have a massive blind spot when it comes to Lua. You talk about how to send it commands through the interface, but you're not looking at what's actually out there.

4. The seeming speed disadvantage of Lua is a chimera. As Lurker pointed out, Lua can run mainly on the unsynced side. There are some exceptions to this, but they're not that big of a deal.

Moreover... most of what makes current AIs relatively slow is big, giant analysis stuff, that isn't going to run significantly slower in Lua. Lua, when compiled, is nearly as fast, and for those large processing tasks, the final difference in efficiency isn't that big of a deal.

Lastly... why should an AI that can play a game adequately necessarily be a giant processing monster? Go play Chicken, and observe an AI that works and is not terribly slow. Once you get done sneering at calling it an "AI", because it's just following some simple rules, you might want to observe that CA has made it their default single-player experience, which says a lot about what they perceive players want- a challenging, fun experience, not necessarily a re-creation of play online.

As I've been saying all along, it's not necessary to build "smart" AIs... it's necessary to build AIs that can give players a fun experience, and work hand-in-glove with game development.

Quit confusing the two things. The whole effort to build "smart" AIs that play BA well is not equivalent to the real need, looking at the present-tense. We have a lot of games that need any AI support they can get, because there isn't even a very dumb, cheating AI available that can work with their gameplay.


****************************************************
Meh, enough bitching. In the final analysis... here are some thoughts about a way forward:

1. AIs need to abandon the closed-cycle mode of development and quit being black-box. I said so almost three years ago, you guys didn't listen, and now we're here.

It's incredibly stupid and naive to develop AIs that were basically just customized for one game type, then wedge support code into them to half-heartedly support new game types. Yet, this is what has been done. Look at every AI that currently exists, and you can see the kludges that were done.

This is a dumb model for development. Game designers should be modifying your AI frameworks for final behavior. AIs should essentially be about developing smart ways to efficiently do certain types of analysis.

2. AIs should use as much Lua as possible, if not be written entirely in Lua. Perhaps it's necessary for AIs to perform a callin to a DLL to perform certain math-intensive tasks. Fine, put those modules into AI DLLs, let Lua talk to those DLLs. I'm not suggesting or supporting allowing Spring games to create or use any old DLL, mind ye- that's unsafe behavior, and should not be allowed. But stuff like finding optimized locations for mines, etc., could all be stuck into those DLLs, allowing for call-in behavior at the Lua level.

Why? So that game designers, who work in Lua, can modify them themselves, instead of having to set up a compilation environment, etc. That puts game designers in the position of being able to have a useful dialog, instead of having to waste huge amounts of time explaining to AI developers the gameplay we're hoping to create.


Here, lemme build the flow of a typical task:

1. Unit is damaged.

2. Lua-side AI code determines Unit can perform action X during combat.

3. Command is issued, via Lua, that makes the Unit perform action X.

No DLL-side behavior was even necessary.


Let's look at one that would require DLL-side behavior.

1. Lua basic determinant code returns values indicating that more of a Resource is needed. The Resource corresponds to Metal.

2. The method in this game for getting the Resource is to build a Mine. The DLL is called, sending arguments for the size of the Mine, the radius on the map, the maximum distance to search away from all candidate Units, etc.- the usual things an AI DLL currently needs to know, to perform this task efficiently. The DLL then runs an efficient search for the best-possible location to put the mine, and returns that value, the second-best location, and so forth.

3. The Lua-side picks one location from a hat, and gives the appropriate orders to the Unit.


See what I'm talking about here? Instead of looking at things as being all-or-nothing... perhaps a more synergistic approach would be better.

after reading this post, i see ooooooonly confusion, and be-wilderness.


let me tell my view of it all:

AI development is not a simple task, and it is usually done by people who have a special interest in AIs. They are prepared to spend a lot of time for that and read a lot.
Your idea is, to let mod/game developers do that.. kind of by the way (profiting from some globally available algorithms, like metal spot finding).
the old/current API for Skirmish AIs (the purely native one) is made for people who want to do only AI development, and of course they want to decouple from games/mods as much as possible.
There are two reasons most AIs perform bad or not at all with non-TA-like game-play mods:
1. AIs have no way to access anything that is LUA based
2. the whole task is just too complex.
with the second point, i mean:
to develop an AI, the AI dev first needs an idea of how to best play a game. then he tries to get the computer to do that. that is a hard task.
this same hard task would have to be done by game devs (by the way), if AIs are gonna be LUA based, or they need an AI dev who is dedicated to their mod/game.

A skirmish AI that would support all type of games that are possible to create with spring (including all fancy LUA stuff)...
I can not imagine that this is possible for a human.
Such an AI would essentially be one big algorithm that works for all games. not even humans can do that. if you think of it:
we play BA.. learn it, get better and better. even when switching to XTA or CA, we have to relearn some things ("ahh, this tower can not shoot to air in this mod, even doh it looks the same"). when playing a more differing mod, like nano blobs, kernel panic, 1942, micromachines.. we have to create a new game model in our heads (of course, we can still borrow some things from the BA model).
a skirmish AI can not do that, as it is high Hz, low parallelism based, in contrary to the human brain.

I would like Skirmish AIs to have access to LUA stuff of games, but i can not imagine that we would see globally(all possible games) capable AIs.


An other point, is what you see as need.
For you, AIs are there to let gamers have a decent game, if they do not want to play online, or for training (when they are new to the game) or for special game modes (tower defence, chicken games, etc.).
For me, the spring engine is just a tool to test AI techniques. The main reason I wanted to do a Java interface, is to make it easier for AI researchers to use spring. most researchers will want to use their preferred language, which may be Java, C++, C#, Phyton, LISP, Haskell...
and the new C interface will make it easier to add any new languages, which to me personally is much more important then the different compilers aspect.
Ok, they could use LUA, which is easy, but a lot of them will not consider that (even if we had a LUA based Skirmish AI interface that works with all mods).
They may have a lot of algorithms in Java or C# already, and would then have to figure out how to use these from LUA, or rewrite in LUA.
I know nearly nothing about LUA, so I can not think of any other arguments.

I'm not confused. I'm aggravated, frustrated and annoyed. You see AI as something fun for grad students to mess with. I'm a game designer- an AI is a practical part of a finished game design. Players expect AI functionality. A game without functional single-player might as well not be released, as it will see very little play online if people can't get comfortable with it. I've had to bend backwards for nearly a year, keeping one faction of P.U.R.E. "compatible" with the existing AIs for Spring, mainly because of these issues, and it's really silly.

Now that the Great Interface Overhaul is over, it's time to talk about this in practical terms. I'm not really interested in theory, and I've harangued these people many times about how little regard I have for the obsession with making AIs "smart". We don't need "smart" AIs. We need AIs that actually do their job.

As for the rest of it:

1. There is no practical reason I can think of that Lua cannot be allowed to callin to an AI DLL. Maybe this function does not yet exist, but really it's just the same as another callin- you're just sending a bunch of parameters to some compiled code, and expect a set of return values that can then be acted upon.

2. You're assuming that things are impossible for game developers, because you're looking at AI like it's a single process. AIs are not single processes. They are a combination of the following activities:

Global strategic decision-making logic. This is the part that you're referring to as "hard", although there is no real reason for most of that to be "hard", for practical game AI. This is sub-divided into several distinct sub-tasks, most of which have no reason to be in a compiled language at all.

Resource distribution and analysis: could be Lua, it's running only every few seconds anyhow.

Attack determination: for a cheating AI, this is pretty straightforward- either look for closest enemy X, or a specific Target, and send attack Units to this location. Gets fancier, if you want to send to multiple targets, etc., but it's essentially the same logic the whole time.

Defense determination: if enemy Units within X of any object Y, then move some units to defend it. Again, this gets more complex, but that's essentially all that it is.

Building placement: this is a task where AIs could directly benefit from the C++ algorithms that are already available in KAIK and AAI. I'll be the first to say that very efficient algorithms for this special-case activity exists, and should be leveraged.

Scouting: for non-cheating AI, it would probably be best to callin to a DLL for scouting analysis.][/list]

Tactical game decision-making logic: In Spring, that's practically not necessary, for the vast majority of games- simply let the Unit sub-AI do its job.

However, this could benefit a lot from a Lua loop that directs the Unit to perform a Command or series of commands. We've already seen automated-micro Lua developed, which other than KAIK is ahead of anything done in Spring AI already.



In short... AIs are nothing but a discrete set of problem solvers. Some are problem solvers that are specific to AI development- efficient utilization of a space, for example. However, the vast majority of the tasks aren't that complex, nor do they need to be tied up in a DLL where game developers can't mess with them.

It should be up to game developers to hook specialist algorithms and processes together into a functional whole.

For example... a Lua callout from a Lua resource-management system requests a place to put a building. An efficient AI DLL then replies with what it thinks is the most-efficient placement location. Then the result is executed with Lua.

Your point about game expertise is exactly my point... why should AIs only be developed by the people least-competent at understanding the games in question? Why aren't AIs developed as modular parts that are integrated into a whole, using Lua as the binding agent? Why aren't AI developers seeing this as a great opportunity to escape the compatibility trap they've been buying into?

Argh your showing a lack of understanding.

Its not hard to make an AI work and play, I built NTai using a handful of C~++ knowledge in a single weekend initially. Currently the lua AIs are not complex, and there are limits with lua AIs but they ahve not been reached, limits which are much further away for native AIs than lua AIs.

You see, the AIs simply 'worked' for TA based games within a week or two of the first two AIs, JCAI and NTai, yet we somehow improved them and made them play better.

Sure placing mexes is easy if you have a spot finding algorithm, pick an unclaimed mex and build. Or is it? Lets try to improve that. What if tis cheaper to build a metal maker? What if the mex spot is in a contested zone and thus not worth the investment because there's a big army currently walking through it. What about enemy bases inbetween you and the spot? Should we send the builder walking through the enemy base? Should it not build it and instead go help another con build a mex that's half built? Should we build a cloakable mex or a mex with a gun on it? Floating mex or underwater mex? Will the mex structure actually build on that train or does it need evening out?

Argh not all of us want to learn lua, and some of us don't have the time to work on learning a new language and maintaining the AIs at the same time. Content developers complain to us, harass us, flame us, and lay the blame all on us. We're supposed to go into their content and fix the AI for them, and do all the work, unannounced, for free.

Sure we could give people something that 'works' but that's not good enough for us, and people don't appreciate that what we do is much harder than they think. We have to handle a lot fo use cases. This is why I believe NTai and kernel panic didnt work where a simple lua AI did. NTai was not flexible enough and could not cope with the paradigm of masses of units in a world where there were enough units and payoff to justify the architecture and extra unit behaviours that were utterly pointless in a game like kernel panic.

What's more KP didn't have the problems TA style resource models do. Nor does chicken AI need to deal with these issues.



Argh you have a habit of waltzing into this forum, calling us slackers, telling us your disapointed in us, and saying everythings our fault, then preaching to us how it should be done. You have offered us no help, you have not told us anything of value.

We are told lua<->AI is not as hard as it seems, however we have no experience with this and those who have figured out are not sharing. Perhaps you can enlighten us? You do seem intent on saying we've abandoned content, which is not true, we simply have a roadblock, and we don't know how to shift it. We have ideas (but we cant implement them), and plans (that we cannot complete). Content is not abandoned, it is simply out of our reach.


As for the great redesign, it has been implemented, but it has not been deployed.

I was writing a post to this when hoijui submitted his, makes my life easier cos now i can just add +1

maybe arg, it would help you understand if you take a look at some AI code, KAIK for example.
remember that KAIK works well with BA (which practically never changes content), works soso with XTA and is nearly till totally unusable with other mods. still, it is quite complex already. this amount of code would be needed for each mod/game (with your idea of the AI system in place).
also.. if you are willing to do this for your mod, what exactly are you missing in LUA? Can't you do that now already (like the CA guys did with the chicken AI). If the only thing that is missing is way to call native dlls, then we could add that maybe. it would be easy to have cmake build a MetalSpotFinderAlgorithm.dll, if that is all you want.
But this will not make the Skirmish AI interface obsolete.

about what we need:
i told you what i want, what i need. it's different from what you want/need. there is also a "we" that needs/wants the same stuff like me, and would not agree with you.
if WE need to make spring more popular in the gaming community, and more interesting fro players, you be right.
if WE need to make spring better for AI researching, i am right.
i can't see how you can see things so clear.

What's more KP didn't have the problems TA style resource models do. Nor does chicken AI need to deal with these issues.

Well, then, look at that as an example of how to innovate. Chicken is simple, and works, largely because the AI blatantly cheats.

Instead of seeing AI development as simply a way to model human players, see it as a game designer does- as a trainer that's designed to teach people to play your game at a basic level.

Nobody on the game design end of things seriously thinks that, as Hoijui put it, that AIs should be available that play everything like a human does, and as well. What we need, and why I'm bothering to spend some time yelling about this, which you make it sound like I do often, when in fact I haven't bothered in months... is to get you to shift paradigms here, and think about what we need AIs to actually do.

Sure placing mexes is easy if you have a spot finding algorithm, pick an unclaimed mex and build. Or is it? Lets try to improve that. What if tis cheaper to build a metal maker? etc.

That is precisely the kind of problem game designers should be handling, not AI developers. We need fast core algorithms for the difficult problems, but we should be responsible for how that information is handled.

Content developers complain to us, harass us, flame us, and lay the blame all on us.

No, for the most part, they've given up hope and they've just quit posting around here, other than occasionally begging for AIs to be fixed for their game.

In fact, so far as I can tell, I'm about the only content developer who routinely comes here and bothers anymore. I'm here complaining, flaming and harassing because I think that you guys need to fundamentally re-think your gameplan moving forward, or this whole area of the Forum represents nothing more than research about stuff that has very little in the way of practical application outside OTA mods. This is a very serious problem- the engine's capabilities have grown, but your outlook as a group has not.

Sure we could give people something that 'works' but that's not good enough for us

Yeah, right. Give us something that 'works', and we'll be happy that we have something that works at all.

We are told lua<->AI is not as hard as it seems, however we have no experience with this and those who have figured out are not sharing. Perhaps you can enlighten us?

The handler code for Lua exists to pass either the return values of a given function within Spring (the callins) or to pass arguments to a Spring function (the callouts).

To make it possible for Lua<->AI communication to occur, you just need a new callout for Lua (Spring.LuaToAI, or whatever), with something like these parameters: DLL name, arg1, arg2, arg3 ... etc., where it is expected that one knows the required arguments and their values for a valid argument (i.e., it's documented with the AI DLL).

There's nothing fundamentally terrifying about this- Lua's just a way to communicate with Spring. Spring can already pass information to-->from AI DLLs, so I would not think that this represents a fundamentally difficult problem, except that obviously AIs would need to be designed to expect incoming messages and to act (or not) accordingly.

If the only thing that is missing is way to call native dlls, then we could add that maybe.

I see that as a fundamental step in the right direction, yes. There are certain tasks that are hard problems, and it'd be very nice to see them created as modules that could be called upon individually (and, as an added benefit, AI developers would not have to reinvent those wheels, either, as those areas became relatively optimized).

i can't see how you can see things so clear.

Well, obviously this is my POV as a game designer. I have nothing against AI research, per se, I simply think that if it is not addressing the practical needs of players and game designers, then it's not actually interesting, and moreover the research value is a lot greater if you can show that it actually solved a real need.

Moreover, I think that for those of you looking at your papers or presentations in the future... what's going to be more impressive to a future employer- demonstrating that you made an AI that plays one particular game well, or demonstrating that you've constructed a framework that could be redirected to many game designs? If I was an employer, I'd be a lot more interested in the latter, because of what it says about how you think strategically- solving today's problem whilst developing technology that can be used for tomorrow's.