Spring RTS Engine

I wanted to start a conversation here on what everyone's ideas are for unit learning. Things like what factors to include (damage, hp, kills, etc), what algorithms to use (weighted efficiency, overall effectiveness, etc), and others.

One of the things I'm trying to decide on is whether or not to factor unit resource cost into the learning to average out costlier units from cheap ones (unit efficiency vs resource cost), or to leave it out (general unit efficiency).


Leaving it out seems bad, because then we have the Krogoth problem. Krogoth is so powerful, that it should have the highest kill ratio of any unit in the game. Would that mean that the AI should only build Krogoths?

Putting it in also doesn't seem to work so well, as then we get the Peewee situation, where peewees are so cheap to manufacture relative to their damage output, that every expensive unit gets left out.

Any comments?

First you need to categorize the basics.

Attack, Fast Attack, Scout, Assault.

Then by type, since trying to build the High Kill Ratio Battleships, on a desert map:

Sea, Land, Air.

Then account for what they're attacking:

Unit, Building, Defense, Mass.

Finally, account for resources:

Metal, Energy, Time.

A Krogoth is a Land, Assault, good against buildings, but high cost.

A Peewee is a Land, Fast Attack, good versus units, low cost.

A Sumo would be Land, Attack, good versus defenses, med cost.

Right, I've already got a bunch of categories, but how do I have the game figure out sumos are good vs defenses, peewees are good vs metal extractors, and such?

My categories are currently:

Ground
Air
Defense
Buiding
Sea

Each of these has two parts, a low hp category, and a high hp category.

Lets do an example:

Lets say that a Sumo runs amok and kills off 20 peewees, 3 LLTs, a Guardian, and 3 Solar Plants.

Say it and the guardian kill off each other at the last moment.

The peewees each hit the sumo for 25 damage before they died, for a total of 500 damage.
The LLTs each hit the sumo for 150 damage, for a total of 450 damage.
The lone guardian pummeled the sumo for a while for 2500 damage.
And of course, the solar plants didn't do too much.

(Ignore the fact that the sumo has 10,000 health for the moment.)

The peewees take 100 metal each. The LLTs take 300. The guardian takes 2200. Solar plants each take 150. Sumo takes 2300.

What sort of efficiency changes for each unit should we get out of this skirmish?

AAI's learning system has 6 different categories (land, hover, sea, air, arty, stationary defence) and keeps track of which unit kills other units. the only other thing it takes into account are the costs.

maybe i should also take health into consideration - but prevent the ai from ending up building only sumos :)

Hmm... Sumo = teh leet?

Remember that we have to take unit speed into consideration as well. So a slow, high damage, high health unit, is considered an assault unit.

The change would go like this:

Peewee's, bad versus Sumo.

HLT's, bad versus Sumo.

Guardian, average versus Sumo.

Solar Plants, use exclusively against Sumo... to delay it so the guardian gets more shots in.

I have one question about implementation of unit learning algorithm:
How can I detect when one of AI units is doing damage to enemy? There are UnitDestroyed, UnitDamaged and EnemyDestroyed events in GlobalAI class, but there isn't EnemyDamaged event. How did you manage with this problem in your AIs?

Sorry for my bad English.

i dont check for damage dealt to other units, in aai only kills count (of course there are some disadvantages of that system)

In NTAI I have a single efficiency value though admittedly I could do with a other categories as you have done.

What I have ended up doing for damage is something similair to damage/metalcost where damage is the lost hp. That way health cost and fire rate are automatically factored in.

I think i also did the same for kills only hpofkilled/metalcostofkiller and I added a lesser multiplier to the damage doing thing so ti didnt have as much of an efefct as killing off a unit.

If a unit is killed or damaged I do the same calculations then subtract the result.

That way i have a nice efficiency system that works, the initialising values to the untis hp means thigns liek fusion reactors get an efficiency too rather than the rare fusion explosion raising efficiency and fusions never showing up on the threat matrix. The simplicity of the algorithm makes it faster than running a much heftier algorithm, as I can expect this calculationc an occur many many times.

However unit learning through values like that cannot simply be affected through unit killing/damaging other units as it isnt enough to represent learning efficiently, the values need to be affected and used by other systems than just building choices.

An example of that is attack targetting and the threat matrix in NTAI which relies on being filled by efficiency values, so units killing units changes how the AI percieves and reacts to an enemy threat.

However i shall not go into this as much as I'd wish unless others say so as I have gone into great thought about this over Epic and I dont want to give things away.

I think the AI should adapt to the attacking units.

Keeping a list with counts of incoming units by category (anyway old data should be cleaned up). Then regarding on this list the AI should know what to build, maybe fixed data (bad for unit expansions).

Another approach to find out what to build, could be an effectiveness relation between ressources and dealed damage. So if the AI recognizes an low tech unit does better damage compared to ressource cost, than a more expensive one it will build the cheaper one.

Anyway this does only apply for building units and not building base defense. Cause there should always be anti-air coverage even if the enemy never attacked with air. Especially around expensive buildings.

EDIT: The ressources/dealed damage effectiveness should be handled seperately for every threat-category.

I don't know what the AI can do in this game, but this would be some improvements i guess.

If you have been following my series of arguments with the AI development community, and the outcomes (which, now that I have an AI that I can configure with predictable outcomes, is actually getting somewhere interesting)... then you can probably skip this, because I've said most of it before, elsewhere. However, since you're new to this, here's my rant on this topic. Disregard it if you wish, but I'm a modder- you're an AI developer. Want your AI to be promoted? Want your AI to give players a good game? Then pay attention.

Rant-mode ON

Personally, I think that all of these numeric approaches are utter crapola. Want a demo? See how NanoBlobs plays with NTAI 7.5XE if you doubt that modders can and do understand the relative strengths/weaknesses of their units better than a piece of code ever will.

You want your nifty new AI to own the humans, in a fair fight? Fine. Give me a fully-configurable AI interface that allows me to script what the AI should do when it spots a given unit, cluster of units, or a stationary defense, and give me smarter sub-behavioral scripting that I can call given conditions. Give me the ability to tell the AI to gather 20 Pewees, 10 Flashes, and 10 Mortys, if it spots a HLT and a Guardian in the same sector. Give me the ability to tell the AI not to attack under certain (modder-defined) conditions.

Quit assuming you know jack about how to play, or about how mods are designed, or about game-balance! If you did, and had the billion-and-one other skills required, you'd be mod designers. Instead, look at what you're doing this way. You're basically building a piece of engineering. If you're doing a good job, then you're building software that allows modders to do their job, which is game design. Which, in the professional world, includes specifying parameters and control for AI. Good modders who really care about their end product can and will design good AI scripts, if you give us tools that are worth using and produce useful outcomes.

It's simply stupid/overweening to assume that an AI is learning anything useful if, say, it gets 50 Peewees owned by one good player who has placed two HLTs on small hills to either side of a pass that the AI is dumb enough to keep using.

C'mon. Don't kid yourselves. What is it "learning", that is worth "knowing"? That HLTs always > Peewees? What rubbish! What a human would learn is not easily transformed into numeric logic. A human who knows how to play would learn, "hey, if I want future Peewees this way, they're going to get owned. I should attack the HLTs first, even if all I have is Peewees at the moment or back off until I have the proper force concentration to attack through this defensive line."

Go on. Break that down into numbers. I dare you. Your brain will melt. Or your processor. Either way, it's not happening. And you're never going to get this approach by simply throwing iterations at an "efficiency calculator".

Instead, solve the problem by giving modders, who actually understand game design... the tools to address specific situations. For example, in NTAI 7.5XE, I can now assign "target weights", and NTAI will tend to attack certain targets before others. Now, the way it's currently coded doesn't really work very well for NanoBlobs (mainly because NanoBlobs is already flooding the network once you're past the early game), but it will work very well for slower-paced mods that don't have hundreds of units dying per minute. Does this "target weight" feature involve the AI making "intelligent decisions" or using any kind of "efficiency calculation"? Hell no! I want that AI to attack the Lord (in NanoBlobs, this is the Commander-equivalent) on sight, period. If the Lord is spotted, every attack unit should re-route to its location, trying to stomp on it, because that's what a human would do (at least, an intelligent one).

The approach used, where various numbers about outcomes are somehow supposed to result in smart behaviors is a complete dead end, and I've been watching various people beating it to death for months now with some amusement. If you want smart behaviors... program 'em, and give modders ways to access them in a way that makes sense. Then let modders who actually care about AI support configure a script. A common interface and agreement on the rough meaning of standard parameters and commands would also help quite a bit, because then y'all wouldn't be constantly re-inventing the wheel. AF has at least a plan for an interface, but I think that it's equally crucial that people get their heads wrapped around a very simple idea: if your AI requires me (a game designer) to learn a completely different set of configuration commands than another AI, and that other AI is more advanced / polished... why should I waste my valuable time on learning anything about your AI? Think, people. Modders are busy and we want stuff now. I am not learning 5 different sets of configuration scripts- I am going to learn the one that works best and is easiest to get done during the playtesting cycle. Making up your own (if you even bother... and if you don't, forget it, I'm not touching your AI) just to be "special" is a waste of everybody's time. Instead, announce that you're extending the common standard, explain why, and document your code. That way, you're not wasting anybody's time if your idea is useful. It's straight-up Open Source common sense.

Even AAI doesn't use a purely numeric "efficiency approach" at this point, otherwise Submarine wouldn't have broken compatibility with mods that aren't very OTA-like. Or maybe it's the assumption that everybody thinks that mexes are vital to good gameplay. Or whatever.

Which is too bad, since AAI at least gave everybody else a useful benchmark to test against, and if it had been kept stable and universal, it'd be more useful than it actually is. What I find ironic is that Submarine isn't supporting the current NanoBlobs when it was about the only mod that's been made that gives AI developers a good starting place for evaluating the design philosophies behind efficiency calculation in a rational way, with a real, live testing environment Again, thanks for missing the point...

To sum up a somewhat-rambling post (sorry, I keep going on side-jaunts cuz I'm at work and keep getting interrupted, plus this IS a rant)...

AF's work with me on NTAI's attack patterns just confirmed for me that "learning" is completely over-rated. The NanoBlobs configuration, which plays a very mean game, uses no "thinking" whatsoever. The AI is purely scripted and state-driven, and really it just follows simple linear instructions, for the most part.

Instead, what we really need are AIs that will do smart (i.e., human-designed) things, like, for example, the way that NTAI 7.5XE will gather units of a given Attackers group in a random place along a patrol path, then send them in a group. This works wonders for attack effectiveness, and involves zero "learning". Due to the way that this was coded, it usually involves a semi-random attack vector, too (although I personally think it needs to use a wider seed value for the patrol paths, as it doesn't hit the side flanks quite well enough yet, this is, again, not anything we're going to see AIs "learn" ).

Once more Argh, you demonstrate your remarkable talent for telling us things we already knew. And at the same time, you've managed to insult the entire group of us. Congratulations.

Creating an AI is not exactly the easiest thing in the world I might remind you. And this isn't exactly a simplistic, and smooth running interface. What you're looking for isn't an AI, it's a simplistic, and smooth running interface. If we had one to work with, we'd be able to give you what you're demanding in a very short time. Since we don't you have two options.

1)Download the spring source code, and sort out the interface for us. Then we'll give you a nice simple system to work with.

2)Stop bothering us, so we can fix this tangled mess of commands and controls and garbage. We'll get back to you when we figure it out.

Thank you.

Triaxx2 04-05-06

On the contrary I thinks it's high time Arghs words are heeded.

I am willing to make concessions tot eh NTAI format for XE8 and at the same time extend OTAI and SAI to use them, and keep data that they would sue that NTAI does not use, and rename the NTAI folder to AI.

I'm not sure about QAI and AAI but I doubt submarine would ever consider it, and I haven't a clue what Q-Man intends....

For example I'm willing to add in map data and OTAI values into the design, and give SAI a nice way of eliminating hard coded values, though I'm not going to key in universal build tags into OTAI and SAI unless they have existing functions such as GetMex().

The problem here is that you're saying that learning algorithms are good etc, and Argh is saying they aren't. In fact your both wrong on a conceptual level.

Learning algorithms can be modified by the modder given an interface. That's why i stored NTAI learning data in tdf format (argh try artificially inflating the value of the Lord in the learning file and it'll have a huge knock on effect on NTAI's perception of the map when it sees the Lord(Lord makes bigger spike on threat matrix, which is noticed by targeting routines)).

Your learning algorithms lack context, which is a huge problem you've overlooked. That is why what Argh preaches works so much better than what you've implemented. Whatmore another point Argh raised can be applied, who are you as an AI developer to tell modders this is how to play their mod?

As for more control, in XE8 every unit can have a task queue (note task queue not build tree), including attack and scout units, and thus you'll be able to define not just what attacks but how it attacks....


Also a note concerning learning algorithms and efficiency values as you originally started the thread for.

These values cannot simply be used for unit construction choice selection, they need to be more integrated into other systems, and affected in other ways, otherwise your 'learning' is stunted and the AI doesn't get the chance to adapt. Think of it not as a value representing learning, but rather as the 'value' placed upon that unit, and that value needs a context. Your concept of this whole thing is all fundamentally flawed, your concept isn't sufficient to effectively use these types of systems.

You'll understand when I describe Epic in more detail but now is not the time.

you've managed to insult the entire group of us.

No, I'm just insulting the people who worship at the alter of efficiency curves and the idea that AIs will somehow magically arrive at the right conclusions based on inconclusive data. When it is quite patently obvious that getting the amount of data out've Spring in a realtime game needed to go beyond very simple counting behavior is not desirable, given performance demands, it would seem that my argument is both self-evident and solid. No amount of improvement to Spring's underlying interface is going to change this.

If all you can really do, at high speed, is count, and doing things like heuristic pattern determination, logical analysis of the terrain, etc. is not really practical, then this is a complete dead end. Why waste time? Instead, take another approach, and do this through pre-built behavioral routines that simulate human-like responses, controlled by modders through a useful scripting interface.

Here's an example, using BOS:

What you're looking for isn't an AI, it's a simplistic, and smooth running interface.

Not true. What I am looking for is an AI that actually behaves in a logical, human-defined fashion, with predictable, useful results.

And if my option is to sit down with Spring's source... fine. Tell me where the biggest problems are, in terms of getting AIs to do what you need them to.

Based on previous posts around here, it appears that you can already send interface commands and get the XYZ positions of units via callbacks to Spring. What more do you need, and where is it broken?

The spring itnerface iself isnt quite broken, rather the AI's themselves are the problem.

However I am redesigning the AI interface in spring next week (ICT CW comes first)

First off, AF is correct. I like the threat-matrix approach, and think it works just fine, if given context. My argument is that you're not going to get this through an iterative approach. Humans can and should steer the boat.

Take NanoBlobs, for example. The Lord is incredibly vital, even in a game where Commander Death <> End. But, as an attack unit, the Lord is mediocre, at best. What learning approach is going to assign the Lord its proper value? Based on Workertime? Well, that might work for NanoBlobs, but not for other mods. Context is everything.

I will try what you're suggesting, as I continue development and testing of 0.4 / 1.0, AF- that's one of the few things I really wanted NTAI to do that I didn't feel was quite right. I want human players to feel the same palpable panic they feel in a real game, against a real player, when the Lord has been spotted (trust me, it's quite panic-inducing to have several hundred things suddenly change course and head straight for your most important building unit).

in XE8 every unit can have a task queue (note task queue not build tree), including attack and scout units, and thus you'll be able to define not just what attacks but how it attacks....

... and that kind of thing is exactly what we need.

be aware that I've noticed a flaw, untis killing lord will probably lead to lords efficiency going down or up not sure which, but make it an astronomical value, nto too high though else units will flock to where the lord has been rather than to the lord........

Threat matrix values only go down if it's the 0,0 square or you have allied units in that sector, else they build up.

I would imagine, on balance, that the Lord probably gains efficiency based on your calc. The amount of Buildtime in a Lord is quite small, and a Lord can and will kill quite a few things before dying, most of the time.

hmm, well look in the file and see by what ratio the value has increased, remembering its initial value is the health ingame + what it produces energywise+metalwise I think.

AF put it best I believe. The Spring AI interface is much like spaghetti. You don't know where the other end is. I've dug into every version of the code released, and tugged on the metaphorical strings. The problem, is I never get the same response twice, and sometimes I get several.

What I would really like is a list of what does what. I don't even really need specifics. But at that point I can go in and give you at least a translation interface, so that if you tell it gather four peewees and attack the commander, it's going to do it, instead of trying attack the commander with the factory.