NTai XE10.1b

[lale]

Source for MAJOR issue identified!
- The NTAI performance problem, NOT build placement.

Setup:
Any non flat map (eg. no green_field or speedmetal)
Config file that techs to level 2.
Max 350 unit per side (or under 1000 units combined.)
Single core cpu @2Ghz, plenty of ram, fast graphic card.

Problem:
With RAI or QAI, my FPS (frames per second) stays for the entire game (several hours) around 20FPS with the above unit cap.
With NTAI, my FPS also stays around 20fps (with stutter every time a ├óÔé¼┼ôbad pos├óÔé¼┬Ø marker is set). However about 15 to 40 minutes into the game it suddenly (within 10-40 seconds) drops to 2fps.

Observation:
When 1, 2 or 3 units start spamming (from spring, not NTAI logging) ├óÔé¼┼ôCan├óÔé¼Ôäót reach the destination├óÔé¼┬Ø to the console, the framerate drops drastically (from 20fps to 1fps), making a game with NTAI unplayable.
The framrate returns to 20fps, ONLY when all units spamming ├óÔé¼┼ôCan├óÔé¼Ôäót reach the destination├óÔé¼┬Ø are destroyed.

This happens with all units (L1, L2 & L3. Land, Water and a few times also for Air)

• I├óÔé¼Ôäóve seen a sub spam ├óÔé¼┼ôCan├óÔé¼Ôäót reach the destination├óÔé¼┬Ø because the tries to attack land units
  I├óÔé¼Ôäóve seen a krogoth spam ├óÔé¼┼ôCan├óÔé¼Ôäót reach the destination├óÔé¼┬Ø because it tried to exit the factory, before the factory had ├óÔé¼┼ôreleased├óÔé¼┬Ø it. (it wasn├óÔé¼Ôäót at 100% health due to damage taken during construction)

- All above conditions (and many more) which leads to ├óÔé¼┼ôCan├óÔé¼Ôäót reach the destination├óÔé¼┬Ø, leads to extreme drops in fps, and unplayable game vs. NTAI.

Probable cause/theory:
When a unit spam ├óÔé¼┼ôCan├óÔé¼Ôäót reach the destination├óÔé¼┬Ø it asks NTAI for new orders, constantly, and with a very short time interval (few frames between new requests for new orders).
Since NTAI gives, most of the time, the same order to this unit, this leads to a CPU hog lock.

Please AF: FIX THIS!
I still feel a game vs. NTAI is a playable experience, even though:

• Stutter due to build placement eating cpu cycles before making a ├óÔé¼┼ôbad pos├óÔé¼┬Ø mark
  Solobuild only working as ├óÔé¼┼ôreduced parallel build├óÔé¼┬Ø
  ConstructionExclusionRange not working at ALL.
  b_rand_assault and b_assault not fixed (though the fix is simple, see earlier post)

But this issue absolutely needs a fix!

Every player with a single core CPU (even a Core Sole OC to @4GHz) is going to find NTAI unplayable due to this issue. (Goidse, 1v0ry_k1ng and DJ is affected, see previous posts in this forum)

Btw. The reason I haven├óÔé¼Ôäót released my config file is due to this issue, I can NOT playtest against NTAI, when my config reaches higher level, as this issue is most prominent with big units.
I will release my config within three (I need some time testing config with big units) days after you release a new NTAI version with this bug fixed.

I will make a new post this evening, with a logfile from above issue. But since the logfile don├óÔé¼Ôäót show cpu usage, I do not think much info can be derived from this.

[Goidse]

I agree completely!

Thats why I build a config wich start raiding very soon in the game. It will try to kill the enemy in the first 20 minutes, otherwise the cpu is overkilled :).

[AF]

I’ve seen a krogoth spam “Can’t reach the destination” because it tried to exit the factory, before the factory had “released” it. (it wasn’t at 100% health due to damage taken during construction)

This affects all AIs and has never been noticed before. This should be in its own thread.

For submarines, I have no idea how to test if A can attack B. This means that I cannot determine if the sub can attack the land unit or not. If you look up you'll also see that I dont have a set of checks to test if a unit is a submarine to begin with.

As for logfile, the logfiles just going to give me the same info you got from the console. What I need if a common repeatable cause that i can fix because your describing it as if it has 100 causes.

Identify the problem (not the effect) and suggest a solution. In this case krogoths not at 100% but finished building is not the same problem as submarines attacking ground units.

And goidse, if you had these issues why didnt you post them instead of waiting for lale to post?

[Goidse]

at page 55, somwhere on the end, I said this:

After a while the AI just overkills my cpu :s. Maybe some CPU tweaks would be awsome!
When a bot can't reach a destination it keeps spamming that and the cpu-use peaks :/.

I've thought about the can't reach destination problem too, but after a while you can't say what exactly overloads your cpu.

Anyhow after a long play, 35-40 mins +, the game just constantly drops to 1 fps. You are spammed with bad building pos, attack points setting and can't reach destinations, so I can't determine wich causes the biggest overload for your cpu.

[lale]

OK, my overlong posts sometimes confuse more than states a clear issue.

The problem is simple:
When an attack unit (for any/all and whatever reason) spasm ├óÔé¼┼ôCan├óÔé¼Ôäót reach the destination├óÔé¼┬Ø NTAI start to eat massive amount of CPU cycles.
(Since I do not have precise insight into Spring AI code interface, I can't be more clear on the cause. But I still think this has something to do with attack units constantly requesting new orders from NTAI)

Solution:
Hmm more problematic, I will dedicate my next (overlong) post to some suggestions.

------------------

And AF don├óÔé¼Ôäót be to hard on Goidse, I├óÔé¼Ôäóve spent over 23 hours of tedious game testing hunting for the cause for this bug.
(I should have read Goidse posts more carefull)
I didn├óÔé¼Ôäót know if the problem was in the spring engine, or in NTAI.
And I didn├óÔé¼Ôäót know if this only affected single core CPU├óÔé¼Ôäós.

DJ and 1v0ry_k1ng is also affect, but like me and Goidse did not know what the cause was. So they still think it├óÔé¼Ôäós directly related to the ├óÔé¼┼ôbad pos├óÔé¼┬Ø stuttering.

I only discovered the issue by chance, as I used the ├óÔé¼┼ôsettings├óÔé¼┬Ø for spring to disable ALL console logging, for some reason the ├óÔé¼┼ôCan├óÔé¼Ôäót reach the destination├óÔé¼┬Ø was still displaying in the console. When I ctrl-d the offending unit, I got a big increase in FPS, and Bingo ├óÔé¼ÔÇ£ cause found!

[AF]

To make clear my point:


Cause:

?

Effects:

When an attack unit (for any/all and whatever reason) spasm “Can’t reach the destination” NTAI start to eat massive amount of CPU cycles.

Solution:

no cause found

I need to know what the unit is doing that causes the engine to give that error message. Spring is giving that error message, and I dont know what its giving the error message for. I dont like sausages. Why dont I like sausages? Whats the cause? Me not liking sausages, thats only the end result.

Pause the game. Select the unit. Look at its orders. Determine why springs not liking it.

I cant fix a bug unless I know the cause, repeating the effect/end result in bold and larger letters wont gelp me if it's already been said.

A log tells me what the cause of an error is. Thats why most bug reports are useless because they're far too ambiguous, they give an end result, an effect, and leave out 'the cause', then they continue to describe the 'effect' not the 'cause'.

[lale]

I cant fix a bug unless I know the cause, repeating the effect/end result in bold and larger letters wont gelp me if it's already been said.

Sorry, I wasn├óÔé¼Ôäót sure my post was clear enough├óÔé¼┬ª

A log tells me what the cause of an error is. Thats why most bug reports are useless because they're far too ambiguous, they give an end result, an effect, and leave out 'the cause', then they continue to describe the 'effect' not the 'cause'.

Fine, but I thought you said that the log didn├óÔé¼Ôäót help in this case?

Pause the game. Select the unit. Look at its orders. Determine why springs not liking it.

HELP , I need you to educate me on some of the fine point of spring/NTAI ├óÔé¼┼ôbehind├óÔé¼┬Ø the scene "debug-like" abilities.

I press Pause. Holding the cursor AND SHIFT-key over the unit, normally tells me were it is going. BUT in this case, no line or attack point to destination??
OK: I just discovered the path finding displaying with the ├óÔé¼┼ôB├óÔé¼┬Ø key NEAT
!!!!!!

Got these to screenshots from a sumo spamming ├óÔé¼┼ôCan├óÔé¼Ôäót reach destination├óÔé¼┬Ø
The red fork pattern in front of the sumo is blinking, when the simulation is running.

Btw. somtimes and enemy attacker approces the spamming sumo, then the sumo unsticks for a moment, (and framerate goes back to normal) destroys the attack, and start spamming again (fps drops to 3 again).

Will post more, what do you need? - please advice?




(framerate is 34, as simulation is paused, when resumed at 0.3 speed, fps drops to 3fps)

OK I also see this green patten but not so often:
(Red and white dot on radar is enemy airplanes)

Finally this is zoom out (offending sumo is in lower middle), there are no "order" lines to be seen:

[AF]

hmm thought you had to actually select the unit before shift worked.

Press f2 with the sumo selected to see if it can pass that part. If a builder unit recieves a move order failed it gets given a small area reclaim order incase its something reclaimable.

If the sumos repeatedly trying to get somewhere and its stuck then that would cause it.

[AF]

I've corrected a few things.

Most notably:

- A new geothermal placement algorithm
- A fix to prevent assemblers going after geo spots with enemy sockets.
- Removed markers
- Removed debug peewees
- Removed bad build pos (6) message
- Removed attack pos marker message
- Attack routines and b_assault/b_rand_assult now use the same attacker check so theyll all check user definitions of whats an attacker rather than relying on a dynamic algorithm that doesnt always work.

Download here

Thankyou for all the help and reports.

*Note: This build doesnt attempt to fix the above issue but when its resolved therell be another release*

[lale]

Press f2 with the sumo selected to see if it can pass that part.

OK I will make some screenshot, with "f2" display when I come home.

I do not think any "rubel" was near the sumo in shot 1 and 2, as the sumo was later attacked and moved fine, both north east,w & south of its pos.

If a builder unit recieves a move order failed it gets given a small area reclaim order incase its something reclaimable.

NEAT, but it is very rare that a builder makes a single message of "can├óÔé¼Ôäót reach destination". And if it does, this is VERY quickly cleared by (presumably) NTAI.

If the Sumos repeatedly trying to get somewhere and its stuck then that would cause it.

OK, I think I will make some screens with the small and dexterous reaper tank instead, as this unit should be more manoeuvrable than the Sumo, and yet it constantly spam's "can├óÔé¼Ôäót reach destination"

In some very rare cases, this also happens for airplanes. If I come across one, I will document (screenshot) it.

The issue also happens for maps without water, so it├óÔé¼Ôäós not solely about water and land units trying to fight each other.

[DJ]

glad it's not just me who had these problems, I gave up on my config over a month ago (back on april 22nd) because when i reported this issue nobody paid any attention. Once this is fixed NTai is gunna be kick ass. Even the config i had which was quite simple wiped the floor with the other AI's.

[lale]

OK AF, i've got some screenshots for you, documenting the issue.

The shots are take from a relativ flat map, no water, but with a trench system.
All, except the first cons. unit, resulted in extrem drops of fps. "Unit Slow Update" and "Global AI" gets high, around 20% during these problems.

(Funny: The percentage numbers in performance "box" never adds to more than 80%)

In order NOT to fill up this thread, I made a new thread with the screens:

http://spring.clan-sy.com/phpbb/viewtop ... 059#191059

I also got a logfile coresponding to the images, if your interessted.

Tell me what do you need more:?:

[zwzsg]

Thanks for the release AF. However, I just noticed a little problem:
- NTai build socket/window inside its own kernel/hole

This raise the following issues:
- Doesn't look pretty to have minifac stacked onto bigfac.
- Prevent the kernel from functionning, crippling the AI
- Makes the AI invicible in my new ONS mutator

(Yes, I know it's an engine bug, but, still...)

[AF]

fixed that problem

[1v0ry_k1ng]

besides the crash problem, my NTAI runs smoothly. im just waiting for a scouter/mex hunter setting, intelligent defence placement and microed attackers ^^

[AF]

Once 0.75 is released, I'll not be adding any major new features save those that I've disabled in hopes of fixing.

I will however do the following:

Implement an IModule proxy that when called passes control through the lua<->AI interface to the mod lua scripts. It will be the modders responsibility to implement their interfaces and their mod specific keywords and share what they've done with other config builders and other modders. mod A has unit morphing? Tell toolkit there's a lua keyword called lua_morph and add it to the task list then implement the keyword in lua. You want a special attack routine such as the mex scouting/raiding? Implement a lua_raid keyword that implements that task.

Tasks in NTai are not one off events. The new task system allowed for tasks to carry on indefinately. As soon as a task is flagged as invalid it'll be deleted and NTai will move to the next task.

I'll also implement a task manager module that can be pushed through lua, with the default behaviour being to load the tasklist made by toolkit and add it. This way you could implement dynamic task managers for AAI/RAI style economics rather than tasklist based economics, or add specific clauses to tasks, or make it so the task lists arent randomly picked but the first corck gets the first corck task list etc...

Those two extensions should make the AI extremely customizable.

[1v0ry_k1ng]

THE FOUR THINGS NTAI NEEDS, a paper by sir proffesor 1v0ry

1) SCOUTER TAG
a tag which causes the unit to be send on its own to target an enemy mex. when the mex is destroyed, it targets another mex. when there are no mex, it returns home. it takes no part in the attack grouping system.

this allows raiders and harassment.

2) MODIFABLE ATTACK GROUPS
an option to define:
a) the initial attack group
b) the change size of the next attack group, with the option of % AND addition. ie, increase by 10% and add 2.

this allows compensation for the way that economies in spring grow expodentially.

3) WORKING TAGS:
singlebuild
solobuild
guardian
gaurdain mobiles

4) A LOOP TAG, GAURD FACTROY NOT TO APPLY TO AN IDLE FACTROY, .

technically 5

[lale]

a tag which causes the unit to be send on its own to target an enemy mex. when the mex is destroyed, it targets another mex. when there are no mex, it returns home. it takes no part in the attack grouping system.

I've found NTAI to be good at taking out mexes, along the way to attack pos.
But I think it├óÔé¼Ôäós important that the mexes should be choice at random, every time a new scout is tasked.

2) MODIFABLE ATTACK GROUPS
an option to define:
a) the initial attack group
b) the change size of the next attack group, with the option of % AND addition. ie, increase by 10% and add 2.

So by the time NTAI techs to krogoth, it needs to build 20, before attacking
This is not and easy solution.
In the current build of NTAI retreating (due to damage) units get back to base, are repaired, and then attack again, so every attack size grows in strength. It works, just make constructors do and B_offensive_repair_retreat.

3) WORKING TAGS:
singlebuild
solobuild
guardian
gaurdain mobiles

OK, I need to BITE you on this one.
What├óÔé¼Ôäós your beef with Singlebuild
Would you please elaborate ├óÔé¼┼ôworking├óÔé¼┬Ø singlebuild?
- as I find Singlebuild to be working very well!

4) A LOOP TAG,

Huh?
All factories and constructors loop the list; allways.
If your lobbying for NON looping constructors & factories. You really have to elaborate, as I se no point in having idle constructors standing around??
And any other thing I might do with a non-loop, can be recreated within Max and Min energy section of config file.

GAURD FACTROY NOT TO APPLY TO AN IDLE FACTROY, .

Agreed however, I├óÔé¼Ôäóve noticed that b_repair, assists factories, but not idle ones. (It does however also help nearby constructors.)

[lale]

My short list of things to improve in future versions:

NTAI behaviour adjusted according to map type (metal, water, waterland, land, flat, rough (rough = filled with cliffs / steep hills)
This could be done two ways:
Let NTAI have configfile per maptype: eg ├óÔé¼┼ôBA-waterland.tdf├óÔé¼┬Ø or ├óÔé¼┼ôBA-metal.tdf├óÔé¼┬Ø
OR
Have B_RULE, B_FACTORY etc. keywords prioritize according to maptype. (e.g. b_mex primarely builds uwmex on watermaps)

Solobuild with better efficiency.

ConstructionExclusionRanges, working.
(btw: haven├óÔé¼Ôäót test if this works in newest build)

And subs, not targeting land units. (Water units should all have a check, to find out if they can reach attack pos or attack target, otherwise they should retarget.)

[AF]

You'll have to provide a little help when it comes to what can target what, I dont know how to get that information.

He wants a loop keyword because b_guardian stops when it hasnt got anything left to repair, which is silly as the unit would just sit and do nothing, consuming extra cpu cycles. Besides the way the task system is setup it'd be a nightmare to implement in the current setup. Too much work. Itd be best waiting for lua extensions and using a while(true) loop.

Also, define hilly/steep/cliff. How is water different from waterland? These are things I cant do in any specific way. It would also require too much work. I have a lot of projects and right now NTai development follows the line of 'if its not broken dont touch it'. As I said IK, how do you know b_guardian isnt working if you cant tell when the task is executed? offensive repair and retreat wont work 90% of the time because its not applicable to all situations. e.g. comamnder gets offensive repair, nearest unit is other side of map, it didnt repair a unit because there wasnt one nearby, oh noes it mustnt work.

Right now this is the following checklist for what I consider a good config:

- Beats the competition
- No humonguous bugs or flaws (it built a lvl 2 factory but the factory has no tasklist huh!?!??!)