Feature request: Remove unit cheat and/or Reset/restart game

[allanmc]

We are developing an AI with RL (Reinforcement Learning). When using RL we need to train the AI throughout several thousand games.

For this purpose we use the headless spring, which works very well. However, it is a problem that you can't directly reset a game from the AI. This means that you need to do a complete restart & reload of the game for every match. This loading takes ~13 seconds, which is alot compared to the actual gametime of 5-10 seconds at speed 120. Waiting the ~13 seconds for every single match is a huge waste of time.

As an attempt to solve this, we self-destruct all our units and add a new commander (using cheat) at the start position. This way we can somewhat start a new game, but it is an ugly approach. It still has the problem that a self-destruct explosion destroys the ground underneath, slowly revealing water and making the area unusable.

So now to our actual feature request Why not either add a remove-unit cheat (without explosion and countdown), or even better add a possibility to reset/restart the game, without having to reload all the spring resrouces again. Such a reset/restart would simply reset the map and reload the AI's.

[imbaczek]

add a luarules gadget to the mod you're doing your training on - this will be the simplest solution for now.

[Tobi]

Also, with regards to the map getting damaged, you can use the engine option NoMapDamage to disable map damage altogether.

[allanmc]

Thanks for the tips!

I am not familiar with the posibilities of LuaRules in Spring, so what exactly would be possible with such a script?

[Umrug]

You have to edit the mod file itself. (Unzip it with 7z, and zip it back with 7z using normal compression mode). Look at luarules\gadgets inside the mod, Those are lua scripts that are, how do I put this, unlimited in their power to change game state.

[Error323]

Hi allanmc,

As I am a student A.I. I would really like to know some of the details of the RL approach like:

• How did you define the states?
• What actions have you got?
• What is the reward function?
• What RL algorithm are you using specifically? (Q-Learning, Sarsa,...)

I have been thinking about an RL approach for quite some time, but so far I'm quite sceptic about it as the state-action space is too large or too abstract for guaranteed convergance.

Very curious!

[allanmc]

• How did you define the states?
• What actions have you got?
• What is the reward function?
• What RL algorithm are you using specifically? (Q-Learning, Sarsa,...)

Hi Error,

This AI is part of a Master Thesis in Software Engineering, which i am doing along with three other people (on this forum: allanmc, initram, jepperc and shredguitar).

Currently we are trying to use BN to classify the opponent, and RL only to build the base (reach many labs as quickly as possible). In this simple RL we just have 3 build actions: Lab, Solar, Mex. The counts of these are used as state variables - Lab and solar can be in the range 0-19, and Lab in the range 0-4. The time used to build any of these buildings is used as negative reward, which ensures that the AI eventually figures out that it needs resources to build quickly. The goal state is reached when 4 labs has been build, and this results in a high positive reward. This has been implemented with standard Q-Learning, and works very well.

Right now we are in the middle of expanding the use of RL to more usefull cases, such as complete base building, attacking, scouting, etc.. You are right that with these bigger problems, the state space, among other things, can become a problem. Therefore we are currently looking into the possibilities with hierarchical RL: http://www.ijcai.org/papers/1552.pdf

[Pxtl]

You also should probably be culling the wrecks between restarts. All the unreclaimed stuff that existed before the big Self-D is still going to be sitting there.

Got a screenshot of the 100th game? I want to see what that map looks like after that much mayhem.

[AF]

features too, you want a map with little in the way of trees and metal rocks