SpringGrid

[aegis]

this should probably run ok also on a netboot environment, as long as each node has enough memory.

duh? :D
that's what I was using.

also, @LuaAutoEnableUserWidgets, I thought I changed that in the patch I gave to hoijui... also thought he added headless_setup.lua
huh.

[hughperkins]

also, @LuaAutoEnableUserWidgets, I thought I changed that in the patch I gave to hoijui... also thought he added headless_setup.lua
huh.

Well, sometimes things slip through the net. Or maybe they are hidden in the git aether somewhere. Are there any other changes that you made that are not in current springheadless branch? http://github.com/hughperkins/springheadless.

[hoijui]

yeah sorry.. there were 3 values in the enforced config i got from aegis i did not submit:

Code: Select all

	configHandler->SetOverride("LuaAutoEnableUserWidgets", 0); // not sure about this one?
	configHandler->SetOverride("LuaModUICtrl", 0); // or this one
	configHandler->SetOverride("LuaUI", 0); // or this one

he had them all on 1, but it was unsure if it was needed..
maybe if the widget works now, you dont have to set the others to 1

i dont remember what happend back then, .. i think it was somethign like.. aegis wanted to do an other widget, that handles general widget management on headles spring or something, and i wanted to commit them together.

[hughperkins]

AF,

You may or may not be pleased to know that it looks like your AI is now compatible with SpringGrid, and there are a bunch of matches running now between NTai, AAI, KAIK, RAI and E323AI.

http://manageddreams.com/ailaddergrid/s ... ircount.py

Before you complain that E323AI is a cheating AI, yes there are lots of possible tweaks that can be made to springgrid/ailadder...

If you update the source-code through the website (you'll need to log into http://manageddreams.com/springgridstaging at least once with an OpenID, then I can assign you rights to do that), *please* make sure that the name and version correspond *exactly* to what is in the AIInfo.lua file, which is not currently sanity checked, so any errors basically break stuff

Obviously, these are both things that can be tweaked in the future in the botrunner code, but they haven't been yet. Few things are perfect...

[AF]

Do you have configuration files etc setup and verified that they're running and NTai isn't just a commander sitting idle?

[hughperkins]

Do you have configuration files etc setup and verified that they're running and NTai isn't just a commander sitting idle?

No. You are responsible for making sure that the source-code package pointed to by the AI url will install appropriate config files.

You can install multiple config file versions by creating a new source-code package and registering it with a new version name
- please make sure that the version in AIInfo.lua corresponds *exactly* with the registered version name!

I have no short-term plans for providing any other ways of configuring configs, because I feel the current method:
- works
- is relatively simple
- is relatively flexible
- gives you a lot of power over exactly what configs you want where, because you control the installation step of your source code

[hughperkins]

Hmmm, the E323ai vs E323ai v2.83 matches are sllloooowwww. I dont fancy paying 6p per E323ai match. I might add a faster game timeout, and E323 can either run his own botrunner, with a longer timeout, or make his AI run a bit faster. Maybe a later version of e323ai does run faster anyway?

Edit: or maybe it is the botrunner that is slow I guess. I might shutdown the botrunner node and reconsider some additional diagnostics implementation such as game elapsed time and so on.

Edit: plus, everything is coming up as a crash. Grrrr :-O Must be a bug in botrunner causing that. Back to the development board... :-/

[AF]

How do I do that? I'm not even sure how the current source package your using is assembled or where to even locate it.

[hughperkins]

Sure. Ok, so the build process that the botrunner does is essentially:
- unzip into the spring build tree, underneath AI/Skirmish
- run standard spring 'make' then 'make install'

Actually, it doesn't run exactly 'make' but 'make <name of AI>', because that's faster. Also, it doesn't run exactly 'make install', but 'cmake -P AI/Skirmish/<name of AI>/cmake_install.cmake', again because it's faster.

But you can assume and test that it does 'make' then 'make install'.

Also, it doesn't do exactly unzip the tar file into AI/Skirmish. What it does is to untar it to a temporary directory, and search the directory tree for a file called 'VERSION". when it finds that it copies the contents of htat directory, including subdirectories, to AI/Skirmish/<ai name>.

What it *should* do is check for data/AIInfo.lua, instead of VERSION, and it probably will do in the future, but I made a mistake and looked for the VERION file instead

Sooo... to prepare your sourcecode:
- make sure it builds and isntalls correctly inside the spring build tree
- tar.bz2 the AI's directory underneath AI/Skirmish
- make sure that the name and version in data/AIInfo.lua are correct because whilst the botruner doesnt directly use these during compilation, unitsync does, and this will be used to subscribe the ai to hte website.

[AF]

Unnecessarily high barrier to entry is unnecessarily high!!!

tar.bz2/thar be no linux here

I'm a windows user, none of the tools I use can generate tar.bz2 archives.

It would be much easier to put stuff in a zip with winrar/7zip, as most users building with visual studio who run with windows, may never have needed to build such an archive, and wouldn't know how. A google search results in lots of answers, all of which are linux specific. We aren't all linux users!!!!!!

cmake

I do not have mingw32 or any version of make or cmake installed, I haven't the time ( or the will) to setup and maintain an install of mingw32 on my Vista machine, and I use Visual Studio to develop my AI.

The contents of the archive to be submitted

I still do not know what the equivalent NTai.tar.bz2 would actually look like when unzipped.

All your post tells me is that I can expect the root folder to contain an NTai/XE10/ directory with an AIInfo.lua and probably a configure and make file in the root directory.

I also do not know if by AI/SkirmishAI you are referring to in my context c:\springinstall\AI\SkirmishAI or My Documents\Dev\SpringSrc\AI\SkirmishAI

I'm quite annoyed that the ladder has turned into a humongous cloud initiative that requires me to learn all this new stuff and submit source code using a build system I do not officially support (supported = is(VisualStudio);).

The Cloud

There is no need for any of this anti-cheating stuff, cloud stuff, virtual box. It's surplus to requirements.

When you have basic ladder listings, I'll be able to run my machine knowing I am enduring my parents power bill wrath for a cause and not so that I can have my machine used as an exercise in cloud computing.

Closing

I'm sorry if I'm sounding horribly negative, and Im not insulting or trolling you, but to me this project has been horribly and vastly overcomplicated unnecessarily, and it still hasn't managed to yield any useful results.

We have no AI ladder listings, we cant even use this for automated AI training, all we have is a fancy way of launching AI battles on a remote system. So far even the NTai battles recorded show no evidence that NTai was ever intialized or loaded.

So can you understand why I might be unsatisfied and critical?

[hoijui]

AF.. just.. shut up!
that just had to be said about this post.

the cloud thing is there cause hugh has a netbook only, and he explained 2 times in the last few posts, that you do not need that stuff, and he explained it in dummy language.

about the tar.bz2 thing, why not just say: "Please make it posible to use zip instead of tar.bz2, as it is easier for windows users"
with words like this, you would not have to insist your post is not trolling or insulting, cause it actually would not be so anymore.

about you not willing to learn how the spring default build system works:
either learn it (you can do that by looking at the code, its pretty simple), or ask for an alternative to upload binary AIs (could be a setting that is default off, to not cause security issues). for this option, please reffer to the last paragraph of this post as well.

"We have no ..."
everyone else that i know of coding an AI for spring, could use it as it is already.

closing
smoke some weed from time to time?

[AF]

They can use it to do what it does, but it doesn't really do anything yet =/

I've also made clear that the cloud part is unnecessary and that once I can justify keeping my machine on as a botrunner, it will be kept on as a bot runner.

Also I feel the need to place greater significance on things involving windows because here in the AI forum we have arrived at our own version of the 'just use lua' ignorance that plagues the other forums, aka the 'just use the default Linux way', which in my opinion is incredibly patronising, and causes damage by neglecting academia who would want to use visual studio and other setups.

[hoijui]

They can use it to do what it does, but it doesn't really do anything yet =/

I refuse to belive that you know what id does, as you dont even know how to use it. It does stuff already, and even doh it does probably not yet have all the statistics you want, it already could be used for learning AIs -> see first line of my last post.

I've also made clear that the cloud part is unnecessary and that once I can justify keeping my machine on as a botrunner, it will be kept on as a bot runner.

Which would be relevant if you were the only targeted user and the only dev of SpringGrid.

Also I feel the need to place greater significance on things involving windows because here in the AI forum we have arrived at our own version of the 'just use lua' ignorance that plagues the other forums, aka the 'just use the default Linux way', which in my opinion is incredibly patronising, and causes damage by neglecting academia who would want to use visual studio and other setups.

Your feeling is wrong there, it is cuases actions that are contraproductive to its own wants -> try to supress it.

[AF]

Please explain how I would retrieve the learning files and output of an AI running on the cloud without forcing it to be dumped into infolog.txt or implementing a network stack?

There is also a difference between not going all out for maximum support, and a blanket dismissal of everything that is not the default build system. Also you should remember by now that I have to use spring and my AI on machines for which I cannot install or use the default build system.

[hoijui]

again, you dont have to use a cloud, and to retrieve, you had to make your AI send them to you.

i told you in the last post, you can do learn how the default build system works just by loking at the code, and it is really simple. note, you dont have to install something (except you dont have notepad) to look at code.
an other tip: NTai is already built with CMake in the main repo, so you could look at that code for even less troublesome setup (possibly up to the point of simple copy paste).

[AF]

If you read back, you'll note that my criticisms weren't centered around how do I setup cmake, I know that, normally ./configure, make, make install suffices, and Im not the one issuing those commands so its besides the point, but I still don't know what format to put the contents of the zip or tar.bz2 in, and its assumed I do.

[Tobi]

Maybe read this post a few more times. For me it answers all questions you come up with later in the thread.

For making tar.bz2 archives just get msysgit and use `tar cfj bla.tar.bz2 my/folder'.

Also note that what you describe in your last post is how you use an autotools based build system, not how you use cmake. (hint: it doesn't have ./configure)

[AF]

That still doesn't tell me where my configuration files would go =/ am I just to dump it all in the data folder? My AI folder sits inside another folder inside another folder inside the normal folder, do I include that file structure in the archive? or do I go to the folder that sits in the same folder as my VS oroject and bundle that?

[hughperkins]

That still doesn't tell me where my configuration files would go =/ am I just to dump it all in the data folder?

Yes, I feel that that sounds like exactly what could be an appropriate implementation.

My AI folder sits inside another folder inside another folder inside the normal folder, do I include that file structure in the archive? or do I go to the folder that sits in the same folder as my VS oroject and bundle that?

Soo.... leaving aside what we mean by 'spring source' for now, let's say you have a direcctory springsource, and I hate typing \s so just pretend they are / ok Oh, fine have some \s :

Code: Select all

c:\springsource

In this directory we have:

Code: Select all

c:\springsource
   \rts
   \game
   \AI
      \Skimish
          \AAI
          \KAIK
          \RAI

Add your directory next to RAI:

Code: Select all

c:\springsource
   \rts
   \game
   \AI
      \Skimish
          \AAI
          \KAIK
          \RAI
          \NTai
              VERSION
              blah.c
              foo.c
              \data
                  AIInfo.lua
                  AIOptions.lua

Add in a file called 'CMakeLists.txt', which you can copy from KAIK, RAI, or AAI. They're all identical:

Code: Select all

c:\springsource
   \rts
   \game
   \AI
      \Skimish
          \AAI
          \KAIK
          \RAI
          \NTai
              VERSION
              blah.c
              foo.c
              CMakeLists.txt
              \data
                  AIInfo.lua
                  AIOptions.lua

Add in your configuration data:

Code: Select all

c:\springsource
   \rts
   \game
   \AI
      \Skimish
          \AAI
          \KAIK
          \RAI
          \NTai
              VERSION
              blah.c
              foo.c
              CMakeLists.txt
              \data
                  AIInfo.lua
                  AIOptions.lua
                  ntai_config1.xml
                  ntai_config2.xml
                  ...

Publishing to botrunner

For publishing for botrunner, simply tar.bz2, or .tar.gz, or just .tar, the NTai directory, so the archive file will contain:

Code: Select all

 \NTai
     VERSION
     blah.c
     foo.c
     CMakeLists.txt
     \data
         AIInfo.lua
         AIOptions.lua
         ntai_config1.xml
         ntai_config2.xml
         ...

Building

cmake

cmake is a meta-buildsystem: it generates build files for other build systems. Not 'it can', but 'it does': it is incapable of actually building anything directly itself, on any platform (I think? )

cmake is cross-platform. It can generate build files for ... too many to list here... take a look: http://www.cmake.org/cmake/help/cmake-2 ... Generators The list includes:

# MSYS Makefiles
# MinGW Makefiles
# NMake Makefiles
# Unix Makefiles
# Visual Studio 10
# Visual Studio 10 Win64
# Visual Studio 6
# Visual Studio 7
# Visual Studio 7 .NET 2003
# Visual Studio 8 2005
# Visual Studio 8 2005 Win64
# Visual Studio 9 2008
# Visual Studio 9 2008 Win64

Now whether our actual spring build system will pump out brilliant build files for all of those is another question, since I'm going to guess that some of the code will need tweaking to build in different systems, but in this case we're only discussing NTai, so 100% of the code in your AI is 100% under your control, so you can make sure that it does build nicely under visual studio.

cmake will pump out build files and if you ask it nicely it will pump out Visual Studio build files, and if your code will compile in visual studio, these build files will probably work just fine.

Not including spring itself, but that is out of scope. I assume you've figured out already how to get hold of a spring build that is compatible with your visual studio built AIs?

Getting cmake to spit out build files

Run cmake-gui, and in cmake-gui:
- set the 'source directory' to c:\springsource
- set the 'build directory' to c:\springsource\build

cmake-gui is I feel rather nice, and you can get it from

http://www.cmake.org/files/v2.6/cmake-2 ... 32-x86.exe

It's a Windows application, in a normal Windows window. No need for any cygwin stuff or anything like that. Just a normal Windows executable AFAIK. Fairly light-weight I think.

- click on 'configure'
- tweak the default build configuration if you wish
- click on 'configure' again
- click on 'generate', choose the build system you want to generate for
-- this is where you can try selecting other build systems. I bet there could be an error or two if you select visual studio as an output, so you might have to fix those errors, and commit those fixes to git.
- your build files should pop out into the c:\springsource\build directory

Building

Open visual studio, load the appropriate buildfile. Try building, fix the errors, repeat until it builds ok, commit the fixes to git.

What happens to the configuration files?

By default, the generic CMakeLists you copied into NTai will copy *everything* in the 'data' subdirectory into the root of your installed AI, so your AI when it is installed will look like this, and let's say you configured it to install everything into c:\spring:

Code: Select all

c:
   \spring
       \AI
           \Interface
           \Skirmish
               \AAI
               \KAIK
               \NTai
                   \XE10
                        SkirmishAI.dll
                        AIInfo.lua
                        AIOptions.lua
                        ntai_config1.xml
                        ntai_config2.xml
                        ...

Detailed notes:
- the exact name 'NTai' will be whatever was in data\AIInfo.lua file when you ran the configure
- the exact version 'XE10' will be whatever was in data\AIInfo.lua file when you ran the configure

Looking again at the configuration, you're free to put the config files in whatever subdirectory you want by just creating appropriate subdirectories in data.

For example, if you want the final installation structure to look like:

Code: Select all

c:
   \spring
       \AI
           \Interface
           \Skirmish
               \AAI
               \KAIK
               \NTai
                   \XE10
                        SkirmishAI.dll
                        AIInfo.lua
                        AIOptions.lua
                        \config
                           ntai_config1.xml
                           ntai_config2.xml
                           ...

...then simply, in the source-code directory, move the config files to a subdirectory 'config' of 'data':

Code: Select all

 \NTai
     VERSION
     blah.c
     foo.c
     CMakeLists.txt
     \data
         AIInfo.lua
         AIOptions.lua
         \config
            ntai_config1.xml
            ntai_config2.xml
            ...

... presto! by magic, ntai_config1.xml and ntai_config2.xml will end up in a subdirectory 'config' of the installed AI:

Code: Select all

c:
   \spring
       \AI
           \Interface
           \Skirmish
               \AAI
               \KAIK
               \NTai
                   \XE10
                        SkirmishAI.dll
                        AIInfo.lua
                        AIOptions.lua
                        \config
                           ntai_config1.xml
                           ntai_config2.xml
                           ...

Conclusion

... none of which addresses the issue of how to pump learning data back upstream, which is admittedly a challenging problem.

Nevertheless this post does attempt to address:
- how to install config files
- how to get the config files you want to end up where you want
- how to use cmake on Windows
- that cmake is not I feel a build system, but is I feel a meta build system, that generates build files
- that cmake is capable of generating build files for visual studio

Let's solve these issues first, and then we can think about addressing the learning data issues, ok?

[hughperkins]

Update to git:
- can use postgres now (tested with 8.4)
-- mysql still works