Page 74 of 95

Posted: 14 Sep 2007, 13:01
by AF
Thats a heck fo a lot fo work for a custom profiler. The NLOG define statements arent at the start fo every single function call and it would require me to set up the custom profiler to take into account all sorts of things. Far too much work.

It would be easier to just refactor and rewrite things until the problem vanishes.

Posted: 15 Sep 2007, 11:25
by Tobi
Search replace { with {PROFILE_POINT , make sure not to replace it for type definitions.

Code: Select all

#define PROFILE_POINT NLOG(__PRETTY_FUNCTION__, __FILE__, __LINE__)
then grep functionname logfile | wc -l and somehow combine the if blocks within a single function (by grepping for functionname and linenumber?) :P

it's probably more useful though to get gprof working one way or another, reinventing this kind of (profiler) wheel seems pretty much a waste of time ;-)

Posted: 15 Sep 2007, 13:02
by AF
And that compiles under VS2005?

I'm not sure that would be of any use without then falling into the trap of havign 60 thousand instances of that string in the entire project for every single pair of squigly brackets including ones that're just there for boost::mutex::scoped_lock

That and there's no grep command under windows and msys is broken for Vista x64, and scons wont work at all.

imbaczek tried profiling but I had the stuff setup to always produce 0.75b2 compatible AIs, now I've changed that it might work.

Posted: 15 Sep 2007, 13:18
by imbaczek
I tried a mingw build, but while it worked, it didn't generate any data.

Total commander has a pretty good grep in it's search tool, also codeblocks has find in files (not sure if it has replace.)

Posted: 15 Sep 2007, 13:21
by Tobi
First hit on google for gprof shared object wrote: In 32-bit profiling, shared objects cannot be profiled with gprof.
So you'll have to compile it with Spring into a single executable, or maybe use 64 bit profiling.

And also make sure you look for gmon.out in the correct directory. Since Spring calls chdir() it may end up in the Spring data directory, not in the current directory, and confusingly enough it does output an empty gmon.out in the directory from which you started the program. (applies only on Linux)

Posted: 15 Sep 2007, 13:25
by AF
Its still thousands of lines of code, and on top of that there are lots of places were functions are nested at least once.

At minimum all functions are nested inside a CNTai function and a Global function.

And I do have several active projects.

And logs using the existing NLOG identifiers generate so much data that its relatively pointless. Even if I did get all the statements workign and in place its far too much data to interpret.

So unless one of you guys wants to build the custom compiler from scratch for me its not happening.

Posted: 15 Sep 2007, 13:37
by Tobi
It was just an idea since you tend to exclude the more reasonable alternatives like getting the right tool for the job (working):

1) get linux + gcc + gprof + some glue code to link NTAI to Spring statically
2) get money and buy a profiler from MS if you insist on using Vista
3) get time and make gprof working on mingw/cygwin

(There's a reason I still dualboot and never switched to either Linux or Windows completely...)

But well, I fear in the end someone else will have to do it for you anyway...

Posted: 15 Sep 2007, 17:20
by AF
*_* I have to statically link it to spring?

I don't know how to compile hello world via command line under windows nm linux, and Im not exactly rich with regards to spare time.

And I do have ubuntu and XP on my old machine which is a paint to use but that doesn't necessarily mean I know how to use linux. Most of my *nix command line skills are using sudo apt-get type commands I googled in firefox, the rest are basic commands like cd ls sudo su chown or chmod. All the rest fo my linux abilities are GUI based.

And meanwhile VS2005 Team suite appears to be thousands of punds
Possibly £2000+ some editions are £8000+

An MSDN subscription including VS2005 Team suite costs £8882

So right now my options are:
- Save up thousands of pounds and buy Team suite edition of VS2005
- Abandon my projects an spend the time learning to use linux, or learn linux stuff in my spare time and fix NTai in a few months time.
- Have somebody else do it
- rewrite code un till I just happen to rewrite or remove the offending code.

I dont understand why its so hard for users to profile libraries. I can easily retrieve a profiler that gives nice graphs and stats for any java program or library, and it appears that all the VS2005 profilers run fine as long as your profiling .Net assemblies or managed c++ CLR code. Even code blocks has a profiling option but they all disable or break as soon as you mention C++ dll.

Posted: 15 Sep 2007, 17:33
by AF
Still I wonder what results IKs experiment would give.

Posted: 15 Sep 2007, 17:55
by AF

Posted: 15 Sep 2007, 19:47
by Tobi
That's because basically every moron* can write a .NET profiler while a C++ profiler is much harder to make.

In .NET you can "just" make your profiler load the assembly, loop through all instructions of all methods of all classes and emit profiling instructions at certain points in between, all using the reflection API. Then run the instrumented code and show the results to the user and voila, you got a profiler.

OTOH, in C++ you need to either loop through machine code instructions BY HAND (e.g. valgrind), there is no easy API for that built into C++, or you need to extend a compiler to emit profiling instructions (e.g. gcc with -pg option in combination with gprof).

Extending a compiler is almost per definition harder then messing around a bit with built-in APIs of some programming language, so we can conclude it's an order of magnitude easier to make a .NET profiler then it is to make a C++ profiler.

I presume something similar applies to java.

*In practice you may need to have some more skills then a total moron has :-)

Posted: 17 Sep 2007, 18:54
by DJ
new version crashes spring as soon as it builds a mex, think its the unit finished command that causes it. The offending piece of code is

Code: Select all

BPlans->erase(i);
However I can't find where this method is implemented, removing the call stops the crash but NTai crashes shortly after in the the unitIdle method.

Posted: 17 Sep 2007, 20:57
by AF
I believe that's you'll find the implementation of erase() in the standard template library.

Code: Select all

deque<CBPlan>* BPlans;
I assume your refering to:

Code: Select all

void CManufacturer::UnitFinished(int uid){
	NLOG("CManufacturer::UnitFinished");
	if(initialized == false){
		G->L.print("CManufacturer::UnitFinished Initialized == false");
		return;
	}

	if(BPlans->empty() == false){
		for(deque<CBPlan>::iterator i = BPlans->begin(); i != BPlans->end(); ++i){
			if(i->subject == uid){
				if(!i->inFactory){
					G->Actions->ScheduleIdle(uid);
				}
				BPlans->erase(i);
				break;
			}
		}
	}
That deque container is shared by all the NTais on that system, hence why its a pointer. I think the planning system could do with an overhaul, Im not quite sure how I'd go about it though.

It's feasable that you could remove the plan system but that could lead to organisation issues, such as units trying to build things over the same position then arriving to find its not possible because another builder has already started a construction.

Plans also provides the means by which a builder can tell what its building and a building can tell who its being built by.

Perhaps its crashing because a second NTai is invaliding the iterator causing the first NTai to crash?

Posted: 18 Sep 2007, 10:29
by DJ
i always test with just one NTai in the game, especially where threads are concerned. The bug is easy to reproduce, just create a task list with b_mex at the start.

Posted: 22 Sep 2007, 20:50
by AF
My next version has a new behaviour system that adds to the new task system. At the moment I have 3 options, attack, metalmaker, and auto, with auto being the default.

auto - backwards compatibility. Auto assigns behaviours based on the untis type.

metalmaker - switches the unit on or off based on the energy available

attack - gives the unit the attack routines that make it seek out and attack enemies that come too close.

There'll also be cloak which will assign the cloaking code I commented out. This should allow me to delete the ancient Assigner class, a class older than even the chaser class which has been rewritten several times. Assigner class still has the same structure and logic as NTai 0.23 released in 2005.

Posted: 24 Sep 2007, 08:31
by AF
I've committed a lot of the above mentioned changes and a miriad of fixes relating to threading mutexes and boost::shared_ptr referencing in combination with std containers. I've also cleaned up a lot of random things.

The bug with mexes you mention was an STL container expanding destroying the module in the process, probably caused by the creation of a new CUnit object and its addition.

I've gotten the two new behaviour classes in game but I have yet to verify that they're actually functioning. First indications suggest they aren't.

Posted: 03 Oct 2007, 13:13
by Smiff
sorry if this is mentioned before, but i tracked down a desync error in spring to this bot.
when running this bot (and only this bot, others are fine) on my server, i see predictable "sync error in frame x" (first is always frame 17) messages on one particular PC on my network (happens to be the slowest, co-incidence?).

I am not sure if this error is supposed to be fatal, but after a while the game seems to go out of sync, which never happens to me otherwise.

therefore i have stopped using this bot in LAN games, which is a shame as it seems quite smart.

Posted: 03 Oct 2007, 13:17
by AF
Bots shouldnt be capable of causing desync.

Posted: 03 Oct 2007, 13:21
by Smiff
AF wrote:Bots shouldnt be capable of causing desync.
i don't know what exactly is causing the desync, sorry im just a user :/
but i am 100% sure that i get sync errors while running this bot and not with any other. whether this is due to that PC being too slow, a bug in spring or whatnot i have no idea.. if you want me try things i can.

Posted: 03 Oct 2007, 13:24
by AF
If you're running them and they're giving a lot of lag then that can cause slowdown induced sync errors.

The builds of NTai in this thread as of late for 0.75b2 shouldnt be used on older systems and are best suited to multicore cpus.