callback: GetFileSize & ReadFile

[Reth]

Lately I have been trying to track down any inefficient code slowing the loading time of my AI. Unexpectedly, I narrowed down a certain section of code to the two callback functions GetFileSize() & ReadFile().

Code: Select all

	...
	string mapFile = cb->GetMapName();
	mapFile = "maps\\"+mapFile.substr(0,int(mapFile.size())-4)+".smd";

	double mapTimer = clock();
	int mapFileSize = cb->GetFileSize(mapFile.c_str());
	*l<<"\n Code Timer 1: "<<(clock()-mapTimer)/CLOCKS_PER_SEC<<" seconds";

    if( mapFileSize > 0 )
	{
		*l<<"\n  Searching the Map File: '"<<mapFile<<"'  File Size: "<<mapFileSize;
		char *buffer = new char[mapFileSize];

		mapTimer = clock();
		cb->ReadFile(mapFile.c_str(),buffer,mapFileSize);
		*l<<"\n Code Timer 2: "<<(clock()-mapTimer)/CLOCKS_PER_SEC<<" seconds";
		...

Below are the log entries produced for two paticular 24x24 size maps.

Code: Select all

 Code Timer 1: 3.02 seconds
  Searching the Map File: 'maps\TheAbyss.smd'  File Size: 4554
 Code Timer 2: 2.98 seconds

Code: Select all

 Code Timer 1: 6.47 seconds
  Searching the Map File: 'maps\Total_Destruction.smd'  File Size: 2518
 Code Timer 2: 6.40 seconds

Keeping in mind, My computer: dual AMD Athlon 2.5GHz (rated 5Ghz)

This problem seems to be due to the fact that some maps are formated in a way that all or a large number of the files within the archive are a part of the same 'block' and the callback functions simply re-extracts the file for each call. While opening these particular map archives with 7-zip I found that it really just takes some 3,7 seconds respectively to extract any of the map files within them. Does anyone know of a way that I could get around this problem?

[AF]

What exactly are you trying to do with what you've read from the map? There's nothing I can think of that you could extract from a map that spring doesn't already expose except maybe tags in the smd files at which point lua is replacing that iirc anyway.

Nonetheless a speed up here could well speed up loading times

[det]

I thought .sd7 (7zip) compressed each file individually, so this doesnt make any sense. In any case, you can try using .sdz (zip), which I am certain compresses each file individually. Also, it uses a block-size of 32k for each file.

[Peet]

I thought .sd7 (7zip) compressed each file individually, so this doesnt make any sense.

It does, unless it is a solid archive.

[Reth]

The code is just to detect if the "WaterDamage" property is set under the [WATER] tag, the code works perfectly fine as it is, so, it's just an issue of speed.

How people compress their maps really isn't up to me, so I'm guessing that these functions really just weren't meant to be used with map files. Obviously this isn't a problem on all maps, about 2/3 of the maps will run through the code in a flash.

[Kloot]

Does anyone know of a way that I could get around this problem?

Write the .smd contents to a file and read that (instead of the map
archive) when the same map is loaded again? They're just a couple
of kilobytes in size anyway.