Multithreading

[Cabbage]

Hello chaps,

When i last played there was a (very sketchy) multithread.exe i believe some people were using. I' ve had a quick look around but cant find anything:

A.) Is this still avaiable / does it always crash and;

B.) Where can i get it?

Cheers!

[smoth]

should come with the spring install.. springmt or something like that

not crashy from what I heard but very hang-y(not a word I know) wrt lua stuffs.

[Cabbage]

Cracking, found it.

[Google_Frog]

Be careful it prettymuch only works with BA's default lua from what I hear.

[smoth]

I probably should state that it is WIP, zerver is not done, so before someone screams shitsux, understand he is working on it.

[zerver]

Only Balanced A. and Tech A. have been tested. The current situation is as follows:

Code: Select all

************** SPRING MULTITHREADING VERSION IMPORTANT NOTICE **************
LUA BASED GRAPHICS WILL CAUSE HIGH CPU LOAD AND SEVERE SLOWDOWNS
For best results disable LuaShaders in SpringSettings or the Edit Settings menu
Press f11 to open the widget list, which allows specific widgets to be disabled
The LUA-DRAW(MT) value in the upper right corner can be used for guidance
Safe to use: Autoquit, ImmobileBuilder, MetalMakers, MiniMap Start Boxes

Other widgets could work too, but the engine could crash or hang so use them at your own risk.

[forest_devil]

with default widgets for me all of these work

ba
tech a
sa
nota
xta

[smoth]

not surprising.

[Cheesecan]

Welcome back Cabbage.

Spring is buggy in both versions. Choose your disease of choice.

[Cabbage]

Some disease's are more sought after than others!

I had a little play (BA) and the diffrence is pretty stunning. The little Lua usage warning never went over 35%.

I have a Phenom II 965 running at 3.8 and on the largest games (single thread) the slowdown is pretty noticeable, CPU usage got a point where it started killing hte frame rate. With MT it remains 100% silky smooth even when tripling simulation speed and FPS remains in the 50-60's.

It's like suffering chronic fatigue syndrome all your life and suddenly upgrading to sexual exhaustion!

[CarRepairer]

With MT it remains 100% silky smooth even when tripling simulation speed and FPS remains in the 50-60's.

It's like suffering chronic fatigue syndrome all your life and suddenly upgrading to sexual exhaustion!

But the catch is you only get to have sex with llamas.

[phas]

With MT it remains 100% silky smooth even when tripling simulation speed and FPS remains in the 50-60's.

It's like suffering chronic fatigue syndrome all your life and suddenly upgrading to sexual exhaustion!

But the catch is you only get to have sex with llamas.

ahaha i was going to choke when i read this

ok, but there's any hope in the future to have a multithread version that works well with LUA scripts or is an architectural problem that is not going to be solved?

[SpliFF]

It's difficult to solve and there are competing ideas on the best solution. Ultimately the problem is twofold:

1.) It is difficult to multithread opengl calls. There is a translation layer developed by zerver but it appears to have high overhead under certain conditions - hence the issue with poor framerates in certain widgets.

2.) Lua is a state machine. It is designed to change its state through a linear set of stack changes. If these changes are done out of sequence, as can often happen in a multhreaded application, the state will break or desync. There are Lua threading implementations and the possibility of using multiple states but these are serious architectural changes.

Most proposals for multithreaded Lua revolve around splitting the monolithic Lua state along boundaries (like synced vs. unsynced or widget vs. gadget) but since Springs' architecture supports communication across synced/unsynced and shared data like WG and GG there will probably need to be made a compromise with backwards compatibility or performance in certain operations. This is an ongoing discussion but as I understand it there are implementations of the different approaches being developed and the devs will at some point decide which approach is superior.

[phas]

Does spring development have issues in backward compatibility? I mean, maps or lobbies should not be a problem in this scenario. The only thing that modifications you talk about could break seems to be mods and custom widgets.

If this is correct, my opinion is "screw backward compatibility": all the currently played mods are very actively developed (seems to me) and i think that mod developers would happily sacrifice some hours of work to adapt their mods to the new interface if this will bring to a waaaay more powerful spring engine.

[zwzsg]

Does spring development have issues in backward compatibility?

Devs, maybe not, but modders and players do.

If this is correct, my opinion is "screw backward compatibility":

Only if you volunteer to personnaly fix the millions of lines of Lua code produced for Spring so far, and to personnaly guide the hand of every poor soul who downloaded outdated widgets.

all the currently played mods are very actively developed

Let's kill every spring mod every 6 monthes, and then lament Spring never took off outside BA?

and i think that mod developers would happily sacrifice some hours of work

No, they would not. They would hate the devs and rage all over the place for grudgingly having to work a couple extra days just to keep the mod afloat instead of implementing new features.

waaaay more powerful spring engine.

Not waaayyy. Maybe 20%-50% faster at most.

[phas]

Only if you volunteer to personnaly fix the millions of lines of Lua code produced for Spring so far, and to personnaly guide the hand of every poor soul who downloaded outdated widgets.

et's kill every spring mod every 6 monthes, and then lament Spring never took off outside BA?

One fact is that millions of loc of lua were produced for spring, another is that all are actively in use. Also, on the other hand, it happens that a mod or a script break on an update of spring maybe done for a lot less important thing.

Consider it on the other hand, this will probably "make clean" of a lot of unused code and stuff.

From the user point of view, well, system like rapid that automatically keep your script and mods up to date may help a lot on that? I don't know how many people actually uses things like that, but maybe some external "update tool" could be written to check if there's an update available for your scripts? (i could probably help on something like that). (for mods it doesn't matter because you HAVE to have the latest version to play)

No, they would not. They would hate the devs and rage all over the place for grudgingly having to work a couple extra days just to keep the mod afloat instead of implementing new features.

Not waaayyy. Maybe 20%-50% faster at most.

Well, performance improvement is not just... performance improvement. It means also bigger maps, more units or more detailed units, and this let you design mod more freely, because you can design a mod that is based on more units or on bigger maps or have more detailed units if you want. I think that mod teams would appreciate that kind of freedom.

[smoth]

Does spring development have issues in backward compatibility? I mean, maps or lobbies should not be a problem in this scenario. The only thing that modifications you talk about could break seems to be mods and custom widgets.

and games, you act like the projects are all just little modifications.

If this is correct, my opinion is "screw backward compatibility": all the currently played mods are very actively developed (seems to me) and i think that mod developers would happily sacrifice some hours of work to adapt their mods to the new interface if this will bring to a waaaay more powerful spring engine.

My opinion is going to be harsh: SHUP NOOB. The current projects are actively developed and you proposed "solution" possibly will set my development back months or more. So when you look at active development you are talking about months of not work, for some projects that is enough to kill them for others like mine that means whole new sets of units and features non-existent. Not like you would understand that because in your mind these are all mods.

[phas]

My opinion is going to be harsh: SHUP NOOB. The current projects are actively developed and you proposed "solution" possibly will set my development back months or more. So when you look at active development you are talking about months of not work, for some projects that is enough to kill them for others like mine that means whole new sets of units and features non-existent. Not like you would understand that because in your mind these are all mods.

Yeah, you are the only person in the world that have written lines of code and spring mod developers are the only people that have ported a software trough a new version of the api

Now, apart from your rudeness, i admit -damn, i know- that it's a big pain in the ass to work to make your already working project compatible to a new interface incompatible with the previous one. And i admit that if the codebase is too vast and/or the modifications to make are too big, this is not feasible.

But on the other side you have to admit that the springrts engine is the base upon wich all mods/games/projects are made and that every improvement of the engine will improve the games while every limitation will greatly limit the evolution of the games.

You also have to consider that breaking the backward compatibility is something that can happen in computer programming and, since it's a big pain in the ass, various strategies have been developed and successfully used to soften it up.

So, i'll reformulate my "screw backward compatibility" with "well, since of course multi-core and then multithread is definitely the close future of computing, in my opinion you should consider if breaking the backward compatibility is a necessary pain and in that case you maybe can find a way to do it as less painly as possible", better now?

[smoth]

I think you need to go develop something for spring and realise we already have to deal with that. We also deal with lobbies not playing our projects, not withstanding maps etc.

We have minor breaks frequently but when you have a massive break which causes you to have to do a massive restructure on a project like gundam which has an entirely lua driven economy.

we are talking about someone ripping all your internal organs out and then saying, no worries, you can heal up.

[MidKnight]

Phas, you raise some good points, but what zwzsg and smoth say takes precedence: They've literally poured thousands of man-hours and years of their lives into their projects using the engine. Asking them to go back and refactor all of that is no small request. They may seem incisive, or rude, but that's because they've spent four or five years working with Spring, and, to them, your statement is broad and sweeping, and fails to take the aims of the developers into account.

It's not just the active, playable projects, either. Breaking backward-compatibility that badly would have a lot of negative consequences across the board. A good example is the old Final Frontier mod, that scifi (or was it someone else?) was trying to update for the new Spring. Significantly changing Spring's Lua API would make that nigh impossible!
I liked FF! Please bring it back!

My advice to you, Phas, is to try working with the engine!
Learn some C++ and take a stab at some bugs, or join a game development team, and write Lua or produce art. That way, you'll quickly gain a better understanding of where the major difficulties are in making Spring MT-compatible.

Best of Luck!