Dev meeting minutes 2010-06-27 - Page 3

Dev meeting minutes 2010-06-27

Minutes of the meetings between Spring developers are archived here.
User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-06-27

Post by AF » 01 Jul 2010, 20:30

0_o I can't believe you all kicked up a fuss over this!

Multithreaded spring is desirable

It won't run with lua gadgets and fancy visuals ( well, gadgets that involve the latter ), so therefore we will disable this for the moment in multithreaded, as it's a WIP anyway and we haven't figured out a solution but someone might find it useful

Tobi points out that only BA might benefit from the limited multithreaded build because it does not rely massively if at all on these disabled gadgets

What part of the above is not blindingly obvious!
0 x

User avatar
lurker
Posts: 3842
Joined: 08 Jan 2007, 06:13

Re: Dev meeting minutes 2010-06-27

Post by lurker » 01 Jul 2010, 20:57

There is no reason to disable the calls. Low-rendering widgets don't impact performance. Just tell people to turn off the fancy worldspace widgets.


SirMaverick wrote:
Tobi wrote:I would try first to make it better use CPU cache.
How? Spring is run on many different systems.
What of it? Syncing spring only runs on x86/x64, so cache lines are always 64 bytes, and the way to optimize for L2 cache use is simply to minimize it.
0 x

Tobi
Spring Developer
Posts: 4598
Joined: 01 Jun 2005, 11:36

Re: Dev meeting minutes 2010-06-27

Post by Tobi » 01 Jul 2010, 20:59

SirMaverick wrote:
Tobi wrote:I would try first to make it better use CPU cache.
How? Spring is run on many different systems.
That doesn't mean locality of memory accesses can't be improved. (Which is good for many (all?) types/sizes of CPU cache.)
0 x

User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Dev meeting minutes 2010-06-27

Post by Argh » 01 Jul 2010, 21:34

What part of the above is not blindingly obvious!
Apparently some of it was, as the quote was a bit ambiguous. Hence why I thought a bit of clarification might be in order.

Re: cache use, etc.- what will that do? I don't understand that stuff, so a layman's description would be useful, in terms of why we'd expect to see a meaningful performance change.

In terms of graphics, et al:

1. Why doesn't Spring allow Lua to set up Vertex Buffer Objects, and do more precaching of geometry on the GPU?

2. To what extent is Unit geometry / textures being precached on the GPU in the current setup? I am aware that the code sends these things once- but why send them at all? Can't they be stored on GPUs?

These are both areas where I feel like improvements might be possible, and get the Lua load down somewhat.

The biggest performance hit I've seen with Lua --> OpenGL is geometry and texture transfers, when you start pushing stuff really hard- the only more costly operation is access to FBOs. I suspect that's also a fairly heavy load on the engine's draw processes currently- Spring's performance in this regard is a lot worse than anything commercial I've looked at lately.

If we could store geometry on the cards, we would, of course, have to face geometry limits and possible crashes, which is a danger. But practically everything modern is doing that, because it allows for really fast instancing operations, amongst other things. We'd have to be very conservative- perhaps allow caching for up to 128MB of data as the default, and allow for end-users to set it lower / higher if they are having problems or want even faster performance. But I think it would be highly desirable.

Speaking of instancing... since Features don't move or turn or animate, why not allow that geometry to be instanced via a shader that moves the geometry to the right worldspace coordinates? That way we only send the display list one time (and if it's already on the card... we're just sending a uniform).

The same can be done for Units, but the process would be slightly more complicated. A vertex shader doing this with geometry already on the GPU's memory should be significantly faster than constructing the display lists on the CPU and transmitting them every draw cycle.
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-06-27

Post by AF » 01 Jul 2010, 22:26

As we add all these features Argh, we run into the issue of hardware compatibility.

While we could make springs overhead a fraction of what it currently is if we used all available technology, the problem in doing that is that spring would rely on features only available in top end cards. We cant rely on features found in OGL 3 cards and DX10 class hardware when a sizeable portion of the user base is running on integrated graphics.

One thing which would help a lot with what you cite as problems, is moving the rendering into a separate thread, which is kind of what this discussion is about.
0 x

User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Dev meeting minutes 2010-06-27

Post by Argh » 01 Jul 2010, 22:46

We've lost a lot by being ultra-conservative, and there are various ways to make it work for lower-end hardware.

The current code could be frozen and declared to be the fallback, for example. There's no reason we can't do that, it's just a flag, and when that code inevitably bitrots, we just declare it dead and move on. Hardware doesn't sit still because we want it to.

But we've gained nothing from our decisions over a year and a half ago to avoid going over the GLSL brink because "some developers can't test", for example. Those developers quit being active anyhow, and the engine doesn't look any newer, and if we're too conservative then we can't make the kinds of radical changes that are necessary to revitalize interest in the engine. I can't imagine showing off screenshots of the current engine to anybody outside this place and getting anything but polite laughter, frankly, after seeing what the modern state-of-the-art actually looks like.
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-06-27

Post by AF » 02 Jul 2010, 00:34

The point being that what your discussing is a subject that should be in a separate thread, and not derailing this one
0 x

User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Dev meeting minutes 2010-06-27

Post by Argh » 02 Jul 2010, 02:35

How so?

This ought to be good...
0 x

Chamrin
Posts: 33
Joined: 05 Mar 2010, 12:31

Re: Dev meeting minutes 2010-06-27

Post by Chamrin » 02 Jul 2010, 02:43

hoijui wrote: you know... about once a week, we get someone with an intel 945 - or sometimes even weaker/older stuff - reporting a bug because spring runs so slow. and half of them say: but i heard spring does not need good HW...
I had to send my radeon 3650 in for replacement a couple weeks ago and have been using a geforce 2mx 400 with 64 megs of ram (this card is nearly 10 years old) but I turned down all the graphical effects, lowered the resolution, and I was quite surprised at how well the game still played. Sure I ran into problems like the terrain overlays (f1,f2,f4, and los) didn't work on the main display, but that's probably just because the card doesn't support whatever feature those use. The game never really slowed down for except in cases where there was an insane amount of units on the screen or smoke from a nuke. The conclusion of this is that spring is far more cpu dependent than graphics card dependent. So anything that will increase cpu efficiency, like multi-threading should be persued. Especially considering cpu's are going towards more cores and more parallel execution.

But yeah, spring plays fairly well on old hardware and you guys should be commended for that. I'm only using a pentium 4 2.6ghz from 2003 and I never had a problem even in the 16 player games until the mid to late games. Air units often slow things down more than other units though.

Cham
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-06-27

Post by AF » 02 Jul 2010, 02:50

Argh wrote:How so?

This ought to be good...
Because the problem at hand is the multi threading, and the problem of draw calls from lua in a separate rendering thread and accessing synced data. Examples of possible solutions being scene graphs andduplicated sims states etc. Actual code performance itself is merely a side discussion
0 x

User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Dev meeting minutes 2010-06-27

Post by Argh » 02 Jul 2010, 02:58

Eh... right...

Like we aren't interested in seeing some things get rewritten with newer techniques that run faster and take full advantage of modern hardware, if it's decided to make MT the default goal.
0 x

User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Dev meeting minutes 2010-06-27

Post by jK » 02 Jul 2010, 03:08

there are 3 discussions atm:
  1. current problems of GML build
  2. how should a future MT build look like
  3. cleanup and optimize code (which may make code ready for parallelization)
PS: item 3 means cpu code and not render code! render code optimization was discussed already and there are already some ideas, but they need a lot of work, so you just can wait until it is done.
0 x

User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Dev meeting minutes 2010-06-27

Post by Argh » 02 Jul 2010, 03:17

<shrugs>

You can't just move the code over as it is; large amounts of it will need to get rewritten, if for no other reason than to avoid callbacks to CPU1, which is the current problem, unless I've misunderstood here (unless you go with my sim without regular rendering, pathfinding, etc. idea, which I still think is attractive just because it'd be brutally simple by comparision).

So I guess I just don't see the point in trying to cross that bridge and not do a clean sweep, frankly. Less work in the long run.
0 x

User avatar
AF
AI Developer
Posts: 20669
Joined: 14 Sep 2004, 11:32

Re: Dev meeting minutes 2010-06-27

Post by AF » 02 Jul 2010, 03:28

How about for now we add in an extra lua state for rendering but don't actually move stuff over, so that we can use it for stuff that will only ever be used without access to synced data, eg displaying the backgrounds of a GUI, base metalmap overlays, lualobby controls.

It wouldn't be of much use to developers right now, but we could implement the common groundwork needed for more wholesome solutions ontop of it
0 x

User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: Dev meeting minutes 2010-06-27

Post by Argh » 02 Jul 2010, 03:44

That's a start, but it won't really solve the big problems.

Most of the serious Lua OpenGL stuff isn't just some TextRects; usually you have to have text (and sim calls to fetch things) and that wouldn't make a dent in the serious things, like UnitRendering, P.O.P.S., and various shader stuff. Heck, it wouldn't cover 90% of the Widgets, frankly. Hence why the whole "two synced sims that do different things" thing is a fairly attractive notion.

I looked up "scenegraph", since that seems to be the Term of the Week.

It's pretty much what I am talking about, in terms of changes to unit rendering. It wouldn't be necessary if there were two sims in lockstep, ofc, but what the hell, it's another idea. Here, this is how I see it working:

1. CPU1 simulates the position and rotation states of all Units, like normal, but doesn't do anything about the display lists, etc.

2. CPU1 then sends a list of changed Piece states and Unit velocities / headings to CPU2. Note, "changed". And stuff like Features only is included when birth / death occurs.

3. CPU2 has stored the Unit geometry; whether on the GPU / CPU doesn't matter at all. It now has a scenegraph of the Unit / Piece states, does a search to determine what should be rendered, and for Units that need to be rendered sends a call to the GPU with the uniforms for position and rotation and velocity (and a display list, if it's not already stored) and the current time.

4. GPU vertex shader does the matrix transforms and rotates the Unit's Piece geometry so that it's positioned correctly in worldspace.

5. A material shader on the fragment side rasterizes the texels.
0 x

User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: Dev meeting minutes 2010-06-27

Post by aegis » 10 Jul 2010, 11:09

hoijui wrote:z: see last two commits:
http://github.com/spring/TASServer/comm ... ptPassword
(they are already in master)
http://cgit.springlobby.info/cgit/aegis ... 85d8142d05

now we just need lobby/autohost support?
0 x

User avatar
Licho
Zero-K Developer
Posts: 3803
Joined: 19 May 2006, 19:13

Re: Dev meeting minutes 2010-06-27

Post by Licho » 10 Jul 2010, 11:49

Will this mean you can reconnect and replace existing player of same name who is about to time out?
That would be nice.
0 x

User avatar
BrainDamage
Lobby Developer
Posts: 1164
Joined: 25 Sep 2006, 13:56

Re: Dev meeting minutes 2010-06-27

Post by BrainDamage » 10 Jul 2010, 12:06

Licho wrote:Will this mean you can reconnect
reconnecting would be possible, but what do you mean with
Licho wrote:replace existing player of same name who is about to time out?
if you mean irc-style multiple logins, no idea
if you mean snatching someone else's username, the password will prevent that ( that it's the entire purpose of it )

me & hoijui also added a whitelist mechanism that allows spectators to be added midgame trough the management port ( so yes, people can join running battles too as spectators ), but it's off by default

to enable it:
set WhiteListAdditionalPlayers to 1
both are spring config options
then, when a user joins the battleroom and sends it's password, the autohost sends to spring trough the management port:
/adduser username password
and that user should be able to connect to spring even if he wasn't in script.txt
EDIT: fixed option name case
Last edited by BrainDamage on 10 Jul 2010, 22:24, edited 1 time in total.
0 x

User avatar
Licho
Zero-K Developer
Posts: 3803
Joined: 19 May 2006, 19:13

Re: Dev meeting minutes 2010-06-27

Post by Licho » 10 Jul 2010, 12:15

If you crash at startup (or freeze), you want to reconnect ASAP.

Atm if you try to reconnect, game assigns you new name with _ at and then claims you are not in script.
You have to wait until your first connection attempt (with already killed spring) times out from host.

This is pretty annoying. Most lobbies have some way to reconnect after spring crash, but its pointless because player is greeted with "name_ not found in startscript".

Engine could instead "kick" previous player with same name and keep this new one.

Reconnects are common for some windows users (about 50% of my own starts in windowed mode spring freezes)
0 x

User avatar
hoijui
Former Engine Dev
Posts: 4342
Joined: 22 Sep 2007, 09:51

Re: Dev meeting minutes 2010-06-27

Post by hoijui » 10 Jul 2010, 12:17

the sp compatFlag support is in TASServer too now:
http://github.com/spring/TASServer/comm ... d92a0b2e72

thanks aegis!
0 x

Post Reply

Return to “Meeting Minutes”