Lua Garbage collection info

Lua Garbage collection info

Discuss Lua based Spring scripts (LuaUI widgets, mission scripts, gaia scripts, mod-rules scripts, scripted keybindings, etc...)

Moderator: Moderators

Post Reply
User avatar
Beherith
Moderator
Posts: 4925
Joined: 26 Oct 2007, 16:21

Lua Garbage collection info

Post by Beherith » 24 Oct 2013, 13:32

In the latest engine develop (94.1.1.1400+) there is an entry in the debugging view called Garbage collection. In some BAR test games, this quickly climbs to 20% cpu use, and is most likely luaui related (as /luaui disable changes garbage collection to 3%).

How can we measure which widgets/calls are producing the most garbage?
What general guidelines should be followed to make the garbage collector's job easier?
Should we set all unused local variables to nil once we are done with them?
0 x

User avatar
Silentwings
Moderator
Posts: 3511
Joined: 25 Oct 2008, 00:23

Re: Lua Garbage collection info

Post by Silentwings » 24 Oct 2013, 15:11

I'd bet this is related to funks_smenu, which loads & unloads alot of unitpics. When it 'unloads' the pics it seems to cause a memory leak in luaui but I'm not sure which of funks_smenu/chili/engine is to blame yet.

(At some point I or funk will write funks_smenu so as it doesn't do this because its a very expensive way of making the selection menu. But I suspect doing so won't fix the fact that its possible to make chili/luaui leak memory.)
0 x

User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Lua Garbage collection info

Post by jK » 24 Oct 2013, 18:15

Beherith wrote:In the latest engine develop (94.1.1.1400+) there is an entry in the debugging view called Garbage collection. In some BAR test games, this quickly climbs to 20% cpu use, and is most likely luaui related (as /luaui disable changes garbage collection to 3%).
<_< seems dedicating it to its own event was useful >_>
Beherith wrote:How can we measure which widgets/calls are producing the most garbage?
Not possible
Beherith wrote:Should we set all unused local variables to nil once we are done with them?
That's not how Lua works.
The top most reasons for garbage:
  • tables: try to reduce their creation, reuse them when possible. Clearing them takes mostly more time than creating new ones, so e.g. only reuse them when the keys in it are always the same like {1,2,3}, {id=...,name=...} so clearing isn't needed. Also buffer often used tables to `static globals`, e.g.
    bad:

    Code: Select all

    function Foo()
      Spring.Bar("pink ponies cannot fly", {options=1}, {}) --note: lua should cache the string _most_ of the time, but the tables get recreated each call! -> spams garbage, when called often
    end
    good:

    Code: Select all

    local optdefparams = {options=1}
    local emptyTable = {}
    ...
    function Foo()
      Spring.Bar("pink ponies cannot fly", optdefparams, emptyTable) 
    end
    (PS: this is true for inline function creation too)
  • strings: avoid `str = str ... "foo"` in loops for string appending, when you want to concat multiple strings use table.concat it saves _a damn lot_ garbage
0 x

User avatar
Beherith
Moderator
Posts: 4925
Joined: 26 Oct 2007, 16:21

Re: Lua Garbage collection info

Post by Beherith » 12 Dec 2013, 19:43

There is a lot of garbage collection going on in BA, by default (with default widgets on, and running a long, heavy replay) , it progresses as the following:

Image

Note how towards the end game, garbage collection takes more time than Lua itself.

So I tried to find which widgets cause this, and disabled each widget one by one in BA. There was no directly causative widget, the drop in garbage collection was very gradual as I disabled every widget.

But I did notice that after disabling _every_ widget, the garbage collection load was still 2x as much as if I did a /luaui disable. The load from garbage collection was still similar to the entire(!) Lua load:
Image

By the way /luaui disable brings the load of garbage collection down to about 4%. Disabling every widget leaves it at 6%.

Any thoughts?

More info for the uninitiated like me: http://lua-users.org/wiki/OptimisingGarbageCollection
0 x

User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Lua Garbage collection info

Post by jK » 12 Dec 2013, 20:13

0 x

User avatar
Beherith
Moderator
Posts: 4925
Joined: 26 Oct 2007, 16:21

Re: Lua Garbage collection info

Post by Beherith » 12 Dec 2013, 20:40

Ok, ill test it as soon as the build trickles downstream so I can use the awesome new springlobby features :)

On a side not, is calling a lot of Spring.* functions that return tables also considered to be excessive table creation?
0 x

User avatar
Silentwings
Moderator
Posts: 3511
Joined: 25 Oct 2008, 00:23

Re: Lua Garbage collection info

Post by Silentwings » 12 Dec 2013, 20:59

I have a question too. For example, does

Code: Select all

local x,y,z
for unitID in pairs(fooBar) do
  x,y,z = Spring.GetUnitPosition(unitID)
end
cause less gargbage collection than

Code: Select all

for unitID in pairs(fooBar) do
  local x,y,z = Spring.GetUnitPositions(unitID)
end
?
0 x

User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Lua Garbage collection info

Post by jK » 12 Dec 2013, 21:29

Beherith wrote:On a side not, is calling a lot of Spring.* functions that return tables also considered to be excessive table creation?
Yes
0 x

User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Lua Garbage collection info

Post by jK » 12 Dec 2013, 21:30

Silentwings wrote:I have a question too. For example, does

Code: Select all

local x,y,z
for unitID in pairs(fooBar) do
  x,y,z = Spring.GetUnitPosition(unitID)
end
cause less gargbage collection than

Code: Select all

for unitID in pairs(fooBar) do
  local x,y,z = Spring.GetUnitPositions(unitID)
end
?
no
0 x

User avatar
smoth
Posts: 22295
Joined: 13 Jan 2005, 00:46

Re: Lua Garbage collection info

Post by smoth » 12 Dec 2013, 23:36

SilentWings:
jK wrote:bad:
function Foo()
Spring.Bar("pink ponies cannot fly", {options=1}, {})
--note: lua should cache the string _most_ of the time, but the tables get recreated each call! -> spams garbage, when called often
end
because each time the function "Foo" is called it has to on the fly create 2 tables at parameters 2 and 3.
jK wrote:good:
local optdefparams = {options=1}
local emptyTable = {}
...
function Foo()
Spring.Bar("pink ponies cannot fly", optdefparams, emptyTable)
end
The two tables are already created and reused(it is just an address to the table)

your example doesn't really have to do with JK's example. Always reuse a table. The less table creation the better, not just to do with garbage collector, every time you make a new table, it hurts performance.
0 x

User avatar
Beherith
Moderator
Posts: 4925
Joined: 26 Oct 2007, 16:21

Re: Lua Garbage collection info

Post by Beherith » 12 Dec 2013, 23:53

jK, that reduces garbage collection load on luaui enabled with a /give all from 9% to 4%.
0 x

User avatar
Silentwings
Moderator
Posts: 3511
Joined: 25 Oct 2008, 00:23

Re: Lua Garbage collection info

Post by Silentwings » 13 Dec 2013, 00:06

your example doesn't really have to do with JK's example
I hadn't meant it to involve tables - all I wanted to do was create some data multiple times as opposed to a single time.
0 x

User avatar
smoth
Posts: 22295
Joined: 13 Jan 2005, 00:46

Re: Lua Garbage collection info

Post by smoth » 13 Dec 2013, 00:31

I don't follow what you mean here.
0 x

User avatar
NeonStorm
Posts: 173
Joined: 23 May 2012, 18:36

Re: Lua Garbage collection info

Post by NeonStorm » 19 Dec 2013, 15:21

Each time you add a value to some table, the table's memory size changes.

Is it better to use linked lists and link emty (for re-use) together?
That will be a lot of work...

Or maybe we should create a free-to-use tables list, sort it by length and re-use them everywhere if it got about the size you need.
0 x

User avatar
Silentwings
Moderator
Posts: 3511
Joined: 25 Oct 2008, 00:23

Re: Lua Garbage collection info

Post by Silentwings » 19 Dec 2013, 16:38

Neonstorm wrote:Each time you ... memory size ... linked lists and link emty (for re-use) ... a lot of work... tables list ... re-use them everywhere ... size you need.
Please stop polluting serious threads with speculative chatter.
0 x

User avatar
jK
Spring Developer
Posts: 2299
Joined: 28 Jun 2007, 07:30

Re: Lua Garbage collection info

Post by jK » 19 Dec 2013, 16:50

Silentwings wrote:
Neonstorm wrote:Each time you ... memory size ... linked lists and link emty (for re-use) ... a lot of work... tables list ... re-use them everywhere ... size you need.
Please stop polluting serious threads with speculative chatter.
Correct, code KISS. any complicated reusing of tables SUCKS, don't do so! it will be slower! only save redundant table creations like passing {} to a function each call etc. (create a `global` emptyTable and reuse that etc.).
0 x

User avatar
Beherith
Moderator
Posts: 4925
Joined: 26 Oct 2007, 16:21

Re: Lua Garbage collection info

Post by Beherith » 08 Nov 2015, 22:46

We have some large chili menus in BAR that are making the garbage collector eat 10% of cpu with the widget on, and only 3% with the widget off.
I changed the MaxLuaGarbageCollectionTime to 1ms, that dropped the load back down to 3%.

Are we doing something potentially unsafe here?

Code: Select all

CONFIG(float, MaxLuaGarbageCollectionTime ).defaultValue(5.f).minimumValue(1.0f).description("in MilliSecs");
0 x

User avatar
Silentwings
Moderator
Posts: 3511
Joined: 25 Oct 2008, 00:23

Re: Lua Garbage collection info

Post by Silentwings » 08 Nov 2015, 23:49

To note: our two Chili widgets that cause a heavy GC load do so whilst not actually doing anything that allocates any significant mem (at least, not explicitly), they just own a large number of chili controls, which are not altered.
0 x

Post Reply

Return to “Lua Scripts”