Multithreaded code: shared variables

[gajop]

How does one write multithreaded code with shared variables using Spring?
Are there any existing constructs (macros, classes, functions, etc.) that you've created to handle variable locking, unlocking (mutexes/semaphores) and similar?

I'm supposed to finish up the code with the pr-downloader callin support, where pr-downloader callins happen asynchronously and must have their events properly queued (consumer/producer pattern) in the main (Lua) thread.

I didn't bother writing anything decent until I figure out how to do it right, so this is what I have right now: https://github.com/gajop/spring/blob/pr ... pp#L38-L64 , which is not thread safe and has memory leaks.

[Kloot]

There are a few custom mutex classes under System/Threading, but those mostly exist for performance reasons.

What's the big problem with doing it right, anyway? Conceptually all that needs to be maintained is a shared queue which Lua can put requests into, and a worker thread which takes these out in order and hands them to the downloader lib. You can use polling, provided the worker gets to sleep a reasonable amount of time between polls and does not lock the queue for the entire duration of consuming a request (i.e. it should only hold the lock to pull out descriptors).

Download status events (started/finished/failed) should have their own shared queue filled by the worker and emptied by LuaVFS, so the callouts are properly executed.

Since the downloader itself is probably not threadsafe (AFAICS) but can process multiple requests, you'll want to limit yourself to only one thread for this.

[gajop]

There are a few custom mutex classes under System/Threading, but those mostly exist for performance reasons.

That's what I'm mostly curious about, since I'd rather not impact overall engine performance by something that should mostly only be related to ingame lobbies.

What's the problem with doing it right, anyway? Conceptually all that needs to be maintained is a shared queue which Lua can put requests into, and a worker thread which takes these out in order and hands them to the downloader lib. You can use polling, provided the worker gets to sleep a reasonable amount of time between polls and does not lock the queue for the entire duration of consuming a request (i.e. it should only hold the lock to pull out descriptors).

Actually the Lua -> pr-downloader (callouts) isn't a big problem as those functions would be called rather infrequently so shouldn't affect performance much. (The worked thread can be pretty much paused until a download is requested).

Download events (started/finished/failed) should have their own shared queue filled by the worker and emptied by LuaVFS.

The only potentially performance sensitive part I'm concerned is the emptying of the shared queue by the LuaVFS. As I understood it, it should poll to see if any new download events have occurred in (or before) each Update, and then convert those events into Lua callins. The problem here might be if that queue needs to be locked, accessed and then unlocked each time the check is performed, as it might cause a performance hit as indicated by the whole ZK/windows mutex performance issue we've had during the previous versions.
I could probably have it access a bool to see if any changes have happened, and only lock the queue in those cases (which might delay events by an update, but that's a non issue).

[Kloot]

I'd rather not impact overall engine performance by something that should mostly only be related to ingame lobbies.

Just put the worker to sleep *liberally*. Two seconds of latency between polls is fine.

As I understood it, it should poll to see if any new download events have occurred in (or before) each Update, and then convert those events into Lua callins. The problem here might be if that queue needs to be locked, accessed and then unlocked each time the check is performed, as it might cause a performance hit as indicated by the whole ZK/windows mutex performance issue we've had during the previous versions.

If you check 30 times per second, sure, but that would be complete overkill because for download status reports you can get away with a much lower frequency. Even the progress callout doesn't need to be spammed for every new byte that trickles in.