I think it would be nice to allow reloading archives while developing, as right now we need to restart each time we add or remove a file.
I would like to provide a way of reloading the entire archive, and collecting the list of files from disk at the time of the reload.
Couple of ways to do it:
1) reload the entire archive each time /luaui reload or /luarules reload is invoked, and Lua devMode is true.
Pros: Pretty intuitive and doesn't require active thinking.
Cons: Might be expensive to go to disk each time we reload. Wouldn't help if we just toggle on/off a single addon that tries to load a newly created file.
2) Provide a new /reloadarchives or just /reload command that can be used to manually reload all loaded archives. Doable only when Lua devMode is true. It could also do /luaui reload, /luarules reload and so on, as necessary
Pros: Doesn't make /lua* reload commands any more expensive, and it could even be automated with external fs monitoring tools, like springmon
Cons: Requires devs to know about this other command, and use it when necessary.
Right now I'm mostly interested in addon files, but I think it would be cool to allow reloading defs, map files, and artwork, with some constraints.
I think this is very important as restarting and setting up the same situation can be very time consuming when developing new games (LD). Also it's probably confusing to new devs.
PS: I'd like to work on this, just got a bit of a cold now so I figured I'd just write this out first.
Thoughts?
Allow reloading archives while developing
Moderator: Moderators
- Silentwings
- Posts: 3720
- Joined: 25 Oct 2008, 00:23
Re: Allow reloading archives while developing
I don't think this is intuitive - a reload xyz should only reload xyz. Also, it could make it very difficult to test widgets that had luarules interaction, or reacted to specific sim/luarules states (e.g. when the luarules state communicated to the widget typically depends on luarules having saved info from events in the past).1) reload the entire archive each time /luaui reload or /luarules reload is invoked, and Lua devMode is true.
Pros: Pretty intuitive and doesn't require active thinking.
If it's possible, as a clean sweep, sounds great.Provide a new /reloadarchives
Imo a /reloaddefs command, which was even possible in the past, and these days would also have to re-run _post, would be useful too. I don't think reloadmap is needed.
- Forboding Angel
- Evolution RTS Developer
- Posts: 14673
- Joined: 17 Nov 2005, 02:43
Re: Allow reloading archives while developing
Well, spring already trips over large .git histories in .sdd. The idea of forcing a reload worries me a bit. However, it could be easily argued that reloading content would not be the same as spring doing it's cache thing when spring.exe is run.
Honestly, I would much rather that the cache issue be sorted out. Can't spring be made to just completely ignore .directories inside game .sdd s? I hesitate to call them "hidden" directories, because lolwindows.
If spring didn't take 5 minutes to load on utterly random occasions, I would be so much happier. Granted, the long loading times generally happens directly after a push/pull (this is why I blame spring reading .git). If this wasn't an issue, then I would be all for reloading content via a command.
Edit: Holy shit... Is spring recreating cache because of crlf->lf (because windows and git) line endings?
Honestly, I would much rather that the cache issue be sorted out. Can't spring be made to just completely ignore .directories inside game .sdd s? I hesitate to call them "hidden" directories, because lolwindows.
If spring didn't take 5 minutes to load on utterly random occasions, I would be so much happier. Granted, the long loading times generally happens directly after a push/pull (this is why I blame spring reading .git). If this wasn't an issue, then I would be all for reloading content via a command.
Edit: Holy shit... Is spring recreating cache because of crlf->lf (because windows and git) line endings?
Last edited by Forboding Angel on 25 Oct 2018, 13:02, edited 1 time in total.
Re: Allow reloading archives while developing
To put it another way, would including new files be appropriate when reloading files used by LuaUI/LuaRules with /luaui reload and /luarules reload?Silentwings wrote: ↑24 Oct 2018, 08:07I don't think this is intuitive - a reload xyz should only reload xyz. Also, it could make it very difficult to test widgets that had luarules interaction, or reacted to specific sim/luarules states (e.g. when the luarules state communicated to the widget typically depends on luarules having saved info from events in the past).1) reload the entire archive each time /luaui reload or /luarules reload is invoked, and Lua devMode is true.
Pros: Pretty intuitive and doesn't require active thinking.
It is kinda hard to define what those files are, as they might appropriately be put outside of the LuaUI/LuaRules dirs (often the case for config files, libs, etc.), so effectively updating the entire archive file list might be necessary
I am starting to think /reload might be the best choice.Silentwings wrote: ↑24 Oct 2018, 08:07If it's possible, as a clean sweep, sounds great.Provide a new /reloadarchives
Imo a /reloaddefs command, which was even possible in the past, and these days would also have to re-run _post, would be useful too. I don't think reloadmap is needed.
Is there a need to have a separate /reload* for all different types of things? (Lua, defs, etc.) Might as well just reload everything that has changed, as we have to go to disk and retrieve files.
Re: Allow reloading archives while developing
I've seen you mention this before and I'm going to investigate what the issue might be. We really shouldn't be loading that by default, and you already claim to have it in your springignore.txtForboding Angel wrote: ↑24 Oct 2018, 19:33 Well, spring already trips over large .git histories in .sdd.
Re: Allow reloading archives while developing
This comment should clue you in why using .sdd's as repository roots is a bad idea:
OT: implementing separate /reload*'s (/reloadscripts, /reloadsounds, etc) would be manageable and precedented, but a monolithic command or reloading entire archives severely complicates things.
Code: Select all
// If we are here, we could have invalid info in the cache
// Force a reread if it is a directory archive (.sdd), as
// st_mtime only reflects changes to the directory itself,
// not the contents.
OT: implementing separate /reload*'s (/reloadscripts, /reloadsounds, etc) would be manageable and precedented, but a monolithic command or reloading entire archives severely complicates things.
Re: Allow reloading archives while developing
Are you talking about development or playing? Since I don't know of an alternative to .sdd for development.
Hmm I suppose you're right, a full reload might be better done with Spring.Reload, but I do think we need to refresh the archive file list so we can load new files through VFS. That should be done in one command.
It might be a bit of a PITA if that's all the command did, as users would have to write additional /*reload* commands after that. That's why I suggested that maybe /luaui reload should also reload the archive (maybe just update the file list?)
Think we're nearing a stage where it would be good to start with implementations and see how it behaves
- Forboding Angel
- Evolution RTS Developer
- Posts: 14673
- Joined: 17 Nov 2005, 02:43
Re: Allow reloading archives while developing
IDD, Unfortunately, it must be this way in order to utilize rapid, or so I was instructed back when I was setting up git for use with rapid. So, Unless I'm mistaken, anyone who wants to utilize rapid simply doesn't really have a choice. Much in the same way that I actually wanted a release branch and to have rapid use that. However, afaik, the way rapid is set up can only look in the master branch.
It would appear that our hands are tied.
- Silentwings
- Posts: 3720
- Joined: 25 Oct 2008, 00:23
Re: Allow reloading archives while developing
I thought the .springignore file was meant to fix these issues? I wonder if maybe there is some confusion here about what was meant.
Re: Allow reloading archives while developing
Archive entries listed in springignore are only exempt from checksum calculations (obviously unwanted on .git and other sync-breaking data), but the main issue with sdd-as-repo usage is that when you update any file inside an .sdd the OS has to throw out part of its disk cache, and since VCS'es like git tend to touch many files at once just for bookkeeping purposes you can get massive loading time spikes seemingly at random when the engine determines (by examining the root st_mtime stamp supplied by the OS) an archive to have been modified and is forced to rescan all of it. This will keep happening even if your springignore.txt contains a '*'.
Quite literally no part of Spring is designed for archive contents to be added (or deleted) at game run-time, and even plain changes to a majority of file types (say models) won't be picked up easily. These are the proverbial good intentions paving the road to hell.gajop wrote:I do think we need to refresh the archive file list so we can load new files through VFS