The last three days have been very productive! Here’s what I’ve done:
Monday: Having identified what I thought was the entire source of memory leakage (global values brought out of save files and then not appropriately deleted when you save, reload, quit, etc) I have managed to get all of these to clear themselves out. This has at least halved – although sadly not completely fixed – the memory leakage! Nevertheless, this is really good progress. The memory leakage issue has been reduced by around 50-60%, and in fact the normal memory usage of the game has been reduced by about 30%, which is a nice added benefit. Great progress! But: I’d like to get more, and obviously ideally get things completely fixed; this fix reduces the problem from “severe” to “middling / minor”, but it’s still there. The remaining memory leakage is still at this point something of a mystery.
Tuesday: It has become apparent that some extraneous memory is being used by storing solar system data. I have for some time been meaning to remove the system generation element, both because it isn’t in keeping with the setting / context of the game, and because it doesn’t currently have any actual effect on the game, even in terms of world generation. This has therefore has now been shifted to a behind-the-scenes element that generates values which might be of use in later cultural elements (e.g. using the names of planets for something) but the visuals, which took up a reasonable amount of memory, are now gone. In turn the planet generation bit of “Create a World” now goes straight to generating the Earth, rather than cycling through potential solar systems. This has saved a bit of memory in-game, and reduced the memory leakage a little more.
Wednesday: I’ve spent most of today searching for the source of the remaining memory leakage issue, but sadly without finding it. I thought it could be a duplication of certain console windows being created over and over within the game, but that doesn’t seem to be the answer. Another possibility was that the local map the player was on was still being held in memory when you went back to the menu or did something else; but that also wasn’t the case! Mark of 3 years ago had apparently fixed that. I also considered the possibility that a large number of data points for things like camera position and the like were being stored far too long; also not the case. My final idea was that there might be some issue with my use of shelve, the Python library I use to save and load data within URR – maybe I wasn’t closing files properly, maybe I was doing too much when the files were open, maybe I was loading from files repeatedly, or something – but this also doesn’t seem to be the case. I’ve tried making a whole bunch of changes and nothing has altered anything, and equally, doing a bunch of experiments with shelve outside of URR haven’t generated any issues.
So this is where we are at the end of Wednesday. The Windows 10 problem is fixed; the memory leakage is 50% fixed; but I’m not sure what the next attack on the problem is for Thursday. But don’t worry, I’ll think of something. More soon!
Did you try using a memory tracer like the one here https://mg.pov.lt/objgraph/ ?
It should definitely be useful for your case.
Are you planning a Linux and/or MacOS release? PY2EXE should be quite capable of packing them, but of course you’d need to have the OSes yourself.
Hey Roberto, this sounds really useful! I think I’ve pretty much fixed things now but I’ll definitely keep this in mind if something else goes wrong memory-wise…
I always thought that the fact that the solar system generation is in the game means that it is a part of some game mechanics, be it astrology or art, and also maybe a part of the conspiracy, like the extraterrestrial origin of man or Planet X man-lizard overlords secret government bla bla, but from the storytelling perspective that sounds too cliché. Still really cool thing to have in the game and awesome experience for a player to witness the world generation with all the numbers!
By the way, what is exactly the winning condition, if there is one? Will there be an ending screen in the 0.8?
Haha, yes, the solar system was going to influence the generation of some stuff, but what that stuff was I haven’t yet got to. No win condition just yet, that would take more time and I cannot possibly justify more time before release! But assuming I do a 0.8.1 bugfix, I might chuck a win condition in there. OR I MIGHT NOT. We’ll see: the priority is getting it OUT, and everything else comes later.