Hello friends! Welcome back. An exciting entry this week.
To begin with: you know how just a few entries ago I said that the 0.11 release was going to have just one clue type, i.e. the rhyming riddle clues? And I’d done about 95% of the required work on these, and they were looking really cool and novel…
…and I was excited to get them into the game? Well, those are now implemented, of course, and in the process I also added a huge amount of extra detail to, and then implemented, world map clues as well! I think these are also extremely cool, and should yield really interesting gameplay when combined with the rhyming clues:
At that point, though, I felt like two clue types were definitely sufficient, especially after the time (a full year, on and off) that it took me to implement the rhyming riddle clues. At the time I did say – and I’m sure I’ll be correct about this – that the rhyming clues are likely to be the single most complex clue type I ever implement in the game. I decided to start with these partly because I think there’s just something so compelling and intriguing about a rhyming riddle (especially one procedurally generated!), and also to get the toughest one “out of the way” first. Coming after that to the world map clues, this prediction certainly seemed to be true thus far. These took me a couple of weeks of focused energy, but nothing beyond that, and I couldn’t be happier with how they’ve come out as well. Nevertheless, with these two in place, I definitely felt these were going to be the two clue types for the 0.11 release (still very much aiming for this year, and getting absurdly close now as well?!)… but then a day came and I just found myself gripped by inspiration. I wanted to see whether I could get another clue type into the 0.11 release – and also in such a way that would allow me to properly conclude the 1/4 riddle rhyme types that isn’t yet fully implemented (i.e. because it’s pointing to specific areas in local maps). I sat down, gave the entire day to the task, and just decided to see whether I could make massive progress on another clue type in a single inspired burst of effort. If it didn’t get me very far I would probably leave it for the future, but if it did get me far then suddenly we’d be adding even more clue variety into the 0.11 release, and far more rapidly than I had first anticipated. I really do like how these clues look, and I was really hoping to get the generator going in record time:
And without further ado – I think we all realise I wouldn’t have done that huge build-up had this not gone incredibly well – I’m now incredibly pleased to say that after about a week of focused work (building on a super-productive first day of these), Ultima Ratio Regum can now successfully procedurally generate a third clue type, which is to say, the “local map” clue! These can point the player to a specific location either on a local map grid (i.e. 200 x 200 outdoor tiles) anywhere in the game world, or within a building. I think these look incredibly cool and have a visual richness and style that is obviously distinct from the actual appearance of those places – i.e. how they look on the screen when the player is actually walking around them – but also very distinct from the world map, and give off that sense of being something more precise and detailed than the world map equivalents are. While these are not finished, a large number can currently generate in clear detail from all kinds of different in-game contexts – see below for detail – and the foundations of the complete generator are well in-place, with only further development for more complex scenarios being required to finally bring it all together. I honestly couldn’t be happier with this achievement and being able to add a third clue generator type into the forthcoming release, and so in the rest of this entry I’m going to go over the process of the generator’s creation, the challenges, how the generator works, and then (of course) finish by showing off a whole bunch of variants this generator is able to produce. So, let’s get into it:
The Double Generator
To begin with, I realised that this generator had to be able to take in two different kinds of input, and produce the same kind of output from them. This is different from all the other generators so far – though it will be duplicated with some others later, so I suppose this was good practice – and posed an interesting design and coding challenge that I hadn’t really encountered before. Specifically, what I mean by this is that there are two different data sources which both need to feed into the generator for this. The most obvious data source is an actual generated local map grid, or a building – i.e. you’ve generated a local map, and then you encounter a note of this type pointing you towards something of interest on that local map (or indeed on another local map already generated, I suppose). In this case the generator needs to be able to look at some area on the local map the player is currently on, and translate those into the map’s format. That’s relatively simple when it comes to terrain, but then when we’re thinking about objects that might appear on that tile, and handling things like height, and differences between inside and outside areas (as well as the map edges and limits and so forth), it does all become rather more complicated – though not unreasonably so. However, the hard is that it also needs to be able to do this for terrain or building interiors which do not yet exist, i.e. which have not yet been generated by the game. Readers will remember this is one of the major challenges I’ll need to overcome for the “realistic picture” clue type as well, and while I do essentially know how this is going to work, it’s a rather demanding task to implement it for all possible buildings and all possible outdoor terrain areas – so for now I’ve gone with a proof-of-concept to demonstrate that this can be done, before later going back and ensuring this works for all local map clues, and picture clues, AND the third layer of rhyming riddles as well. Point is – we need to be able to predict generative elements in advance.
So with this realisation, it was clear I needed to create essentially a function that takes two types of data – “real” generated data with actual terrain, features, and all the rest of it, and “abstract” non-generated data, such as the intended layout of a mansion that hasn’t yet been generated – and be able to understand both of them, before then funneling them both into the same second half of the function that will generate the note containing the clue information. In turn, once I figured this out, I also realised that I was actually going to need the same thing for the third-level rhyming riddles, i.e. those directing you to a specific place on a local map or within a building – we have to be able to predict what the local map will look like, at least in part, or predict what the interior of a building will look like, at least in part, so that we can create a useful clue about that location even when it doesn’t exist, and when the player finally gets there, it does indeed exist as it should. The only way to do this is to create a gigantic function into which the game can feed quite literally hundreds of possible other generators and grids of strings that are designed to generate either indoor or outdoor things, and for each one have a kind of “key” explaining what in that grid means what (e.g. in one grid ‘s’ might mean place a statue there, but in another it might be ‘S’, or even some other character, if I used ‘s’ and ‘S’ for something else), and then have it translate any possible grid, using the correct key, into a new grid which accurately depicts how that final thing is going to look, before the player gets there, and THEN to force that into existence when the player actually generates that area. That final part we’ll come to later – for now, I needed a function where you can feed in a string grid – such as the one below – and it’ll understand any grid, coming from any function. This meant getting started with a trial generator as a proof-of-concept, and then expanding it out from there.
This brought me to castle. Castles – an example of the string grid for the octagonal shape shown above – are one of the least “generative” things in the game on the outside. Many of the characters here can still vary, e.g. being guards, or statues, or heads on pikes, and we have different heights and various other things here, but castles are still one of the simplest exterior things to generate. This was because when implementing them I was really struggling to create a more varied generator that actually met the design and aesthetic objectives I had in mind, so I decided to just give in and make these much more “standard” than most areas (the insides of castles are wildly varied, indeed more than many other buildings – but the outsides are reasonably uniform, even if there is more variety than the above suggests). All of this meant they were a perfect first test for this function. So, the function takes in two things – it takes in the grid object, filled with strings which are filled with characters (like ‘u’ or ‘#’ or ‘ ‘ or ‘G’ in the above example) which the game will interpret when generating the actual thing being generated, and it takes some information about the grid, which I began as just the label of what the grid is – so I gave this the label “Outdoor Castle”. Then the function is basically a dictionary of dictionaries. For those who don’t know, in Python a dictionary is a data structure which you can fill with a bunch of “keys” – metaphorically equivalent to a word in a dictionary, i.e. the thing you are looking up is the word – and some information that key represents within the dictionary. A key can be anything, so it could be a letter, a word, a number, a graphic, or basically anything else, but the important thing is that if you have a dictionary called all_grid_dictionaries – which we now do – and you call it with all_grid_dictionaries[‘Outdoor Castle’] – it will create a list of what every character, on that grid, represents when it’s fed into the generator. That second list is itself a dictionary, so it would be something like a dictionary where list2[‘{‘] would respond with ‘drawbridge chain’, list2[‘”‘] would respond with ‘head on a pike’, and so on. Then that’ll get to the point where all the grids, regardless of what characters are used in that grid, will only be producing strings that correctly tell us what actually goes there, and they will thus all be funneled into a single data structure.
So I set about implementing this with castles first, and at first it went well – I created a quick function that could read any spatial generator and note down every possible character (again, by “character” I mean “I” or “S” or whatever here, not NPCs) that could appear in those generators, and then the idea would be to put each generator into this system, get a breakdown of every possible character, and then go through manually and note down what each character represents. I put the castle into this, got a list of the characters that the castle outdoor spatial generators use (again I stress how very “ungenerated” these are, since I just designed a bunch of huge grids for these – every other building has more generative-ness than castles, but that very simplicity made them the ideal place to begin here), and explained to the list what each of these characters translates to (so ‘#’ is always a wall, ‘u’ is a wall at a lower height than ‘#’, ‘r’ is water tiles (i.e. the moat), and so forth. I then created a function that looks over the grids in a generator and selects a 5×5, 7×7, or 9×9 area of that grid, and then uses the translation dictionary to create a new list (in the sense of the python data structure called a “list”) of what each tile is – so this list is no longer [‘#’,’r’,’#’,’ ‘,’ ‘] or whatever, but [‘wall’,’water’,’wall’,’nothing’,’nothing’], or whatever it might be. This meant that anything could be funneled into this system, and then regardless of what characters mean what in that specific generator, they will all turn into that second category, from which both the picture clues and the local map clues could be usefully created. At this point I’d only spent a few hours and I was already beginning to see the potential here – once this system was in place, the final thing would just be to add a second system that subsequently ensures the later generation of that map grid, or that building, follows the section made in the clue creation, but that’s actually not wildly onerous and it’s pretty clear in my brain how I’ll do that.
However, I soon ran into some complexities here. Some generators have sections which aren’t generated in a single sweep, but are instead generated in another way. For example, in parliament buildings, some sections inside offices are generated separately from everything else, i.e. the game uses a large set of strings like the ones above to lay the foundations of the parliamentary building (but there’s of course a lot of variations there, worry not), but then identifies where the offices are and generates them separately – which means if I want to enable a clue to spawn including those rooms, it’ll have to pre-identify how many office rooms will exist, which office type or set of furniture will spawn in each of those rooms, which angle, etc etc… and that’s even more complex. It means I can’t just have it translate one grid into certain ways if other grids are later going to be interspersed and integrated here. One of the worst examples I found of this is the generator for the insides of arenas, which actually crates two grids – one with pillars and the main structure of the arena, and one with chairs and candle stands for lighting – and then synthesizes the two of them. That’s going to be a nightmare to pre-generate like this, while a ton of others have these extra aspects which add an additional layer to something already tricky. It’s possible for some I’ll just have to accept we can’t have local map style clues generating in those buildings or those outdoor contexts because he generator is so complicated that nothing can be predicted without just running the generator (and thus consuming time), but I hope that’s not the case. Helpfully in some cases I’ll definitely be able to set the generator to ignore certain parts of the area, knowing that part will be generated later, and only generating potential clues or directions within a subset of valid parts – that won’t be visible to a player, but you reading this (one of the elite blog readers) will know the secret… but don’t tell anyone.
Other complexities arise in other generators, such as the graveyard generator. Rather than using a string-based grid with high variation in its content, graveyards generate by moving across the map grid in very unpredictable ways, expanding, placing walls and graves, and doing so very organically. I have been meaning to adjust that generator in the future, and to redo the grave generation since it’s currently good but not really sufficient for the level of detail and complexity I want to get, so when I come to do that, it’ll be an interesting experiment to create a generator with the need to allow pre-generated sections being in my mind when I make the generator – this should make things a lot easier, and will also yield far more interesting and varied graveyards (and graves) than we currently have. But the point is, at present, there’s not really any way to pre-generate graveyards without just generating the entire thing, and then we again get into the scenario of spending valuable seconds building a map grid the player might never set foot in. So, after a few days work, I’ve gone through the overwhelming majority of generators or generative elements, and coded them according to whether they can be immediately slotted into the pre-generation generator (now there’s a title), or whether they are basically finished aside from an issue involving how doors are placed and generated – again, very difficult to pre-generate, and I’m going to have to look into more closely – and then a category of “Not Finished”. Some of these have just one minimal additional element to factor in, but others are so complex that I’m honestly not sure how they even could be pre-generated without just generating the entire thing. For building interiors that actually wouldn’t be too dreadful if need be… but this is where things stand at the moment:
This means we currently have six buildings and the majority of the outside generation in towns and cities viable for this technique – i.e. pre-generating one important piece and doing so in such a way that it can be later “enforced” into that part of the game world itself when being created – which is a good start! Obviously I don’t want to only be able to use this clue generator for this set of buildings for the 0.11 release, let alone beyond, but it’s a really strong and solid start which will lay the foundation for the second half of this generator (i.e. enforcing what has already been generated, but we’ll get to that next week) and proving the whole thing works. And, a lot of the others can be integrated into this system as well, but some of them are really tricky to figure out a pre-generation technique for; many of these involve multiple layers where many parts are generated first, then other parts are generated on the basis of those parts, and other aspects are generated as a result of those aspects, and so forth. As above, it’s possible that for some buildings we will actually need to pre-generate the entire building before the player gets there, though that then runs into other issues with the tracking of door numbers and building numbers and various other things. I’m still cogitating on how to handle these, but the important thing is that plenty of places in the game world are now viable for testing the later halves of the system – but the part before that is shifting elements of the generators out of the actual generator functions, and into a larger dictionary which can be accessed by the clue generation function(s), and the building generation function(s), in such a way that will of course ensure consistency. I began this experimentation with parliamentary buildings rather than castles, which meant taking all the rooms out of the parliament-generating function and into the dictionary, both the main building permutations and the rooms that might appear inside:
And then feeding this into the function I’ve just described that knows what each character represents in each generator and can then produce a logical local map grid in anticipation of what a place will look like… and to my absolute amazement, it actually worked. By inputting a grid that would later be used to generate the interior of a parliament building, the game can search for an interesting-looking (this is an interesting requirement and one I’ll discuss in the next entry) clue and then translate the characters in the parliamentary generation grid into the appropriate symbols for the local map, even if another grid in another generator might be using the same symbols for an entirely other set of items or terrain types. Below I’ve just generated a few examples from one parliamentary subset, and even if they aren’t finished (e.g. generic wood colours for furniture, a generic “swanky floor” colouring, but these are all minor things to be polished later), I think they look really tremendous – and jus expand those in your head to any possible building, any possible colour scheme… and we begin to get the idea. Of note here is that you’ll see anything outside of buildings uses the “abyss” tiles and colours, i.e. a few grey specs within black. I debated having these parts of these notes always cut off (i.e. damaged) but that looked bad and it wouldn’t be clear to the player whether there was actually useful information there or not; I also thought about making them totally black, but I hated how that looked; and I thought about having the outside world that the building sits in appear there in a combination of the inside and outside, but I thought that risked confusion as well, and I liked the idea that you’ll be able to quickly identify something inside a building from that visual – and, actually, it yields a rather cool way to give extra help on some of these clues too, e.g. by using that space to add some abstract concepts or information to point you usefully towards what the right building might be. So overall I think using the abyss visual there is really cool and works very nicely, and I’m more than satisfied with how that’s presently looking, as shown here in three parliamentary examples:
Or here, in examples from the outside of a castle, which of course has no abyss visuals since the map grid is entirely “complete”, as in, there are no inside or outside elements:
Or here from a marketplace:
These are great and I’m so pleased at how smoothly they’re working with the abstract materials, but then there’s also the second half of the generator, i.e. generating a local map clue based on a “real” map grid (i.e. one which has been generated and the player can actually walk around on) vs generating a local map clue from a non-generated one (as described above). At first glance this might not seem very helpful, but there’s actually quite a few significant and valuable use cases where this would apply. For instance, I want there to be entirely contained secrets on some map grids, i.e. which generate when you generate the map grid and are wholly present there. Obviously their rewards and yields are going to be much smaller than more complex ones, but I really like the idea of giving weight and presence to each area contained wholly within that area, and thus encouraging a player to spend more time checking out a given monastery or a given city district, along with the more “globe-trotting” clues. This can also be used in a chain, of course, so we might have a clue you get in location X about something in location Y (so some elements of Y have been pre-generated), then you arrive at Y, solve that, and get a new clue also about location Y (generated on the fly), and then once that’s done, you get a new clue about location Z, and thus head off again. The fact that this generator – and I’ll be able to use the same code for other generators also dealing with local-scale map information, but we’ll get to that in next fortnight’s blog post, or the one after that… – can take abstract data from generators, and “real” data from generated maps, and still produce clues, is the huge strength this thing offers for both expanding the range of possible clues and riddles it can contribute to, and keeping generation time down as low as humanly possible. It also means I’ll be able to quickly generate a bunch of “tutorial” clues for the university where the player starts, too (this is another thing I’ve been thinking about, but again for another entry).
And so, with this done, we’ve now got the foundations of local map clues generating from both abstract generation material, and actual generated in-game spaces, which is a tremendous leap forward. This same code will also be usable to finish off the last bit of the rhyming riddle generator, i.e. the one talking about actual physical spaces the player can walk around and what they should do there / where they should go, and the “picture” clue archetype which will be able to take this data, then imagine what it would look like to be viewing these places from the side rather than from above, and then generate a picture of that for the player to decipher. There’s still quite a bit to do here, but I couldn’t be happier with how they’ve come out; I mean, just feast your eyes on these beauties:
What next?
Well, there we go – I’m just so pleased to report another fortnight of huge progress, pushing through clue type after clue type, figuring out solutions in relatively short periods of time to stuff that I honestly thought was going to be wildly difficult (rather than merely “very difficult”) to solve in a way that didn’t slow down generation time but still enabled the variety and precision needed, and perhaps most importantly, building the data infrastructures required for the game’s future. In this regard the key point I suppose, in the abstract, is that we’re rapidly gaining more and more ways to manipulate the game’s information – world map, local maps, buildings, civilizations, religions, places, items, everything – in interesting and cryptic ways that will pose challenges to the player at a range of difficulty levels, and can be reliably generated no matter what world ends up being created for your particular playthrough on a given day (and doing so without adding so much as a second to world generation!). This isn’t really a single milestone as such, but more a kind of huge, rolling milestone, as things are gathering real speed and I’m beginning to see how such a diversity of clues and information, coupled with a world of the level of detail we’ve currently got generating in the game, are going to slot together in what I hope (!) is a truly interesting, novel, and compelling way. There’s quite a bit more to be done in this generator which is likely what I’ll be covering next fortnight – or possibly the fourth clue type for 0.11 (!) which I’m already, in this same fortnight, partway along developing as well (!!) – but I’m so pleased with this initial massive burst of progress. I genuinely do intend to stop once we hit four clue types for 0.11, but since for a long while I thought we’d be getting one type only (the rhyming riddles), I think this continues to be a fantastic (and remarkably speedy) set of additions. Thanks as ever for reading, everyone – if you think this is super cool please do think about sharing it around on the web with anyone who might be interested, and I’d love to read your thoughts in the comments section below :). See you all in two weeks!