Ultima Ratio Regum 0.11 Update #58: Riddle Rhyme Integration Finished!

Housekeeping Notice: I’ve just upgraded my spam filter. Until now actual comments normally went straight through, but sometimes would be caught until I “okay” them, though I’m not sure how this new one will work (it should be much better). If anything is different or doesn’t work, please email me / DM me on Twitter and tell me!

And now, to our regularly scheduled programming:

Hello friends, welcome back! This fortnight we have a kind of “bumper” entry because the previous update was all about solving a specific – though I’d like to think rather geometrically and intellectually interesting – edge case in world map clue generation, which is now indeed fully handled. This means I’ve had four weeks to be working on other stuff since the last normal update rather than the ordinary two, and that means that, I’m pleased to say, I’ve been able to get rather a large amount done. Specifically, all rhyming riddle stanzas are now fully integrated into the game! This means I’ve finished off the location stanzas which I discussed a few entries ago – I thought they were done then, but it turned out they needed a bit more work – and done all the required work on local map stanzas, i.e. where you should go in a particular location (a tavern, a copse of trees, a crossroad, whatever), and all the required work on what I’m calling “action and reward” stanzas, i.e. do X and you’ll get Y (e.g. stand in a place for five turns, and a secret door will open giving you a bunch of gold ingots – or whatever it might be). This was a ton of effort, a huge amount, but it’s actually all working now, and that marks another major milestone. What has also helped here is that it is with great pleasure I can say that after a fortnight at around 20% coding capacity due to work and life stuff, and then a fortnight at around 60% coding capacity (because both of those improved), and then a fortnight around 80%, I’m now definitely back to 100% coding attention. The life stuff has really settled down nicely, and the big grant I mentioned I was working on is now submitted, and – dare I say it? – was actually looking really rather strong when we hit the submit button, and things are moving along pretty smoothly. So, this now means that two entire clue types are now fully integrated into the game (I’m sure later more variants will come in each, let’s be honest, but I’m still aiming at a proof-of-concept 0.11 release) and the game can use these to generate useful clues for any location on the world map, and do so at a huge range of difficulties, complexities, contexts, and all manner of things. So, without further ado, let’s get into it:

Sorting Edge Cases in Location Stanzas

Firstly, we had some edges in location stanzas to deal with. Specifically these involved some complex scenarios in which some of the locations or information being drawn upon were multi-tile things, such as mountains or chasms, and so the specific area you are in can’t actually be used since that could be referring to many tiles. Extra complexity, however, arose in using those as reference points elsewhere, since that would run into the same issue – if you’re trying to set a y value via obscurely describing a distant chasm, but that chasm spreads over ten different y values, then you’re not actually giving the player much of use. At the same time, however, I do want these to be usable, and so I found a few ways where multi-tide things could still be useful as part of the rhyming riddle generator. There aren’t many such examples, however, so I’d expect these to show up far more prominently in other clues which in some way make use of the world map. I also discovered that, for some reason, dormant volcanoes and volcanoes were not correctly being used when generating the rhyming stanza for a particular location. I’m not sure how this actually originally happened, given how I went about creating the system, but regardless of what specific and presumably acute mental failure led to their exclusion, they are now properly integrated. I also discovered that fortresses had no rhyming riddle generator code precisely because I intend to remove them and make nomadic civilizations genuinely that – nomadic – but fortresses were still generating on the world map, and so occasionally producing crashes when the game tried to use one of them as the basis of a riddle, only to discover no relevant information had been stored. This, again, is now fixed, simply by removing fortresses from world generation (which like I say I was going to do anyway, in shifting to having nomadic civilizations only appear in encampments that actually physically move around their territory – I think this is very cool, though I haven’t got around to implementing it yet) and thus ensuring the rhyming riddle generator never tries to use one as the basis for a line. So, with all that said and done, here are some more examples of locations, and examples of rhyming riddles they might yield:

Where in expanses of sand treads one who sells wares, with grace,

And a dusty shining beetle crawls on a tall orange and red rock, in this place…

Where a red canine is wary and guarded, north and well clad,

And hollows from ploughs are like an obsidian fungus, west a tad…

Where an archaic place of woods and skulls and ribs is most clear,

And some of the fallen sleep in a most chilled grove, found here…

Where water pools around a burnished opener, west, in no wood,

And triple dooms float gently in water, near to southward…

Alongside all of this, I then discovered there was a significant problem handling lakes and oases and ensuring that those actually fully connect to the generator. This was essentially because of stupid reasons involving old code that previously generated and then stored these in a pretty daft way – not how I would program it now, were I coming to world generation afresh with everything I’ve learned in the last, er, fifteen years. However, this didn’t take too long to fix, as I had happily actually put in the right code for generating rhyming riddles using oases and lakes as the information to be processed, but they just weren’t being accessed. Instead the game was just producing a “????” string to describe the name of these locations, and so the game then couldn’t look up a location with that name in the central database of locations and the constructions of their names, and so that in turn prevented anything useful from being generated. I also discovered that there were rare sequences in which a valid pairing was actually being rejected far too early rather than being the proper time to “try” to form a sensible rhyming couplet – this was happening when only one of the cardinal directions had any useful data in it, and the way the code was written meant that it wasn’t even trying to generate one that doesn’t use directions, but solely uses information from that location. What I mean by this is that if we imagine a university (with a name, so data point #1) in a bioregion (with a name, so data point #2) but nothing in any direction, rather than just using the university’s name and the bioregion’s name to generate the couplet, it was instead giving up because it couldn’t find anything in orthogonal lines to try with instead. This was obviously a mistake, and a big one, and I’m glad I found it – I therefore added an “overriding” bit of code that will let the generator advance even if there’s nothing to be seen in any direction, as long as two pieces of data exist for the specific location the generator is trying to use (normally the case).

I also found a bug whereby the same location, e.g. a mine to the north, could actually be used twice in one stanza, which obviously makes no sense because it gives you nothing to triangulate with. It also led to some funny-looking couplets where the same thing was being described twice in different but somewhat similar ways. This has been sorted, as has another bug in which sometimes a west and an east point (or a north and a south point) could be given, rather than one from each pair, which of course again makes triangulation impossible. You’d always need one to be horizontal and one to be vertical in order to point towards a specific tile on the world map, so that just wasn’t giving us anything of use. I actually thought there was code in there to prevent this exact permutation, and there was, but it turned out it wasn’t very good and thus needed to be adjusted to handle a 1/3 scenario where the code’s attempt to ensure this outcome wasn’t activating properly. I also discovered two other bugs involving lakes and oases which I resolved in this process. In the first case I found that if a lake was trying to do what I call the “XX” rhyming style – i.e. two “exact” (hence “X”) descriptions of the location itself, rather than two clues pointing at things in other directions (“DD”) – it wouldn’t activate because the lake counts as having river on it, and a river is a multi-tile “structure” on the world map, and thus would get rejected for being too imprecise. Whoops. This by chance also put me onto a somewhat related issue with oases, where they were not registering correctly because the oases tile is a forest tile rather than a lake tile (but just appearing in desert), and so the game was unable to figure out how the name should be taken and turned into a rhyme.

Once all of this was fixed, yet another issue also presented itself as I realised, to my disappointment, that there are still quite a few grammatical mistakes in the location stanzas being generated. These are generally things like plurals and singulars failing to correctly be reflected across the entire line (this kind of stuff is somewhat difficult in procedural content generation in general, and incredibly difficult when you’re generating entire sentences with complex structures that themselves draw on information which is, itself, generated!). There are also a few cases where a ” ” is not appearing in a stanza where it should, and also the opposite problem, with a ” ” failing to appear when it would be appropriate in a sentence, such as directly after a comma. These don’t appear often, but they appeared far more than the incredibly occasional minor grammatical bug that I might have been able to overlook for 0.11 if the meaning was still clear and I was struggling to find the issue and get it out in time. No, unfortunately, these are much more common than I can accept, and so these will need to be dealt with in due course. I confess that this dispirited me a bit, both because this generator was so much work to create, and because I actually put so much work into trying to ensure these sorts of things didn’t happen! There’s actually code there designed to track and figure out singulars and plurals! But, clearly, it is insufficient and something is missing. However, since I really emotionally and mentally need to call it “done” on rhyming riddle integration after so long, I’m classing these as bugs rather than issues with the generator’s implementation, and I’ve just added this to the (rapidly shrinking) bug list for 0.11’s release instead of dealing with it now. There may be a rare grammatical mistake or two, but the important thing is that this riddle type is now – such typos aside – fully implemented, and working rather brilliantly.

Implementing Local Map Stanzas

With these done, I then got on with the location-in-location stanzas, i.e. those telling you to go to or otherwise do something involving a certain shop, or a certain mansion, or a certain garden, or near a certain stone head or obelisk (once those are implemented!), and things like that. These need to direct you to a particular location within a local map grid, or indeed inside a building that is of course itself within a local map grid, and do so via talking about objects (a door, an altar, a statue, whatever) and giving you some relationship to that object, i.e. where you want to stand or do something relative to the object itself. These actually didn’t take long to implement into the main game file compared to some of the others, although they are going to be very complex to get generating correctly for a given building – although we’ll get to that pretty soon. For now, though, I’ve got them generating correctly for the things that might be in specific categories of buildings, and always generating logical and sensible couplets that make sense and give clear instructions, while also having the scope to usefully scale difficulty. I know I’ve talked a lot about this particularly in the entry before that, but scaling difficulty is one of the most important and one of the trickiest things to handle here. It’s tricky in games which are focused on hand-made riddles, with sometimes something being far easier (or easier to brute force) than it’s supposed to be (e.g. identifying the two European women in Return of the Obra Dinn), or something that has a risk of being far harder than it’s supposed to be simply because the developer didn’t anticipate a different, but entirely valid, interpretation of something that a player might produce. These become even trickier in the PCG context and it’s something I’ll be continually refining and playtesting, but in all clue generation we need to have the possibility for less obscure, or more obscure, information to be included. For some less obscure ones, then, we can have something like this, telling you to get into a tomb, find a particular sarcophagus, start to the right / east of it, and then go as far down as one can towards a wall:

Inside a labyrinth of burial, start east of a tomb showing sword and bone,

Then race straight to the southern-most wall, as if thrown…

While more obscure possibilities are in the generator and successfully create text, though of course I can’t feed anything into them until we have all that extra information generating. For example, for a riddle telling you to enter the parliament building, find a single chair that stands by itself to the left / west of all the others, and then – once you’ve uncovered the particular pattern by which a particular bird of prey moves – make the same movements:

In the place for governing start left of a lonely west chair,

Then do walk like the blackest owl seeking prey – this I share…

This implementation process also revealed some new complexities that I hadn’t quite appreciated before. One of these is the need to make furniture identifiable – by this I mean things like tables, chairs, beds, and the like, and also other furniture like chests of drawers or dressers which I haven’t got around to creating generators for yet, but I certainly will at some point. These currently always generate by selecting a material that would be appropriate for the location it’s found in, so one found in a far-north or far-south polar region of civilization is more likely to use larch as its material, while one found in the tropics may well use teak, and so forth. This has been entirely fine until now and indeed just adds a little bit of the visual variety the player encounters in different regions of the world (this isn’t the sort of thing that gets consciously noticed, but I would hope it adds some subconscious variation into the mix) – but there’s a problem when I will often need the generator to be using indoor furniture as a point of reference for generating location-specific clues (e.g. stand next to a chair and do something). What if the interior area has lots of chairs, or lots of tables, or whatever it might be? Some of this can be resolved by using furniture in specific locations, e.g. “furthest west” and the like, but another good way to enhance this is going to be by adding in some new variation to the furniture. As such, I’ve now added a new world generator element that creates a five or six unique generated species of tree alongside the real-world defaults – these will be with unusually red, unusually orange, unusually yellow, unusually white, unusually green, and unusually purple, woods (or just five of these if it can’t successfully generate all six). All of these are in fact real colours of rare woods (as Google will show you!) so we’re certainly not drifting far into fiction here; but having these as very rare colours for generating furniture will be a big help in making these riddles far more precise and exact. This will also rather nicely enable me to add more variation to the wilds, e.g. by small areas of forest that have a large number of these rare generated trees, and this will be of great use for clue generation, e.g. maybe you’re looking for a grove of Winebark or Bloodtimber in an area, and that then becomes an easily-identifiable landmark to be used alongside other information.

Now with all this done and these generating sensible things, they just need to generate sensible things which are actually real, i.e. which do indeed relate to the generated world. This is one of the major tasks for the next few weeks – this category of riddle can now make things which are intelligent and sensible for the building or area in question that they’re using as the foundational information for the generator, but they can’t yet confirm that it’s actually there and actually doable. This is… going to be quite a large task. I’m actually still pondering how best to make this happen, especially because there are two different – both challenging – complexities here. The first complexity is that sometimes the area we’re telling the player to do something with will have already been generated, and other times that will not be true – so it will have to create something which can be logically true no matter how something is generated. This becomes slightly easier when I think about most of the generators for the outside of buildings, since I can easily create a system that looks at them, decides on a generation, and then sets that generation in stone before it actually generates (which might happen thousands of turns later) and uses that anticipated generation to form the clue for – but it’s trickier for the interiors of buildings which tend to have rather more complex generation systems. I am entirely up to the task, of course, but… phew, it’s a complex one. The second question is whether I adjust the generators to try to add something that can be uniquely described, or do I have the generator look over what has been put in a building anyway, to try to find something that can be uniquely described? I.e. should the generator try to place a chair in a special wood in a certain location in a building, knowing that’ll then be used later on, or should it just look over what is there anyway, and try to identify distinct and interesting patterns? And does this question connect to my solution to the previous question? I’ve said it before, but I’ll say it again – it’s no bloody wonder this is all novel creative terrain, because it’s absolutely deranged in its complexity. But, for now, the code is all into the game’s main file, and generates excellent stanzas – now they just need to be made real.

Implementing Action and Reward Stanzas

Lastly came the “what do you do there?” stanzas, i.e. the ones telling you what to do when you’re at the right location, such as placing a certain item on the ground, or looking at a certain tile, or things of this sort. This was actually the easiest and smoothest implementation of all – like the above one it still needs connecting to “reality”, but shifting all the required material over from the testing file, and getting these going within the main game file, really wasn’t that hard here. What’s also fantastic is that I have the basic infrastructure in place for having the game detect when you’re in the right place to do the thing, and have something happen when you do indeed, er, do the thing – as noted before in the Roguelike Celebration talk. I’m still incredibly happy with how these are generating, and the next task (as well as connecting them to the actual items the player has and the motions one can do with them) will be to add difficulty scaling here in terms of the items being requested for the player to do something with. It’s pretty clear what you actually do with them, i.e. “promptly put on” for an item of clothing is pretty clear, so the difficulty scaling here will need to come in the obscurity of the item or items being requested. Anyway, here are some the game produced, such as this one encouraging you to find and wear a robe from a particular nation, and then gain access to a relic of the eyeball type:

There an lower-class robe of red and purple, promptly put on,
To find an optic both most sacred and most gory, from an age bygone…

Or this one, telling you to play a certain tune, on a certain instrument, to find a chest containing whale oil, in a desert:

Then do play a sannibir and play the Wolf-King’s last tune,

To find a lump of wax from deep inside a whale, below some dune…

The other good thing is that all of this is now implemented in a modular way. What I mean by this is that the game doesn’t have to generate four stanzas at once – i.e. one for the area, one for the location, one for the location within the location, and one for what you should do there – but instead can generate three, two, or only one of them, as well as the option to generate all four. This is an easy but really effective way to add a lot more variation to what the player encounters – sometimes you’d get all four stanzas, and sometimes you’d just get a handful or even just one, which then needs to be combined with others. This will soon enable me to scatter a full set of stanzas across books, for example, or have one mentioned in a book, another an NPC can tell you, and then the other two components are via maps instead of rhymes… and you get the idea. This gives a ton of flexibility, and once I get started on the quest generation system – which is the next big task! – I’ll be designing it in such a way as to be able to keep track of what information the player has actually been given in these clues, what other information is required, where that information might be found and how difficult it is to access, and all these sorts of things. While in 0.11 I currently anticipate rhyming clues giving you the full set of stanzas, that might not necessarily be the case if it proves particularly easy to generate combination clues, e.g. two stanzas coupled with a world map clue, for example. Indeed, I’m hoping this won’t be too challenging to implement, though having not yet attempted it, and given this is one of those “striking out into totally unknown terrain” things… we’ll have to see how it goes. But the important point is that each stanza type has its own function and its own process, and thus they don’t have to be lumped together if we don’t want them to be. This, like the difficulty levels on the world map clues, for example, just adds so much potential flexibility.

What next?

So there we go! All the four types of riddle rhyme generator are now fully implemented and functioning within the game itself, and can be called upon to create a logical riddle chain, most of which is directly connected to the game world, and the last bits of which are approaching that same status. The next thing, of course, is ensuring that the game will register it when you actually complete one of these (whether from the rhyming riddles or elsewhere) – i.e. I need the game to generate a database of solution triggers for all in-game riddles within a certain generation, and ensure they activate correctly. If the riddle says to stand in a certain place, in a certain town, and hold up a certain item, and you do that… well, then something definitely needs to actually, er, happen. This is itself rather complex, but not wildly so – I implemented a basic test version of this system for some testing a year or two ago, as I discussed in my most recent Roguelike Celebration talk, and it worked really well, so I should just be able to build on that and expand it up to its full size in order to make it work here. Once that is in, I’ll then be able to ask the game to generate a simple riddle thread, the simplest possible, and make sure it can give me a clue that tells me to go to X and do Y, and doing so registers that you have done it. At that point we will really be cooking – but we need to get there first. At the same time there remain a few other polishes that I want in 0.11 (nothing major, but just nice quality-of-life stuff), and a hundred bugs for the 0.11 release to deal with, and music and sound integration to continue with (due to unforeseen circumstances the soundtrack development has been pushed back a few months, but since I’m still working hard on 0.11 myself, that’s no problem – and what we have so far of the soundtrack is bloody fantastic), so there’s still plenty of stuff to do. Thanks as ever for reading, everyone – if you enjoyed the post please do think about leaving a comment below, or sharing the post around! Hope you all have a great fortnight, and I’ll see you all in two weeks for the next update, whatever it may be, as the sprint towards a 0.11 release this year continues apace…