Ultima Ratio Regum 0.11 Update #37: Riddle Generation 5: Specific Locations

Hello everyone, and welcome back to the new fortnightly blog (sorry this one’s just a day late – had a health scare over the weekend, the first / worst I’ve had in a long time. I think things are going to be okay, but it really shook me up mentally / emotionally). This week is another update on the generation of the poetry riddles which are going to be the core middle type for release in 0.11. Not to get ahead of myself, but I’m pleased to say that these are actually coming together far faster and more easily than I had anticipated – I had approximately budgeted the entire year to complete these, and what’s becoming clear that, at most, this is going to be a six-month task (at least to the degree we need them for 0.11)!, which is quite exciting because it leaves the second half of the year free to work on other things before the release of 0.11 towards the end of the calendar year. As such, this week’s blog post covers the almost-completion of the specific location stanzas for the riddles, so let’s get into it:

Names to Ciphers

The main achievement here is that I now have a system which can take any name of anything on the world map, and convert it into a short version that hints at the name without explicitly giving get away. This was months of work, although I was of course working on other stuff at the same time as well. This took so long for several reasons. First and foremost, there are just so many possible names for things on the world map – bioregions, cities and towns, fortresses, tribal settlements, mountains, volcanoes, lakes, rivers, roads, junctions, graveyards, crypts, slums, city district, monasteries, caves, mines, islands, and many other things beside. Each of these has at least one generator for creating its names, and in most cases has several, or a dozen, or even more, and each generator can then in turn create a huge range of names. There are certainly tens of thousands of possible names for settlements or other things on the world map, and that’s not counting names that include a word which has been invented for the sake of the name, such as for instance a town that might be called “Xia’s Rest”, or something like that. This meant that there was a huge amount of data that had to be corralled, and the different generators for the riddle sentences had to reflect different generators for names, because or every different kind of name produces different kinds of data. We might look at a towns name like “Xia’s Rest”, and another then that might be something like “Streets of the Six Ravens”, and immediately see that there’s very different semantics and linguistic data in these two that we could draw on to then generate a clue.

The second difficulty came from the fact that the above issue meant that every single generator had to create a different kind of sentence, yet also create sentences that could usefully be placed within something on each side of them – it’ll be clear what I mean by that shortly. What this means is that there are, literally, several hundred generators present here, all wrapped into a single ur-generator. When the game is trying to find an obscure and cryptic way to discuss a certain place, and therefore doesn’t just have to use the data within that places name to generate a sentence, but it also keeps track of what sort of area is being described, and also what subset of potential name generators was used to create the name of that place, and hence which potential description generator should be used to respond to the data in the name. This was all absolutely mind-boggling, both in terms of the creative work required, and in terms of the technical work required. Each of these generators then needed to be able to find pieces to slot into different kinds of sentences depending on the nature of that place, while also ensuring that every possible sentence, depending on what aspects are involved in the name of that place, are actually being invoked. This, ultimately, is what took so long, and while dealing with the technical aspect was comparatively fast, even if it did bamboozle my brain at least a little bit, the creative work was by far the hardest. So many potential words that had to be given meaning in logic, so many potential scenarios for names which had to be accommodated, so much keeping track of where verbs and nouns and adverbs and adjectives all being used, and so much else besides – and all of that in dozens upon dozens of different generators. Whew. BUT, with all of that done, we then had a system which could generate sentences like these, for the first half of a rhyming stanza:

This was one hell of a task, but I’m extremely happy with how they came out. None of them are entirely trivial, to the point where any aspect of a location’s name is literally or precisely included in the clue, but getting properly familiar with an in-game world shouldn’t make them too difficult to solve either. Naturally, a lot of playtesting is going to be required here both before and after I release 0.11, but overall I think these convey the degree of difficulty and cryptic meaning I’m looking for, while not being even remotely impossible. In my own playtesting I found that even when I can’t work back from the generated output to decipher the name of the settlement or location (even speaking as the game’s creator), it’s very clear when I look at whatever it is that is being referenced and discover the meaning. I get a very enjoyable “ah ha, THAT’S what it means!” moment every time, and I of course very much hope players will get this as well. But now let’s take a bit more of a detailed look at the generators, so here is each of these quotes where I’ve colour-coded the large scale different sections that are being chosen an integrated in order to create the sentence. Beginning, firstly, with this one for the name of a tribal settlement:

Or this one for a lake:

Or this for a volcano:

Or this one for the name of a town:

Or this one for a district of a city:

(I’ve also included a more detailed breakdown of the language generation in the final one, just to demonstrate some of the areas where variation is possible here within this absolutely unhinged system I have for some reason spent about two months creating)

What these are all doing, then, is telling the player that if they look west from a location (the location the poem is hinting at), they’ll find X (whatever X might be); then (see the next blog post after this one), you triangulate that with the other half of the stanza, and you’ve got a precise, single tile, on the world map, to be looking at. I also think and hope that the examples above give some sense of the degree of variation possible here. That’s only five potential generators which I’ve shown above, although some of the generators blur into each other so it’s hard to precisely define that, but I think in each case it’s very clear how the name of the location has been rendered into something which is a riddle, which is a clear an grammatically and syntactically correct sentence, which ends by stating a relative location compared to what it is that the player is in fact looking for, and the things like plurals, tenses, and so on, are all maintained across the generated clue sentences. This last point was also especially hard, and begins even tougher when we think about what the potential ends of sentences might be, and then becomes even tougher again when we keep in mind that not just do these have to have a set of comparatively finite endings to point in a certain direction, but the same mid-sentence parts – by which I mean the red and green and blue – also need to be able to refer to the location you’re looking for, rather a location which is in a straight line from the location you’re looking for. This means that they have to have logical endings that make sense for whatever the statement contains, even though even though there’s hundreds of archetypes for that statement could contain, and then those have to rhyme… and one’s brain begins to melt again. Nevertheless, this is huge progress, and in a fortnight I’ll be showing off the finished second half of this system, which was just as brain-melting, but gives us the rhyming sentence pair for each of these “specific location on the world map” stanzas.

As a final point here, an interesting thing also happened while working on these location riddles. Specifically, for the first time in several years, I found myself not particularly enjoying the process. To be clear, this has now passed, everything is back to normal, and I remain extremely proud of the outcome produced here. But for the first time I found the process to actually be a bit of a slog. I hadn’t found this while working on the civilization parts of the riddles, probably because there was a detail and a poetry to the phrases there that I was able to develop, and also probably because it was still quite a novel thing that I was working on. Here, by contrast, I did find myself becoming a little frustrated by the repetition of the process, and the sheer volume of phrases that both needed to be put into the game’s databases, and the number of ways the game needed to be able to combine those phrases into intelligent things. In a recent Roguelike Radio episode I mentioned that I’ve been reading John Romero’s autobiography / memoirs – which I recommend highly, they’re a great read – and in it he mentions something like this which happened during the final development of Wolfenstein 3D. Although the gameplay was extremely rich (for the time), the process of level design was quite dull because of the limitations of the engine and the comparatively finite number of things that could be created in it. They were incredibly excited by the output, but found themselves becoming quite bored by the creation process. I therefore rather think that I have just experienced something quite comparable here. I’m incredibly proud of and excited by the output, but this particular bit of the creation just wasn’t all that compelling. I just had to sit down and do a ton of work – and quite repetitive work too – in order to get this part of the system working properly. Like I say, all is back to normal now! Just thought it was a moment worth sharing, and a reminder of (for me, at least) the importance of developing many facets at once, so I can switch from coding to art, art to writing, and writing to coding, whenever I feel the need.

However, even with all of this in place, all this only enables us to do one of the two lines required in a stanza for pointing towards a particular location. This lets us tell the player something is to the east, or to the west, or towards sunrise, or towards the pole, and things of this sort – but we need to rhyme it with something which gives another point of reference. This part is proving extraordinarily hard for this part of the riddle generator, because we need to use the same core phrases here, but they need to be functional with more unusual endings. This isn’t going to be a dealbreaker or anything of that sort, but my goodness, this is tough, because all these sentences can easily be put between “Where”/“And” and “to the east”, but what if we need the final word to be “beast”, or “least”, or “deceased”? Well, those then become specific to the content, but we need the sentences to be generated for every possible kind of name for every possible kind of location on the world map… and basically, this is wildly hard. I have some good ideas, and I’m working on it, but this certainly goes beyond this entry. More on this soon! But right now we have the cores required for both halves of the sentence (i.e. everything between the first word and the direction), but a second set of outside parts for the second rhyme in the stanza is tough. Working on it, though! (And now the hard part is done, this next bit will be very different, thank goodness…)

Other Things

There were also a whole bunch of minor things in the civilization stanza generation which had to be checked or taken care of in this latest cycle of development. The first of these was handling the “hegemony” cultural aspect or trait civilizations, which is the term to note that this civilization is very big on ideas of racial purity and things like that. I found myself having to go back through all the potential poetry lines for these in order to make sure that there was no conceivable way that a player reading it could think that I’m endorsing or legitimizing those sorts of philosophies. Now, the reader of this blog might think this is a completely insane fear, but trust me on this one friends – it’s not. The ability of people on the web to read incorrectly into creative works is truly unparalleled (and there’s quite a few examples I could think of here) and so I wanted to be absolutely certain that these clues are purely descriptive about that nation’s beliefs, and thus don’t give any kind of credence to to the real-world idea of racial purity (i.e. I need to clearly articulate the difference between a line like “this nation pursues purity”, which is not good since it implies racial purity is an actual thing, and “this nation pursues what it calls purity”, which is fine and solely descriptive). I got this done, though, and I don’t think there’s any risk of that any more. Alongside this I also had to go through and make sure that the parts of the poetry lines which call upon the description of each civilization’s unique identifier – such as pyramids, stone faces, things of this sort, which I’m super looking forward to generating – and poetry which draws on a nation’s unique weapons, always use the correct code. In my genius I had used unbelievably similar and easy-to-misread variable names for the two of these (sigh) and so I had to make absolutely certain that each of these was always being correctly used within the generator. This was a boring task, but didn’t take too long, even if it was completely mind-numbing (though necessary).

Then, with those sorted, I needed to go back and put in some appropriate code to handle all the placeholders. These would be things like describing the name of a ruler, or whether the a single god or multiple gods are worshipped in their nation, or the name of the god(s), or the name of a prophet, or things of this sort. These will become vastly more important as we get into more complex clues which actively engage with world histories and past events and things with this sort, rather than things the player can immediately spot within the game world, but these were still important to change right now away from placeholders and towards something that will actually integrate more fully with the game’s code. Again, it was not even remotely intellectually demanding to move all these placeholders, but it did take quite a while. Then, I discovered that there were quite a few missing lines for some of the variations, where presumably I hadn’t been able to think of something interesting when I was writing them, or I got distracted, or I’d actually moved on without completely thinking something through, or… well, honestly, I’m not sure what exactly led to these being present, but there were quite a few of them hidden within the generator. It turned out that there were about 100 lines I’d left completely blank, so these again had to be filled out with appropriate generating sentences.

I also found that I needed to make significant changes to some of the databases of words used in generating the names for many locations in the game. This included cities and towns, tribal settlements, mountains, rivers, volcanoes, lakes, and many others. In some cases this was because some of the words within a given name generator might be too ambiguous and non specific, thus making me struggle to create a way to generate a description for them that would be obscure, requiring attention and thought, but also unambiguous – not confused with anything else. Examples include birds like eagles and hawks, which are extremely similar, but the average person – even I, as someone fascinated by wildlife and zoology – would struggle off-hand to articulate a clear distinction between the two, since both are creatures we might refer to in a clue as having sharp eyes, sharp talons, being birds of prey, and so on. A similar issue was present with leopards and jaguars, and also some had to just be removed altogether, such as buzzards, which are just a bit too similar to other birds to have the player quickly distinguish them (again, without looking it up, could you define a buzzard? Define it vs an eagle or a hawk? I don’t think I could). What this meant was that in some generators I simply removed one of the two identical or too similar words, and in other cases implemented new code which said when one is chosen, the other cannot then be chosen within the same world, so you might have a mountain which references a hawk or eagle in its name, but only one mountain can reference either one of them, and no mountain can then use the same other words in its name except with the other animal in its place. In other cases words like dark or shaded or gloomy suggest very very similar synonyms and ciphers, and so had to be addressed in the same manner. This also applies for “deer”, “elk”, “reindeer” and “moose”, with the generators using those words being polished and tightened so that there’s no ambiguity. The key point here, of course, is that we want clues to be ambiguous in terms of players understanding, but not to be ambiguous once the player understands them – i.e. we want the player to need to work to think what kind of creature might have sharp talons (this is obviously a very simple example!), but once they’ve decided on that, they shouldn’t then have three potential locations across the entire world referencing a “hawk”, an “eagle” and a “buzzard”, and yet still fitting the rest of the clue, and thus having to just try all three because there’s no further information. Thus, riddly-ness in wording should not equal ambiguity in decipherment. A logical decipherment must always be the right answer.

With all of those done, I needed to address myself to potential rhyming words which have only a single category right now for a potential sentence – this being the category which will engage with things like histories and past events and things of this sort, which are way more complex and will be used in far more advanced riddles that will come in later in the game. The overwhelming majority of riddle words have a wide number of things which they can call on in order to develop their rhyme, but a few words only have the ability to call on more detailed information, and so these had to be – for the time being – removed from the overall generator in order to ensure it does not waste anytime trying to call them while creating a riddle. With that done, I also identified a range of “bad pairs” of words, where two words might be part of the same rhyming set, but which shouldn’t be allowed to rhyme because they sound bad or silly when together. Examples would be rhyming light with sunlight, or night with midnight, or things of this sort. It’s completely fine of course to rhyme light with might, or night with site, but some rhymes where the words were just too similar, or one word was in fact nestled within the other word in the rhyme, just came out looking really clunky, so I had to look through them all, find every example of this, and wipe them out completely.

Whew.

What next?

So that’s everything for this fortnight! Thanks so much for reading folks – I hope the specific location parts of the riddles read in a cool way (I certainly think they do!) and some of the other description as well gave a sense of some of the many complexities that have to be considered here, in terms of ensuring every clue is unique and unambiguous, making sure nothing is generated which I don’t like the look of, going back then and adjusting aspects of the world generator as result of what the riddle generator needs to do, and so on so forth. This is such hard work, but it really is rewarding, and I can really begin to see the foundations of something special here, especially when I look a year or two down the line – beyond 0.11 – and see many different kinds of riddles present, may different kinds of clues, and an ever deeper and more complex world…but let’s not get ahead of ourselves. As ever, please do share some thoughts on what you thought of all this if you feel like it, any folks you know who would be interested in reading about procedural riddle generation – still a world first! – do please think about sending them this way.

With all that said, I’ll see you again in two weeks for some more updates! Thanks everyone :).

(Oh, and – we’re going to stick to fortnightly updates for now. I’m really happy with this pace.)