Well.
Here we go.
The time has finally come. The tutorial quest – described in this entry, and this one – gave me incredibly valuable initial insights about how to handle the overall quest generator. Having now done so to the degree required in 0.11, I can confidently say that a) such initial experience was absolutely essential and I’m so glad I designed the tutorial quest first, and b) this riddle quest generator gas the potential to become something pretty incredible, which I’ve never seen before in a game, and as I continue to build it up and up in the coming years, it has all the potential I dreamed about it having, and maybe even more. While I haven’t fully tested a single quest just yet, I’m literally just a few days of work away from being able to, after, 15 years, play a quest in URR from beginning to end, and that is just too exciting for words. This is therefore a big entry, but it’s a huge milestone and there’s a lot to say. Thank you all so much for sticking around on this mad journey and this mad attempt to really create something actually new here, and for the first time ever I actually, sort of… see it, now. I see it in the game.
I see it coming into being, and I am pleased by what I see.
Anyway, let’s get on with it, beginning with finishing off the data infrastructure / architecture required for the generator, which is where I started after we left off last time…
Finishing the Architecture
In the previous entry I described some of the foundations I was able to lay for the architecture of the quest generator – contained within a function called make_a_quest – despite being very unwell and frankly very down for the last few weeks. Being able to make such progress on such a huge part of the game… which is, in a certain sense, the most important thing in the entire game… really gave me a boost, however, and I’m so pleased to say that soon after posting that entry I got back to it and continued to make massive progress. To briefly recap, I had set up a system that could select the difficulty for a quest, separate a quest into steps (though right now there’s only one step type, which is “go to a specific location and do a specific thing there”), and then begin to distribute to the player the clues that they would need to complete that step. This is the most streamlined possible quest I could be generating, since it uses the chief clue types I’ve been able to develop for 0.11, i.e. the rhyming riddles, the world map notes, and the local map notes. In the process, however, it was very important to create a flexible and effective system that in the future I would be able to add any kind of input into (i.e. a clue or a piece of information the player needs), add any kind of quest to (e.g. go to a place, talk to a person, look at a thing, whatever), add any kind of outcome to (i.e. what is the solution to the quest, how and when does the game register the quest as completed and give you the reward), and they’d slot nicely into the existing architecture. Given that different clue types might have different requirements, or some parts might be generate-able for a given quest while others are not, and some steps might have multiple alternatives, and all these pieces are constructed from different generators and then brought together in one place, this is a very complicated task. However, this fortnight I set about resolving it, and the first step here was to have the game generate a number of clue elements that should be present for a given quest step, and to check whether those elements have been indeed created. This seemed the most effective way to add maximum flexibility, but to ensure the game always make sure every piece the player needs to succeed is there (most obviously there need to be valid clues, but there also needs to be some kind of valid end trigger, as I just noted above).
So to do this, I’ve now created the foundations of a database which basically says “how many pieces are required for this part of this step?”. So, for instance, if the game decides to give you a single four-stanza riddles, then only one piece – that riddle – is required. Alternatively if it gives you a world map image and a local area stanza, or a local map image and a world area stanza, then two pieces are required. The generator knows how many pieces should be created, and after trying to generate each step, it puts the output into a set called clues if there is indeed anything to put in there, and then at the end of the generation, the game simply checks how many things are in there, and runs those back against the required number. If it’s the same, then it’s good to go – otherwise, it rolls the entire thing again. Within this, of course, each generator for a little piece also has a “keep trying until you succeed or until a certain number of tries have been attempted” function, and by combining both of these layers (i.e. trying until succeeding within each piece, and rerolling each piece a few times as well before then setting it to try until succeeding), I’ve found that we can get something successful generating a huge percentage of the time no matter what you’re trying to generate! Equally, though, the things that it submits for consideration might sometimes be just one thing – e.g. you find a note, and that’s the clue – but in the future it could be multiple things, e.g. there could be many ways to find a particular clue piece of information. It could be some historical data, for example, and maybe that shows up in lots of books, or lots of NPCs can talk about it? As such, the data architecture can now handle sets which can be switched out alternatively, and as long as the set in the final product is above a certain size, which the generator will also (later) be able to calculate, then it gives the thumbs-up on the quest. What I mean by this is this – let’s say that, maybe, a quest’s clue has a piece of historical information. It would then assess the intended difficulty of that quest, and assess how difficult the locations of that information are. It would then create a list of all the places that information can be found, and return them (instead of just a single note) to the function, and if that list is > 0, and the list is >= the minimum required number, then it’ll pass; otherwise, again, it’ll try again. This’ll be essential at world generation for main story quests, where it’ll be distributing important pieces of information across, say, many books, and then generating those books to contain the information in question. With these pieces in place, the generator is looking really robust, and ready to both function with current material and function with as-yet undeveloped material too!
Location Exclusions
The next thing I set about working on since the last entry was location exclusions. By this I mean that the quest generator needs to be able to look for specific kinds of areas and include them or exclude them in possibility based on appropriate metrics. So, the most obvious of course is that for easier quests the location you’re being pointed towards should be nearby, but this needed more to it. You might, for example, get a quest which I specifically want to keep within a given nation, so I need the generator to ensure no element of the quest strays outside the civilization in question; or the same could be true for the area that a particular religion or cult holds sway in. This might be the case as well for particular bioregions, so a quest that starts from a book about a particular species or a particular biome should obviously stick around where that species or biome is found for the sake of narrative sense and consistency. It should also be able to look for nations with particular policies, or look for certain terrain types, or features (rivers, lakes, roads, whatever) and correctly allow or disallow the quest location depending on what is being looked for. This means that in the future I’ll always be able to ensure that the steps in a generated quest are logical by constraining them, if need be, in quite specific ways. If you’re following a set of riddles laid down by a mycologist who discovered a strange new kind of mushroom, and they were from a particular nation, and there’s no evidence they ever left that nation, then everything should be focused on that nation, and probably ideally on or around the biome where the mushroom is found, and perhaps focused on any specific locations where the game might have decided it grows a lot, such as a particular lake or a particular mountain. Expand this to all possible quests or all possible sources, and you get the sense of why this matters. It’s also, as with so much else, important from a difficulty point of view as well, because exclusions allow us another way to keep a quest or a step of a quest close to home or close to the player’s location, but also allow the game to generate quests specifically designed to push the player to go abroad by ensuring all the quest plays out in regions the player hasn’t even discovered yet. Anyway, with all this said and done, there are now around fifty possible keywords that can appear in the exclusion list, and which the game can recognise, and check against a location, to see whether it fails any of them (in which case it’ll reroll and try again). This is important functionality alongside everything else in the quest generator to give the game a great deal of control over crafting quest threads which are interesting, are logical, and are appropriately difficulty-scaled, too.
A location which could never be chosen for a clue if the quest generator was using any or all of the “temperate” or “road” exclusion tags…
Location Rhyme Polish
Next, it became apparent that quite a bit of polish was, alas, still required in the “precise location” rhyming stanzas, i.e. those pointing to a specific map tile. As long-term readers will recall, these do this in one of three ways. The first is to have two sentences that describe the precise tile, so it might be rhyming (in a cryptic manner) a description of the name of the town on that map tile, and the name of the bioregion that town is in. The second is one sentence that is pointing directly at the tile (e.g. again the name of a town) but another that is giving its position relative to something else, e.g. north of a particular mountain that is then described in a cryptic manner. The third one is giving a fully-triangulated position just by its position relative to two things, e.g. north of a particular mountain and west of a particular lake (this of course can only point at one tile in a flat square-based grid, and the rhymes would then describe both of these in a cryptic manner and give the relevant directions in the process). These are all correctly producing rhymes, and sometimes really good rhymes as well, but they had two issues that I’ve resolved in the past fortnight. Problem #1 is that sometimes they produce lines which are simply far longer than the lines in any other stanza; part of this is unavoidable because they’re actually trying to get across quite complex information in a very short number of words, but in the other cases there were definitely particular sets of phrasings and synonyms and grammatical constructions that were adding syllables in, unnecessarily, to these generation outcomes. Problem #2 was simply a large number of slightly inelegant or less-than-ideal phrasings, which could take many forms, such as including the same word multiple times in a clue, or using phrases like “in this place” twice in a clue, and things like that. Those might sound very easy to deal with, but in fact they were rather challenging to remove, because the various parts of these clues – basically the core block of information, and then the rhyming addition – are not generated entirely alike one another, resulting in a situation where I needed to add some new code into the mix to communicate between these pieces in order to ensure that they weren’t repeating words in a displeasing manner.
So, for the first solution, I set about going over all the possible phrasings and words for this rhyming generator. This was Quite A Lot Of Work and largely involved me scrolling through some absolutely vast databases – thousands of components – of rhyming pieces and chunks of information the game can draw on in order to construct these riddles, and continually asking myself: can this be expressed a little better? Can I save a syllable or two here? If the answer was “yes”, I made the change, and then moved on and did this another thousand times. There was a part of me that said this wasn’t a priority for 0.11, but some of the riddles were just not up to my standard in terms of the fluency and smoothness of the writing, and there’s just no way around that. Equally, when I’m so close to release now, I don’t want something to feel like it falls below the quality of everything else, after I’ve put so much blasted effort into everything that’s coming together here. While time-consuming, and honestly quite dull, I do feel this was worth doing, and I’m genuinely actually a lot happier with the sentences that this system is creating now, with very few that are just too long and break any sense of rhythm or timing that you would expect in a piece of (generated) poetry. For the second solution, that was then a matter of adding some new code into the game which identified some of the common issues – such as two parts of a clue using the formation “X yet Y” rather than just one part – and, if it had been used in the first case, it shouldn’t then be used again in the second case. Because of the precise way this is all generated it proved quite tricky to ensure these fixes actually functioned, but ultimately after quite a bit of work I was able to ensure that the phrasings are not just a bit shorter and crisper, but also have fewer of these duplicated weirds or slightly uncomfortable sentence structures that had originally been emerging from just the sheer complexity of the generator. In truth, I don’t know whether anyone will or would have noticed the difference, but having been working with this material for so long, the difference is certainly clear to me – and I’m much happier with what we have now.
Where a rider into battle that glows cobalt walks a lane, two and two leagues east,
And a pouring of liquid metal stands like fire in the deep, seven leagues north and unpoliced…
–
Where one who follow paths that gleam carmine walks a track, south two leagues’ way,
And buildings for livestock are like an argent sprig of wood, ten and four leagues east, night and day…
–
Where knowledge is gained when those from many states are brought together, just to the east,
And a visionless fragment rests at the apex, southward at least…
“Easy” Difficulty Generation
This then brings us onto the next point – I’ve made the easy difficulty significantly easier. What I realised is that for a totally new player even the easy options are going to be a bit too tricky as they previously were before I came to; it might not be totally clear that it’ll be nearby, it might not be clear how the wordplay is being generated, and overall as I was testing just by myself (playtesters – expect an email this month, I hope!) it became clear that even the simplest were just a big step up from the tutorial university riddles. As such, I’ve made a pre-emptive adjustment to difficulty levels in the rhyming riddles, specifically by making them all a bit more likely to generate exact spot clues instead of “X is [direction] of Y” clues, even at the ultra level (though the ultra level will still tend towards the harder ones). Some very basic initial playtesting and consideration shows me that these are definitely not going to be trivial, so I think this will be a really important adjustment. As part of this I’ve also adjusted, as you can see from the above examples, the generator so that on the easy mode (and sometimes on normal mode) it will give you a precise distance! This means that if you’re confident about one element of the riddle you can find the location without needing to find the second; again, I think this is really valuable for the early game, so there’s now a range of ways the game can give precise distances instead of broad distances. As part of this, however, it soon became clear that generating easy-level riddles is going to be, by far, the hardest for the game. When one considers it, this is perhaps not actually all that surprising – an easy-level riddle is the most constrained, the most limited in what it can draw on, how much it draw on, how far from the location it can use information from, and so forth. It can only draw on stuff in a very small range of areas around a location to use as descriptors, and so to respond to this element of difficulty (you see how many factors need considering here?!) I’ve also added in the possibility for the game to, on the easiest difficulties, sometimes use larger structures as a means to point towards a location. What I mean by this is that normally the game won’t use a road, for example, because it has so many tiles, but on the easy difficulty it now can use things like roads, rivers, chasms, multi-tile mountains, and so forth, as a way for the player to zoom in on the general location while using the second half of the stanza to pin down exactly where to look.
More broadly, I am increasingly seeing that the “ultra” level riddles are going to be… really, really, really hard. They’ll only be coming in towards the end-game, by which time I’d anticipate the player having spent many tens of hours in the game world and should be pretty familiar with a lot of the names, but even then there are so many mountains, so many bioregions, so many roads and rivers… and when anything in the entire world could be talked about, that is going to be a difficult business. Again, playtesting will become the really important thing here, but I think for the ultra-level puzzles the difficulty might again have to be toned down a bit if the player is going to have any realistic chance of cracking them. While we don’t need to tone down the ultra difficulty for the physical maps, and we’ll have to see how the ultra difficulty actually comes into being for some of the other clue types, I’m not quite sure how to actually do this for the rhyming stanza clues without just simply dialing down the ultra difficulty to not use elements from anywhere on the map, but simply within a larger range than the “hard” difficulty; and/or the ultra clues would have to be coupled with something else to make them solvable, while still allowing for the generator to use elements from anywhere across the map. I’m not quite sure, but what I am sure about is that for 0.11 we’re only going to go up to the “hard” difficulty – so for the test generated quest I’ll have it make an “easy”, a “normal”, and a “hard” riddle, but I’ll leave the ultra ones out for the time being until we’ve got some playtesting and difficulty scaling on those others. I’m also keeping in mind a very thoughtful comment from a recent blog entry that having all parts of a puzzle scale with difficulty will actually stack on top of each other, i.e. making a normal perhaps 4x harder than an easy puzzle instead of, say, 2x. This is something again that I’ll need to be keeping an eye on as we move ahead…
The Other Two Stanzas
All of this then brings us to the two parts of these quests which are dealing with local information rather than world information, from which the game needs to generate either a map or a pair of rhyming riddle clues. For generating these there are two possibilities here, which I have defined in the game as “On It” and “Not Generated”. In the first case, this would be riddle or clue information being generated for the part of the game world the player is actually in at the moment it generates! This would not be the case for major quests which I suspect will all end up being generated at game start, but will be the case for small, one-step quests, that one might encounter as one goes around the world. In this case this generator – and the next one below – can take information that is currently loaded into the game, e.g. looking at the actual physical 200×200 tile local map that has just been generated and which the player is walking around on, and using that information directly to generate a clue. That could also be the case for a building the player is in, if when walking into the building the game decides it’s going to hide a mini-quest within that building. This possibility when generating local information is by far the easiest because the game doesn’t have to calculate what an area will look like when it is later generated, and nor does it have to set down strict rules for how it will be generated later on so that, when generated, it matches whatever is being talked about in the clue, and thus gives the player the illusion that that was always how it was. In the second case, this would be for creating quests at world generation, when none of the local maps have been generated, and the quest generator does need to interact with a great deal of abstract data in those two ways I’ve just described above. In some recent entries I’ve talked about implementing a system whereby the game can now essentially select a generative outcome in advance (far in advance, even) of that thing being generated, and when when generation time comes – e.g. the player finally sets foot in a building whose generation has been fixed by the game in advance because it wants to use that building in a quest – it will be appropriately generated within the added constraints. The point is that the generator needs to be able to take both options, and produce the required information for both options, which is quite a complicated thing. Or rather, ensuring they both funnel the right information into the quest generator is a complicated thing, since both of those options work entirely in playtesting, but aren’t yet a part of the generator.
Before going onto how I’d managed this, though, it’s worth mentioning that there was also a hypothetical third case as I was in development here, which would be for an area which has already been generated, but which the game wants to generate something new in. After extensive consideration I realised that this was going to be a huge time-drain on quest generation if done after world generation, because if it wanted to do something with a map grid that saved away, it would have to open that map grid just to consult it before closing it again, and that’s reasonably time-consuming (keeping in mind that each local map grid is 40,000 generated tiles, and each one has many different properties, and so forth). This would ultimately prove unacceptable, since it’s a non-trivial time consuming task for the game to open up a saved area when there’s so much detail in that saved area, and this would mean unmissable lag or delay during gameplay while this was done and that information was somehow used to generate something. Now, strictly speaking that could be hidden away alongside another loading screen, e.g. when you approach a world map tile which was going to generate information using another world map tile in this way, but it’s still adding in a ton of time… and ultimately I decided that was just unacceptable. So while this means a subset of quest steps cannot be generated in an area once you’ve generated that area but moved away (i.e. only those that require reference to a specific spatial thing that has already been generated and saved away), it will be a very finite subset (because any other quest in that area, e.g. talk to NPC X, will be completely fine), and the scenario will be very rare indeed (with all major quests probably being created at world-gen, and minor sidequests being made when you go into the map for the first time), so realistically I am extremely confident it’ll never really be noticeable by the player. The upshot of all of this is – local quest information can be generated either at the start of the game in such a way as to then force that generation, later on, to confirm, or when you actually generate an area the game can create a mini-quest in that area, which will be no more than fractions of a second of additional generation time. As development and the quest generator continues, both of these should prove more than sufficient – and it is indeed these which I’ll be moving onto next time to get the quest generator making the other half of the clues, i.e. these local ones. These local clues can already generate, but they do not yet generate as part of the quest generator, and so this is the second part of the quest generator I’ll be hoping to complete before next fortnight’s blog post.
Difficulty Scaling for Civilization Rhymes
Lastly for this week, I returned to generating the clues for civilizations – i.e. pointing the player towards the territory of a particular nation, within which they should then look for something that matches the next piece of information along – because these needed significant additional refinement before they might be ready to go live. In the previous version they were just taking all information about a nation, selecting two pieces that it could find a way to intelligently rhyme, and chucking it into the eventual stanza. This worked perfectly fine for resting, but the issue with this is that some pieces of civilization information are much simpler than others. A simple piece of information might be what kind of leadership the nation has, since the name of the nation can hint at that, spending time in the capital city will give that information, NPCs might readily talk about that topic, and so forth; a similarly simple piece of information would be religious policy as one can easily see that there are monasteries, or see that there’s a cathedral in the capital city, or whatever it might be. If you’ve ever encountered anyone from that civilization with weapons, you’ll know what weapons that civilization likes to use. By contrast, you might have information which you can only get when you’ve looked at their marketplace and noticed all the shops there, e.g. are they selling biographies or novels? Similarly, you might only discover what throwing weapons that civilization prefers when someone, er, throws one at you. This means I went back to this generator and ranked which civilization traits are allowable for different difficulties, with another complication here being unique or repeating traits – there might be two or three or even four nations in a given game which are theocracies, but only one nation with a unique civilizational trait. The unique traits therefore need to show up on the easier difficulties and the more common traits need to show up only on the higher difficulties, while also having those higher difficulties lean into the more obscure traits as well. One last thing which probably needs to be done is to make sure a pair is unique, and I haven’t coded that yet, but this element of clue generation now properly takes account of difficulty much like everything else does.
What next?
So, we now have the first half of the clues integrated into the quest generator, and most of the foundation for the second half of the clues, and a ton of other foundational work for the generator in terms of – in the future – having any possible set of inputs, puzzles, and outputs, all handling themselves correctly. This is huge progress, all things considered, especially because while the problem of a month ago seems to have largely resolved itself as a result of my putting in a lot of work to get things back to normal, in the past week I’ve actually been dealt another major health and physical wellbeing problem, with just as much of a disruption to my life as the other one. I’ve spent some time in Accident & Emergency and things are very tough right now. Despite this, I have (evidently) had a reasonably productive two weeks – or, rather, a productive one week before this hit – and I’m hopeful about being able to get the quest generator, at least in its 0.11 iteration, genuinely finished by next fortnight’s blog post. Once that’s done, it’s just testing and bug-fixing, and then it’s release time. We are moving forward, albeit at nothing like the pace I wanted for the end of the year – but that decision has ultimately been taken out of my hands by these two significant blows to my physical health, and consequently my overall wellbeing and ability to focus and get anything outside of my job work done. Nevertheless, we’re inching forward, and the next post should be a big one (hopefully). Thanks again for reading, everyone – please do let me know your thoughts in the comments, and I’ll see you all in a fortnight!