Custom Tokens


Introduction

-- Definition

-- Scope and Storage

-- Tokenized Access Concepts

Use Custom Tokens Because...

-- You want Dynamic Conversations

-- Your Journal can be Simpler and more Functional

Custom Token Mechanics

Where Custom Tokens Fall Short

Pouring Some Concrete

-- Avoiding Pitfalls

-- Token Duplication

-- Relocatable Block Allocation

-- Horse Market (Conversation)

-- Multi-part Quest (Journal)

Introduction

What are custom tokens? What are they used for? And how do I use them? Those are the questions this tutorial will explore. So let's start at the top with a basic definition of what they exactly are.

Definition

A custom token (e.g. <CUSTOM22>) is a special type of string variable that can be used to insert any dynamic information of your choice into blocks of text during gameplay. It is somewhat similar to the pre-defined tokens (e.g. <FullName>) which are used to insert specific game information into a conversation. What makes custom tokens different from other strings is how they are stored and used.

Scope and Storage

Most string variables, like parameters and variables defined in scripts, or local and campaign strings stored on objects, are identified using a name you make up. Custom token variables on the other hand are only identified by number. They don't have to be declared like other variables - there are just a set number of them, and they always exist. Unlike other variables, custom token #22 in one script refers to the same variable in every other script.

Let's remind ourselves how normal string variables work. When you declare a string variable in one script, then declare another string with the same name in a different script (or even in just a different spot in the same script), you get two entirely separate variables that don't interfere with each other. Setting the value of one has no effect on the value of the other. While local and campaign variables are significantly more global than script variables, they are tied to specific objects. With them you can access the same variable from different scripts, as long as you can find the object it is stored on from both places. A local or campaign string on one object and another with the same name on a different object will be separate entities, local to the object they are stored on. 

With custom tokens that is not the case. Custom tokens are the only truly global variables in NWScript. No matter where you reference them from, you always get the exact same variable. This is probably the reason why they are identified by number rather than by name.

Custom tokens work much more like local and campaign strings than anything else. They start out empty and wait to be given some text to store. You use a function call to change their value from a script, just like you do for local & campaign strings. Custom token values cannot be set in the toolset, only from scripts. This is not short-sighted design, but rather emphasizes their purpose and nature. As you'll see shortly, if you were to be able to set them in the toolset, they would not be the dynamic entities that they are. Setting them to a specific value in the toolset would effectively cause them to revert back to static text, rendering them useless.

Tokenized Access Concepts

Custom tokens are not accessed like other variables, either. In fact, there is (unfortunately) no function that will give you the contents stored in a custom token. Because of this, it is sometimes necessary to duplicate the contents in a local or campaign string variable in order for your scripts to be able to remember what was stored in the token. Instead of regular access methods, and this is where the token part of their name comes from, a special code or placeholder, also known as a token, is embedded into normally static text blocks like conversation dialog lines and journal entries. During play, when the text block is ready to be displayed, the game replaces the placeholder token in the text block with the current contents of the custom token string variable associated with it through its token number.

If you have made any conversations in NWN, you are probably already quite familiar and indeed comfortable with the use of tokens. There are many pre-defined tokens available in the conversation editor. These are the placeholders like <FirstName> or <Lad/Lass> that you embed into the conversation line to be replaced with the correct actual information during conversation execution. Custom tokens are exactly the same, but rather than having a limited pre-defined set of possible substitutions, they can be used to insert just about anything at all. Your scripts decide on the content to substitute in by assigning the token variable a value sometime before the text block containing the placeholder gets displayed. This is what makes them custom tokens.



Use Custom Tokens Because...

Okay, now that you have a basic understanding of what they are and why they are unique, it's time to examine a few common situations where they will come in very handy. Unlike other string variables, their potential applications are limited to conversations and journals. Still, there are hundreds of interesting ways to employ them that simply cannot be covered in a short tutorial. So in this topic a few of the most common useful techniques are overviewed.

You want Dynamic Conversations

Easily the most widely used and probably most obvious way to use custom tokens is exactly the same way you use the pre-defined tokens...namely, to modify static conversation lines to make them dynamically react to situational conditions. Based on the situational condition (the speaker is female) substitute the appropriate text (dynamically react by substituting the title "Lass") in place of the token placeholder (<Lad/Lass>). Being able to store any piece of information at all, and being driven by scripts capable of determining multiple situational conditions and responding to them with a limitless variety of potential substitutions, makes custom tokens far more flexible than the pre-defined ones, which are limited to only one or two different replacements and are based on a single condition. You can even have conversation lines that consist entirely of just a custom token placeholder and have your scripts generate entire lines of the conversation on the fly simply by plugging it into the proper token variable.

A very common task, for which this "entire line replacement" idea works well, is for making dynamic lists players can choose from. For instance, select one of the other players currently online or in your party. You create a conversation with an overly long list of PC responses that all consist of a just a custom token placeholder. Each line references a different token number. You fill up the token variables with the names of all the other players from a script whenever the conversation runs (i.e. before the conversation branch is displayed). Lines associated with empty tokens can be suppressed using a TextAppearsWhen script. What you end up with is a choice list of other players that will dynamically change whenever the branch is displayed. 

Another common practice is to use custom tokens to substitute statistical information into dialog lines. For example, you need 3 items to finish the quest, or your current skill level is 22.

Your Journal can be Simpler and more Functional

The next idea we'll look over is custom token use in journals. Writing journal entries using custom tokens may be one of the most underappreciated features in NWN, because it can save you lots of effort, greatly simplify your journal, and you can do some very cool things with it that just aren't possible any other way. However, custom tokens work slightly different when used in journal entries. I think the best way to illustrate the difference is by contrasting it with a conversation usage.

For simplicity sake we will assume this is a single-player module. Okay, let's suppose we have a signpost placeable with a conversation on it that simply pops up a description of some sort, with a Close option that ends the conversation. When the player clicks it, we want to include something dynamic into the descriptive text that appears. So in the toolset, we edit our signpost conversation and insert a custom token placeholder into the descriptive text at the point where we want the substitution to be made. Next, we use some script somewhere, let's just say we use the same OnClick or OnUsed event that starts up the conversation, to change the token variable associated with the placeholder we inserted. Storing into it the proper substitute text we want. So now, what happens during play?

First the appropriate string is stored into the token variable when the player clicks the placable just before it kicks off the conversation. When the conversation starts, the game "builds" the window to be displayed. Then, when it gets to the box where the description needs to be rendered, it grabs the text of the conversation line from its blueprint, then scans through it looking for tokens to substitute. As it finds them, the appropriate text substitutions are made and then it renders the final resulting text block into the window. During the substitution phase, it handles all the pre-defined tokens automatically behind the scenes, figuring out what to substitute and inserting it in. When it gets to a custom token, it reads the current value stored in the associated token variable, and puts that in. At this point the displayed window has a version of the conversation line with all token placeholders resolved and removed. The conversation itself however still has a text block for the line in it that contains all the token placeholders. In other words, the dialog in the conversation blueprint did not change. The version being displayed has no tokens in it. There is no way to determine looking at it where the token placeholders were originally positioned within it, what they got replaced with, or even how many were there -- if any.

The process of doing the dynamic substitutions is sometimes referred to as "baking" the dynamic info into the target block. The resulting detokenized text, like what is displayed in our example conversation window, is called the "baked" version. Why is this important? Well suppose that the game changed the dialog in the conversation's blueprint to the baked version instead of leaving it alone. The next time the signpost was clicked there would be no tokens to replace. What would appear is the same thing that appeared the very first time the thing was called up because that's when the token placeholders were baked out. So a baked version cannot be "re-baked" again to substitute new info, or "de-baked" to return the tokens where they were. While an unbaked version can be used over and over again, never changing, to generate the text with the most recent info baked in. This is totally irrelevant for our placeable example because for us, the baked version is discarded every time the conversation is closed and it gets regenerated each time the conversation is run.

With journals however, baked versions are important. Our signpost has that conversation which tags along with it and when the conversation starts up, it always grabs the unbaked conversation line and bakes in the info before displaying it. Journals aren't like that at all. Every player does have his own journal, but the full total journal itself is a single entity stored in the module. Player journals contain only a subset of the entries in the main journal file. When you embed custom tokens into your journal entries from the journal editor, the unbaked version is stored in the one and only journal file -- not on every player. With placeables (and creatures and doors), you can set up a conversation in the object properties after painting it. Two copies of the same object can be painted down and maybe one's conversation has a token in it and the other doesn't, or one uses token #22 while the other uses #75. With journals all players get them from the central repository where there is but a single unbaked version of each entry. So all players will be using the same tokens and the same unbaked entry.

Going back to a multi-player mindset, if two players get the same journal entry and the dynamic info embedded in it is different for each of them, what gets stored? Well they get baked versions inserted into their journals. The unbaked version remains untouched in the main journal file. In the conversation example the baked version was always discarded...with journals, the baked version is saved and is what goes into each player's journal. Every player carries around his own baked version of each entry he's been given - baked at the time he received it.

This is great! Because otherwise every time you opened your journal to look at it you would see it changing based on what other players are doing that is affecting the custom token variable values. Remember the token variables are global, so all players share the same ones. If you change one on behalf of one player, all the other players will see the new value. And, since they all get the unbaked version from the same place, all the dynamic portions of their journal entries will be based off of the same token variables. Therefore the baked version is exactly what you want in a journal entry -- the state of affairs when the entry was originally obtained.

However, it also means the entry will not change when you do want it to update. You must keep in mind the intended purpose of journals is really to just track quests. Baked journal entries are excellent for quests, but not so hot if you want to use them for some kind of more dynamic current-status-in-the-server type of entry that updates regularly. To do that for journals, the entry has to be removed and re-given to the player so the most recent info can be re-baked and a new baked version generated to replace the old one. Maybe a little extra work on the scripting side, but it allows you to use a single journal entry in your journal editor, have it look different for each player, and change during play to represent new situations. To do that without custom tokens would require tons of static journal entries for every possible outcome, and for some things could not be accomplished at all.

Imagine being able, in a multi-player setting, to give out journal entries to a party of players, and have it list right there in their journals the names of everyone who was in the party when the quest was completed -- including their classes and levels -- and doing it with just one single journal entry in your journal editor! There is no realistic way to accomplish that using just static journal entries. These are the kinds of things custom tokens used in conjunction with journal entries can give you.

Custom Token Mechanics

The mechanics used to implement custom tokens in your module are so simple it hardly warrants a separate tutorial section. There are only two things you need to know: how to write a custom token placeholder into a text block, and how to change the value of a custom token variable in a script. As pointed out earlier, tokens are identified by number. To write a custom token placeholder, you simply use this syntax <CUSTOMx> where 'x' is replaced by the token number. For example, <CUSTOM12345> is the token placeholder you embed in the text block to refer to token variable #12345. Token numbers can range from 0 up to 2147483647. So there are plenty of different tokens available for use.

Officially, custom tokens 0-9 are reserved for Bioware default scripts. So any you use should be at least 10.

The OC uses 0, 104-106, 1000-1007 and 10232 for plot purposes, but you can ignore this unless you're modifying the OC.

In practice, Bioware scripts use some other codes, which are therefore best avoided. They're in the crafting system (unless stated otherwise). The restricted codes are 777, 2001-4, 9323-4 (Deck of Many Things), 9611-4, 9711-4, 13220-5, 14220-5, 14320-5, 14420-5 and 77101-9 (multiple henchman system).

Custom content packages sometimes use them too, so when you are using custom content and custom tokens, you should take the time to examine the custom content scripts so you'll know which token numbers to avoid if any are used there. Getting token collisions between two separate subsystems can be awfully hard to recognize -- trivial to fix, but hard to find.

Changing a token variable's value is equally simple. Just call the SetCustomToken function passing in the token number and string value as parameters like so:
SetCustomToken( 12345, "Some text");
which will set the contents of token variable #12345 to the string value "Some text".



Where Custom Tokens Fall Short

While custom tokens are an excellent tool, they do have their failings. Yes, they are global, making them very easy to use from anywhere, but they are not persistent at all. Nor is there a function in NWScript that will allow them to persist. Token persistence can only be achieved through manual scripting methods, and since there is no function to retrieve the value of a custom token, any persistence method used will require some additional means of duplicating their values in a place where they can be easily persisted from - like local variables. Another major problem you'll run into trying to persist them is the fact that there are so many possible that you cannot simply loop through them all. You'll also have to keep track of which ones have been persisted so when you restore all custom tokens from the database at module load time, you can know which token numbers need to be restored. This tracking of which ones have been stored will also have to be persisted -- which naturally further complicates the process. Employing named constants, a token duplication strategy, and allocating your tokens in blocks (see the Avoiding Pitfalls section below) will go a long way towards alleviating these complexities.

Custom tokens cannot be "double-baked". What I mean by that is you cannot embed a token placeholder into the string value set into a different token variable and expect the token you embed to be resolved along with the one you embed it into. If you try this, the resulting baked string will have an unresolved token placeholder in it. For example, if you have custom token placeholder <CUSTOM3853> in some conversation line and set its value to "Hello <CUSTOM7273>", it will not result in any resolution or baking of the embedded 7273 token after the 3853's substitution is done. Your conversation line will always say "Hello <CUSTOM7273>" regardless of what value you set custom token 7273 to. In fact, there is no way in a script to generate a baked result so you could not even do a double-bake manually. This limitation applies to all tokens including the pre-defined ones, not just custom ones.

The way to work around this is to use two regular strings, one containing "Hello" and the other containing "World", then concatenate them as "Hello World" before setting the custom token value.


Another drawback with custom tokens is the fact that token placeholders are static. It is impossible to insert a token placeholder somewhere that dynamically changes to reference one of several different tokens based on some condition. When you use a placeholder you have basically reserved a use for that token number throughout your module and only that one token number can ever be used there. The only way to change what token number variable is referenced by a placeholder is to use the toolset to change the token number of the placeholder embedded into the conversation line or journal entry. The token number part of a custom token placeholder cannot be variable.

Of course, referring to things by number is much harder to keep track of than when names are used. Because of this, and the fact that custom tokens are only accessible through the use of a token number, they are difficult to manage and maintain. You can simplify the code side of things by using the const statement to associate a name with a numeric value, and then use the constant name in all your custom token referencing scripts. However, the placeholder side has none of that flexibility. You always have to use a token number in the placeholder and there is no way to associate a name that can be used instead. So connecting the variable numbers with their purpose and particularly tracking them all down later if you must change things around can be a chore.



Pouring Some Concrete

Sometimes the easiest way to learn is through studying real tangible situations. So this section illustrates custom token use with some concrete examples.

Avoiding Pitfalls

There really is not very much to worry about when using custom tokens. 


They are very simple to use, but, as we have seen, once given a value you cannot determine what that value is. This limitation can get in the way when you are doing things like building a dynamic list for a conversation. Since it is dynamic, once the player makes his choice your scripts will no doubt need to know what text they clicked on. 


The only other big problem you can run into is token collisions. This happens when two subsystems interfere with one another because they are trying to use the same custom tokens for different purposes. Because custom tokens are global, the two systems will collide with one another over what the content of the token(s) should be. To resolve this, one system or the other must have its token numbers changed so they don't interfere. With all the token numbers available for use, token collisions are rare, but they do happen, so being prepared for them is only prudent. The two practices introduced in this section can go a long way towards preventing or at least mitigating the impact when these issues arise. Neither is a requirement, but following sound practices like these will help to safeguard you in the long run.

Token Duplication

Sometimes, if you use custom tokens, you will find yourself wishing for a GetCustomToken function. This function does not exist, however. 

So, if you know you'll need to remember a token's value for later use in a script somewhere, there is an easy way to do it. The idea is to duplicate every token as a local string variable stored on the module object. When you need to know the token's value, you just grab it from the module's copy. 

This works well, as long as the variables on the module stay synchronized with the tokens. The easiest way to ensure that happens every time a custom token is changed would be to write a custom wrapper for the SetCustomToken function, then use the wrapper function exclusively to change your custom token values. 

The wrapper would take care of setting both the custom token variable and the duplicate saved on the module. 

Just to be consistent, you might as well write a sister function to retrieve the custom token's value. You won't always need to get a token value, but if you follow this duplication practice rigorously, they will always be there when you want them, whether you need them or not. It is also very useful when debugging to be able to print out token values in a log entry, speakstring, or sendmessage, so even if your code may not need it, you can see how having it there can help.

So to implement a token duplication scheme, you'll want a library script to centralize the two new functions. Use the #include directive to add it into any script needing to use custom tokens. Inside the library is where the new SetCustomToken (and GetCustomToken) function is defined. But you can't use that name because Bioware is already using it, and you aren't allow to redefine it. Instead we use a wrapper to basically give it a new name and add in the extra duplication stuff we need done. Here is one way to do it:

    // Custom Token Library script - custom_token_inc
//::////////////////////////////////////////////////////////////////

// void SetCustomTokenEx(int iToken, string sValue)
// Sets the custom token identified by the number in iToken to the
// string value specified in sValue. Also duplicates the value as a
// local string stored on the module object so the GetCustomTokenEx
// function can retrieve it at any time.
void SetCustomTokenEx(int iToken, string sValue);
void SetCustomTokenEx(int iToken, string sValue)
{ if(iToken <= 0) { return; } // Change the custom token variable and duplicate it. SetCustomToken(iToken, sValue); SetLocalString(GetModule(), "CUSTOM" + IntToString(iToken), sValue);
}


// string GetCustomTokenEx(int iToken)
// Retrieves the current value of the custom token identified by the
// number in iToken by reading the local string variable containing
// the duplicated token value which was stored on the module object
// by the SetCustomTokenEx function. If the custom token was set using
// the default SetCustomToken function instead of SetCustomTokenEx,
// this function may return an incorrect or blank string.
string GetCustomTokenEx(int iToken);
string GetCustomTokenEx(int iToken) { if(iToken <= 0) { return ""; } // Return the content of the module variable being used to duplicate
// the token variable.
return GetLocalString(GetModule(), "CUSTOM" +IntToString(iToken));
}

Simply save that library, then use the #include directive to gain access to the functions from your scripts. Make it a point to always use the SetCustomTokenEx function instead of Bioware's standard SetCustomToken version, and you will have automatically synchronized token duplication. The other function, GetCustomTokenEx can be used to read token values. 

This idea is not completely bulletproof, since you can always set a token using Bioware's SetCustomToken function - GetCustomTokenEx, being unaware of the change, would return the wrong string. Getting in the habit of always using the *Ex versions (or something equivalent) will keep that from happening. 

If you want a persistent version, you can use the campaign functions for the duplicating part. Be aware, if you make the duplicates persistent using the campaign functions, the token variables themselves will not automatically persist along with them. Therefore your OnModuleLoad event must restore the actual token variable values from the copies stored in the database in order to resynchronize everything at module start-up time.

Relocatable Block Allocation

Next let's tackle the token collision problem. If you are using a fair number of custom tokens and find there are collisions with another subsystem or custom content you've added, it can be a lot of work to straighten it all out. Not because it is difficult, but because you have to track down all the custom token use in your code, conversations, and journal entries in order to change the token numbers used. And being a repetitive task it is prone to errors as well. You can mistype a token number or mix them up so they reference the wrong variable etc.

Nothing can be done about the placeholders embedded into the conversation lines and journal entries, all those changes always have to be made manually one at a time. But the code changes that will be necessary can be vastly reduced by defining and following a token allocation scheme designed to make it easy to relocate your tokens from the code's point-of-view simply by changing one line. By token allocation I mean deciding which specific token numbers your system will reserve, or 'allocate' for use.

Since custom tokens are often needed in groups, and it is more efficient to access a group of tokens in your code using a loop, typically you will end up reserving or allocating them in sequential blocks of token numbers. It is very wise to do this whether their use calls for it or not because if they are all in a sequential block it makes things much more straightforward when you have some overlaps and need to relocate them. So whenever you are laying out the token numbering you'll be using, always do it in a sequential block of numbers.

With a block of tokens decided on, you can start writing the code and placeholders to set and access them. But rather than specifying the token numbers explicitly, they can be referred to by 'block address' and 'offset' -- at least in the code. The idea is to define a constant to keep track of the first token allocated in the block you want to use (i.e. the block's 'base address'). Then each of the individual tokens in the block are assigned an offset constant which not only gives each token a name, but also identifies where in the block it resides -- but not explicitly, rather as an offset from the base block address. In the code, whenever you need to access a token to change or read it, you can add the block base and offset constants to compute the actual token number that will be used. 

How will this help with collisions? Well if your code is not written using hard-coded token numbers, then they won't require edits if the tokens have to move around. All you have to do is change the one line that has the block's base number in it and all the code adjusts automatically because it is all calculated rather than hard-coded. As you can imagine this makes changing a huge number of token numbers far easier to do -- at least in the code side of the house. You can move entire blocks around simply by selecting a new base address for it.

To further illustrate how this would work let's look at an example. Suppose we are adding in some stuff that's going to be using custom tokens for three purposes, player statistics, a dynamic party selection list, and a dynamic portal destination list. The statistics will be embedded into journal entries and conversation lines, and the selection lists will be utilized by other subsystems to let players select party members or portal destinations from a conversation list. The party list conversation is going to have a ten line branch for its list, so it needs ten tokens. And we'll reserve another five for future use. Likewise the portal list needs a block of twenty destination tokens and a future reserve of ten more. The statistics are just five individual unrelated items that each need a separate token. Keeping to the strategy of block allocation, this gives us a block of fifty tokens to allocate. The first ten will be for player names, then five unused but reserved for later, then twenty portal destinations, followed by a reserve of ten more, and finally the five statistics tracking tokens. Let's see how we can write our code to make this block easily accessible and relocatable by defining some constant names.

First we give names to the two lists and each of the statistics tokens we'll be adding. The lists will be accessed using a loop so there is no need to define separate names for each item in the list. You could do that if you want to refer to them specifically outside a loop, but for our purposes in this example, they are just treated as a sub-block. Therefore we are just assigning a name to the whole sub-block and the code will add a loop variable offset to get to each one. Again, we need a new library to centralize these constants. The same library used for duplication shown in the previous section could be used for this purpose. In fact it would be extremely smart to do that so all your custom token stuff is all in one place. The constants for our example might look like this:

    
// Custom Token Library script - custom_token_inc
//::////////////////////////////////////////////////////////////////

// Custom Token Constants
const int TKN_BASE = 6000;

const int TKN_PLAYER_LIST = 0;
// ten tokens used for the list = 0...9
// five reserved for future use = 10...14
const int TKN_PORTAL_LIST = 15;
// twenty tokens used for the list = 15...34
// ten reserved for future use = 35...44
const int TKN_STAT_KILLS = 45;
const int TKN_STAT_DEATHS = 46;
const int TKN_STAT_HEADS = 47;
const int TKN_STAT_CROWNS = 48;
const int TKN_STAT_JEWELS = 49;


// ...
// definitions for Get/SetCustomTokenEx go here
// ...

Examining this code we see at the top a constant for the base address of the block containing all the tokens we plan to use. This is an actual token number. Under that are constants for all our tokens, but their values are set to indicate their offset into the block. The player list is at the top of the block so its offset is zero. Fifteen tokens later we find the tokens used by the portal list, etc. If you needed to you could set constants for each list item:
    const int TKN_PORTAL_LIST_0 = 15;
    const int TKN_PORTAL_LIST_1 = 16;
    ...
    const int TKN_PORTAL_LIST_10 = 24;
but for this example it isn't required because we use loops so we only need to know where the first one is.

Next let's see how coding would change to make use of all these constants. What do we need to do with tokens in script code? Either set or get...those are the choices. What follows are some examples of how we would get and set token values in our block using the constants just defined:

    // A script somewhere in the module
//::////////////////////////////////////////////////////////////////
#include "custom_token_inc" ... // Set the player's number of kills to "355"
// Note this is an individual separate token not part of a list.
SetCustomTokenEx(TKN_BASE + TKN_STAT_KILLS, "355");

// Get the player's number of crowns string sCrowns = GetCustomTokenEx(TKN_BASE + TKN_STAT_CROWNS);
// Set all player list tokens to blank
// Note this sets ten different tokens in the list, one at a time.
int iNth;
for(iNth = 0; iNth < 10; iNth++) { SetCustomTokenEx(TKN_BASE +TKN_PLAYER_LIST + iNth, ""); }
// Get the 4th portal destination string sDest4 = GetCustomTokenEx(TKN_BASE + TKN_PORTAL_LIST +3);

...

Now how does it work? Okay, take a look at the lines that are referencing the token number. Instead of a hardcoded number there is a computation. It is taking the TKN_BASE value, adding the offset constant for the token we're after, and, if necessary, adding another offset into the sub-lists. The base number for our example is 6000. So the first line is going to take 6000 add the value for the kills constant (45) to get token #6045. The next line uses a similar computation to get the crowns token sitting in our block at #6048. The next part uses a "for" loop to iterate through all ten tokens in the player list and set them to blank. It uses the same computation but adds the iNth loop variable which acts as an additional offset that selects into the player sub-list of our whole block. The value of iNth starts at 0 and goes to 9, so the loop will alter tokens #6000 to #6009, which are the first ten tokens in the block and exactly what we want. Finally, the last line is extracting the 4th destination from the portal list using a similar computation. The first destination in the list is at offset 0 (within the list) so the fourth one is #3. Thus, we get the value of token #6018.

Ok great, so the math all works out -- what does all this gain us? There are two things that should be very apparent straight out...you are no longer looking at token numbers in your code lines, you now see names. And second, the library section where the names are assigned is a very good source of documentation for every single token we are using (especially if they are named well). But even more importantly is look how easy it is to change the token numbers we want to use if we find that some other system we are using happens to also be using tokens in the range 6000-6049. To move everything over to, oh lets say, start the block at token number 754676, all we have to do now is change the TKN_BASE value in the library to 754676 and recompile all our scripts. One change on one line and every single location in all our code module wide is now updated to use the new token numbers. Yes we still have to go out to all our conversations and journals to change the numbers in the custom token placeholders, but at least our code is simple to maintain. Surely you can recognize the utility of this strategy. Once again, this isn't required stuff, it is just a sound practice to adopt.

I just have one last point to make concerning relocatable block allocation of your tokens. You might be tempted to simplify things even more by incorporating the base constant for the whole block (TKN_BASE) directly into the *Ex functions. If you do that you would only need to specify an offset name for the iToken parameter in all your function calls so they'd be much cleaner because they no longer need the "TKN_BASE +" bit at the front. 

I would discourage doing this depending on how many tokens you are using and here's why. If you are using lots and lots and lots of tokens, the block will be huge. When you incorporate the base computation into the *Ex functions you are imposing a restriction saying all your token variables have to be organized in one single huge block in exchange for simpler function calls. By keeping it separate like the example above shows, you can break up the organization of your tokens into multiple blocks. They each get their own base constant to add in thus making each smaller block relocatable independent of the others. Smaller blocks also have less chance of colliding with other systems and the ability to move smaller pieces around may turn out to be easier. 

Block reorganization is also simplified. In our example if we wanted to change the list size for the player list to 50 tokens, every single token in our system would have to be changed to make room. Easy as pie in the code, just change all the offsets around on our token names and recompile. But you'll feel it when you remember you still have to go change all those damn placeholders out in the conversations and journal entries. Allowing for multiple disparate blocks therefore also significantly reduces the impact when you need to reorganize your tokens in the block. If some tokens can safely stay where they are it won't be so bad. It really depends how many tokens you are using and what you prefer to see in the code. The important thing is to eliminate the hard-coded numbers.

Horse Market (Conversation)


Here's an example that uses custom tokens in conversation.  It allows the player to choose an object from a list - a horse, in this case.


We don't know in advance how many horses will be available, or what their names are, so the list can't be hard-coded.


Let's assume for simplicity that there can never be more than five horses.


The conversation looks like this:

[OWNER] Which horse would you like to buy?
[PLAYER] <CUSTOM2301>
[PLAYER] <CUSTOM2302>
[PLAYER] <CUSTOM2303>
[PLAYER] <CUSTOM2304>
[PLAYER] <CUSTOM2305>
 

We're going to need a list of horses. This is done in the "Action Taken" script on the first line:
// Horse market conversation - initialisation
#include "x3_inc_horse"

void main()
{
   object oPC       = GetPCSpeaker();
   object oArea     = GetArea(oPC);
   object oHorse;
   int    n         = 0;

   // Initialise a counter which is used in the Player conversation options
   SetLocalInt  (oPC, "ptDialogLine", 0);

   // Store a list of horses in the area which are for sale as a local string array on the PC
   oHorse = GetFirstObjectInArea(oArea);

   while(GetIsObjectValid(oHorse))
   {
      if(GetObjectType(oHorse) == OBJECT_TYPE_CREATURE)
      {
         if(HorseGetIsAMount(oHorse))
         {
            if(GetLocalInt(oHorse, "X3_HORSE_NOT_RIDEABLE_OWNER"))
            {
               ++n;
               SetLocalObject(oPC, "bhMarketHorse" + IntToString(n), oHorse);
            }
         }
      }
      
      oHorse = GetNextObjectInArea(oArea);
   }
}

Note the counter ptDialogLine which we set to zero. This is a little trick which allows us to use the same "Text Appears When" script as the condition for each of the player lines:
// Horse market conversation - list the nth horse the PC can buy as a conversation option.

// n - the conversation option line number - is determined by incrementing a counter.

// If there are less than n horses, the script returns FALSE, so that the line doesn't appear.

// The list of horses has already been stored as a local string array on the PC.

const int CUSTOM_TOKEN_OFFSET = 2300;

int StartingConditional()
{
    object oPC     = GetPCSpeaker();
    int    nOption = GetLocalInt(oPC, "ptDialogLine") + 1;
    object oHorse  = GetLocalObject(oPC, "bhMarketHorse" + IntToString(nOption));

    // Store the new counter value for use by the next conversation line
    SetLocalInt(oPC, "ptDialogLine", nOption);

    // Set the custom token that will appear as the conversation option
    if(GetIsObjectValid(oHorse))
    {
        SetCustomToken(CUSTOM_TOKEN_OFFSET + nOption, GetName(oHorse));
        return TRUE;
    }

    return FALSE;
}

If there are three horses for sale, named Rag, Tag and Bobtail, the conversation looks like this in-game:
[OWNER] Which horse would you like to buy?
[PLAYER] Rag
[PLAYER] Tag
[PLAYER] Bobtail

We could add "Action Taken" scripts to each of the player lines to identify the horse object selected and assign it to the player, but that's beyond the scope of this tutorial! 



Multi-Part Quest (Journal)


Here's an example of using a custom token to simplify a journal.


The player is invited to arrange a Royal marriage. Experience points are awarded for finding any solution, with bonuses if the best possible outcome is achieved.


There are 2 eligible husbands and 2 suitable wives, so there are (2 x 2 = 4) possible unions.


The bride may agree to the arrangement, or be forced into it by her mother.


Her brother may be persuaded to make it a double wedding, or not. 


Furthermore, the player may or may not succeed in resolving an underlying religious issue.


So, there are (4 x 2 x 2 x 2 = 32) possible outcomes.


Once the player has negotiated a potential solution, they have opportunities to improve the outcome by further discussion with the parties, before committing to a marriage contract.


This could be implemented as 32 journal entries, but using a custom token allows us to use just one journal entry for all 32 outcomes. It looks like this:

 King Rupert II has agreed to a Royal marriage. The proposal is that <CUSTOM14> When I'm ready, I need to inform Queen Maya, the Queen Mother.

Conversation action scripts (which need not concern us here) record the latest marriage proposal by setting local integers on the PC.


Every time the situation changes, the journal is updated by the following script:

// Write journal for Royal Marriage proposal

void main()
{ object oPC = GetFirstPC(); // This is a single player module string sText; string sHusband;

// The following lines build the text we want to put in the custom token if(GetLocalInt(oPC, "RoyalPrince")) { sHusband = "the handsome young Prince Rupert"; } else { sHusband = "he"; }

if(GetLocalInt(oPC, "RoyalAstra")) { sText = sHusband + " will marry Princess Astra.";

if(GetLocalInt(oPC, "RoyalAgreed")) { sText = sText + " She is delighted at the prospect."; } else { sText = sText + " She won't like it, but she doesn't have a choice."; } } else { sText = sHusband + " will marry Princess Morcasta. She would rather marry Sir Palin, but she doesn't have a choice."; }

if(GetLocalInt(oPC, "RoyalPenric")) { sText = sText + " King Penric II will marry Princess Virginia."; }

sText = sText + " King Penric must renounce all claim to the throne of Morovia";

if(GetLocalInt(oPC, "RoyalReligion")) { sText = sText + " and guarantee freedom of religion on Varota." + " Floristan will renounce all claim to the throne of Varota."; } else { sText = sText + "."; }
// Write the text to the custom token SetCustomToken(14, sText);
// Write a new journal entry. Any existing journal entry must be removed first, because the custom token is baked in. RemoveJournalQuestEntry("jt_royal", oPC); AddJournalQuestEntry("jt_royal", 1, oPC, TRUE, FALSE, TRUE); SetPanelButtonFlash(oPC, PANEL_BUTTON_JOURNAL, TRUE);
}

If the player agrees the least favourable solution initially, the journal reads:

 King Rupert II has agreed to a Royal marriage. The proposal is that he will marry Princess Morcasta. She would rather marry Sir Palin, but
she doesn't have a choice. King Penric must renounce all claim to the throne of Morovia. When I'm ready, I need to inform Queen Maya, the
Queen Mother.

If the player subsequently negotiates the best possible solution, the journal reads:

 King Rupert II has agreed to a Royal marriage. The proposal is that the handsome young Prince Rupert will marry Princess Astra. She is
delighted at the prospect.
King Penric II will marry Princess Virginia. King Penric must renounce all claim to the throne of Morovia and
guarantee freedom of religion on Varota. Floristan will renounce all claim to the throne of Varota. When I'm ready, I need to inform Queen
Maya, the Queen Mother.

The same outcome could be achieved by using multiple custom tokens in the journal, if you prefer to reduce the amount of string concatenation in the script.