Utility Systems

Please tell me how much this site helps you with a number from 0.0-1.0

In a utility system we calculate the utility, or usefulness of each action. It's a normalized value that allows us to compare the actions with each other and pick the one that looks the best right now for the agent. There are two major steps do this and they are data processing and decision processing.

Data processing

This step takes the state of the world and the agent and map them to normalized utility values. This values are called concepts and can be modified and combines to form new concepts.

Example: The number of enemies around the player can be normalized to the concept of threat. The more enemies around the more threat the agent will feel. At the same time we have the concept of homies that is the normalized amount of friends around that make the agent feel more safe. These two can be combined into a new concept called anxiety that can then be used in the flee action to see if the agent panic's and runs for it.

Collect State

The state data need to be collected and it can be expensive so that's why data processing is a separate step from the decision processing. This can run every n frames or update when event's about state changes are revived. Some data will also be older then one frame no matter what, ex is ray casts.

Normalize

This is done with graph functions. Ex ones are Linear, Quadratic, Logistic or Logit.

Combine

Concept values can be combined in a number of ways. The most easy one is an average or multiplication.

Decision processing

The decision process calculate the utility for each action and then use some way to pick the one to perform.

    • Highest: Pick the action with the highest utility.

    • Weighted Random: Pick a choice at random with higher chance for those with high utility.

    • Weighted Top n: Off the top n actions pick one at random with higher chance for those with high utility.

Reference