DevLog 09: How the AI Corporations Think in the Fragile Allegiance Remake

Greetings, Commanders.

This one is long and fairly technical. Game AI is the part of this project I enjoy most (it was the topic of my thesis), and the rival corporations changed a lot over the last few weeks. In DevLog 06 I wrote about the HTN setup and fleet rallying. This post goes further into how the AI decides what to do, how each faction is different, how it designs ships, and how I test all of it without playing a thousand games myself.

What I Want from the AI

The rival corporations are your main opponents, and I want two things from them that pull in different directions. The fights should be fair, and the games should still be hard.

Fair means the AI plays by the same rules you do. It shouldn't have access to any hidden information. It should only know what its sensors and ships can see, the same as you would in its place. It pays for everything it builds and has to manage its money. Higher difficulties don't give it bonus income or let it see more than you do. I'm honestly not sure whether the original AI cheated. Either way, I don't want mine to.

The easy way to make an AI challenging is to let it cheat, so without that the challenge has to come from it playing well. It should expand at a good pace, keep its economy healthy, build ships that make sense against yours, and attack when it's actually ready instead of throwing ships at you one at a time. Most of this post is about the pieces that make that happen.

I also want each corporation to have its own personality. After a few games you should know that one of them tends to dig in and defend, and another one comes at you early, and plan around that. And the AI should react to what you do. If you build your fleet a certain way, it should notice and adapt, like a human opponent would.

Getting all of that to hold up over a long game is the hard part, which is why the last section of this post is about testing.

How the HTN Planner Works

In DevLog 06 I introduced the HTN (Hierarchical Task Network) planner. Here is how it is actually put together.

The Layout

Every AI player is made of several agents, and each agent owns a list of tasks:

AI player
├── Strategy agent (one per player)
│     DeclareWar, SueForPeace, PickWarTarget, BuySciTek
├── Fleet agent (one per player)
│     FleetColonize, FleetDefend, FleetIntercept, FleetService,
│     FleetRally, FleetFormGroup, FleetAttack, FleetAdvance,
│     FleetScout, FleetPatrol
└── Colony agent (one per colony)
      SellMinerals, BuildResponseFleet, BuildBuildings, BuildVehicles,
      BuildMissiles, BuildSatellites, LaunchMissiles, LaunchSatellites,
      IncreasePower/Air/Food/WaterProduction

The strategy agent decides who the AI is fighting and what it researches. The fleet agent moves ships. Each colony runs its own agent that builds, sells and launches from that asteroid, so a corporation with five colonies has five colony agents working in parallel.

The Design

A task has four parts:

• Preconditions that must all be true before it can start. FleetAttack needs an idle combat ship, a hostile asteroid in range, and a consolidated fleet.
• An action, which is what the task actually does when it starts, like ordering ships to a target.
• Rules that must stay true while the task runs. Most tasks only check that the player or colony is still alive. If a rule breaks, the task fails.
• Finish conditions that decide when the task is done. LaunchMissiles, for example, stays active until the launch queue is empty.

Each task moves through a few states: not started, in progress, complete or failed. When the AI thinks, each agent walks through its task list. If a task is in progress, the agent checks its rules and finish conditions and stops there. Otherwise it tries to start the next task. If the preconditions fail, the task is marked as failed and the agent tries again later.

The task list is written in priority order. For the fleet agent, colonizing and defending come before rallying and attacking. But the starting point rotates a little each time the agent thinks, so a task near the top that keeps failing doesn't stop the ones below it from ever running.

The idea of an HTN is that a big task can be broken down into smaller ones until you reach actions that can be carried out directly. The planner supports that kind of compound task, and DevLog 06 described it that way. In practice, all the tasks the AI uses right now are simple ones, and the chaining happens through conditions instead: one task's action changes the game state so that another task's preconditions become true. Each task stays small, and it's easy to test on its own.

An Example Chain: Going to War

Here's what a war looks like from the AI's side, task by task:

1. Threat builds up. Hostile actions against the AI raise its threat value toward you, and so does having your colonies close to its own.
2. DeclareWar starts once threat is over the doctrine's limit, the AI has enough ships and missiles, and it isn't already at war with you.
3. PickWarTarget chooses which enemy to focus on, since it might be at war with more than one player.
4. BuildVehicles runs in the colonies with a shipyard. With a war target set, the AI designs counter-ships against what it has seen of your fleet (more on that below). If a larger enemy fleet shows up near one of its colonies, BuildResponseFleet jumps in to build defenders there.
5. FleetFormGroup puts idle ships into groups, if the AI has a working Command Centre.
6. FleetRally runs while the fleet is still spread out. It picks a safe asteroid near the front and sends the ships there.
7. FleetAttack can only start once enough of the fleet has gathered, so it takes over from FleetRally at that point. It sends the ships that can reach the target from the asteroid where most of them are waiting.
8. FleetService pulls damaged or low-fuel ships back home for repairs and fuel, and FleetDefend sends ships back if one of its own colonies comes under attack.
9. SueForPeace starts if the war goes badly and the AI has lost too many of its colonies.

None of these tasks knows about the others. FleetAttack doesn't call FleetRally. They only share the game state, and the conditions decide which one gets to act.

How Often It Thinks

How often the AI thinks depends on difficulty. On Casual it thinks every 5 game days, on Normal every 2, and on Hard and Extreme every day. Hard and Extreme differ in other ways: on Extreme the AI holds a grudge longer (threat toward a rival cools off half as fast), and it's allowed a few more mines and turrets per colony. None of the levels give the AI extra money or information.

The AI reads its information from the same game state the rest of the game uses: asteroids, ships, balances, diplomacy and shipyard queues. There's no separate copy of the world for it, so the rule about hidden information has to be enforced in each task. When a task looks at enemy ships or asteroids, it should only consider what the AI has actually seen or surveyed.

Doctrines

Until now, every AI corporation played the same way. That made them predictable, so each faction now has a doctrine. A doctrine is a set of values that the task conditions read. Changing a few numbers changes the personality quite a lot.

• Meridian Syndicate, "Merchant Prince". Slow to go to war and quick to ask for peace. It needs more threat and more ships before declaring war, and it sues for peace once it has lost a quarter of its colonies. It builds extra mines, puts research first and keeps a larger cash reserve.
• Helios Combine, "Long-Haul Strike". Builds cruisers with torpedoes, missiles, fuel modules and fast engines, so it can hit you from far away. It also runs two colony ships.
• Kessler Holdings, "Iron Line". Defensive. Extra turrets, more missile silos, armored battleships with point defense. It waits until 90% of its fleet has gathered before attacking. When it finally comes, it comes with nuclear missiles.
• Vantor Dynamics, "Hammer". The aggressive one. It declares war at half the normal threat with only two ships, attacks when 40% of its fleet has gathered, and keeps fighting until its ships are down to 25% health. It only asks for peace after losing three quarters of its colonies.
• Pallas Collective, "Swarm". Frigates with jammers and scanners, three colony ships, and missiles like Vortex, Scatter and Virus.

Every doctrine describes ships by role (gun, torpedo, armor and so on), not by specific items. The AI fills each role with the best item it has unlocked, so its ships get better as it researches, in the same way yours do.

Declaring war is a good example of how the numbers plug in. The strategy agent tracks a threat value toward each rival. War is declared when threat reaches 80 times the doctrine's war multiplier and the fleet is big enough. So Vantor, with a multiplier of 0.5, gets there at 40, while Meridian needs 120.

I also added rivalry pressure. Colonies within 120 units of a rival's colony slowly raise threat toward that rival. Before this, first wars tended to start somewhere around day 800 to 1000, and it varied a lot depending on the map. Neighbors should get on each other's nerves sooner than that.

Designing Counter-Ships

This is the part I'm happiest with. Every 10 days, the AI looks at the warships of the player it's at war with. It only sees ships near its own colonies or its own fleet. For those ships it records their average armor, evasion, health and damage reduction, which weapons they carry, and how big the groups are. It remembers each sighting until that ship is destroyed.

Then it designs a ship against that profile. It starts with an empty hull, reserves bomb slots for hitting colonies, and fills the rest one hardpoint at a time, each time picking the weapon or module that increases firepower × effective health the most against what it has seen. It does this with the actual combat rules: hit chance against evasion, armor as a percentage damage reduction, point defense against homing weapons, EMP jamming, splash damage against groups, and limited ammo.

So if you rely on missiles and torpedoes, point defense becomes worth more to it. If you fly in large groups, splash damage scores higher.

To choose between hulls, it uses Lanchester's square law. In a fight where everyone can shoot at everyone, a fleet's strength grows with the square of its size. That means two cheap ships are often better than one expensive one, so the AI scores each hull as strength divided by cost squared, adjusted by the doctrine's hull preference. Without intel on your fleet, it falls back to the doctrine's standard design.

Not Going Bankrupt

Ships have upkeep, and the AI could end up building more ships than it could afford to keep. Now every purchase goes through a budget check.

The AI keeps a reserve of 10,000 credits plus 180 days of the difference between projected upkeep and income. Projected upkeep includes ships already queued in its shipyards and the new ship it wants to buy. If it doesn't have a colony ship, it also keeps money aside for one. A purchase is only approved if the AI still has that reserve afterwards.

When money gets tight, the AI switches its focus to the economy, even during a war. It stops building warships, skips optional Sci-Tek, missile and satellite purchases, and sells surplus ore. It still builds colony ships and scouts.

Colonizing Better

The AI used to pick colony sites badly and sometimes kept sending ships back to a site that had already failed. Now it avoids asteroids with high radiation, prefers surveyed asteroids with ore, and then picks the closest. Colony ships get the best fuel module, and their reach is based on fuel instead of scanner range.

A new colony also follows a build order now: radiation protection first if it's needed, then sensors and refueling, ore storage, a Gravity Nullifier so the asteroid doesn't drift away, housing, and turrets if enemies are close.

There's also a new FleetAdvance task. Idle warships move forward to the colony with refueling that's closest to the enemy, so the AI's fleet slowly creeps toward the front during a war instead of sitting at home.

Letting the AI Use Everything

It turned out the AI was missing a lot of the game. Life-support buildings were looked up wrongly, so it never built them. It never built a Ship Yard or a Weapons Factory. Checking whether it could buy a missile actually charged it money every time it checked.

After fixing that, each AI has its own Sci-Tek unlocks and uses every missile and satellite type it has unlocked. It builds landing pads, repair, screen generators, ore teleporters and Gravity Nullifiers. Damaged or low-fuel ships go home for service. And if it has Asteroid Engines, it now points them at the nearest colony of the player it's at war with.

Keeping the AI from Slowing the Game

The AI runs inside the game's per-day update. That update is spread over several frames: once it has used more than 4 ms in a frame, it stops and continues on the next frame. So even when several AI players think on the same day, the game shouldn't stutter. I also log a warning if the planner takes more than 2 ms, so I notice when something gets expensive.

Testing with Autoplay

Changing AI behavior is risky. A fix in one place often breaks something somewhere else, and I can't play a full game after every change. So I built an autoplay mode.

In autoplay, every player is controlled by the AI, including the slot that's normally yours. It can run from the Unity editor or from the command line with options for seed, speed, number of days and number of runs. It can also run headless, without cameras, UI or audio, and as fast as the computer allows. A game ends after 3,000 days or after 300 days of stalemate.

During a run it records a timeline every 10 days, a log of events and a summary. A small script reads the results and checks a list of things that should always be true:

• every game finished
• no AI went bankrupt
• every faction built a colony ship and expanded
• every faction still alive holds at least two colonies
• no colony was lost twice without a known attacker
• every missile launch had a recorded outcome
• colony point defense fired when it was attacked
• at least one fleet attack happened, the first one by day 900, and some bombardment damage was done

It uses three fixed seeds, so if something fails I can run the exact same game again and look at what went wrong. In autoplay the AI is also never allowed to actually go bankrupt. If its balance goes negative, it's reset to zero and counted as a failure, so I can find the cause instead of just watching a corporation disappear.

What's Next

The doctrines are still first drafts, and the numbers will change. Now that autoplay exists, I want to use it to compare the doctrines against each other and see whether any of them is clearly stronger. Next month I'll probably go back to the player side of things. You can follow along on the roadmap.

Signing out,

• Jari