Civ4 AI Survivor Season 4: Conclusions

It was another season of miserable failure for Germany, which failed to get either of its two leaders into the playoffs for the fourth year in a row. Bismarck and Frederick both continue to have zero points and the ongoing incompetence of Germany makes less and less sense every year. France similarly saw eliminations of all three leaders in the opening round while England was unable do better than a weak showing from Churchill in the Wildcard game. After a strong performance from Ramesses in Season Three, Egypt was once again back to more mediocrity with double eliminations in the opening round. The most surprising name at the top of the list has to be Mansa Musa, who had won his opening round game in every previous season before suffering a First to Die fate in his Season Four opener. I suspect that he'll bounce back next year unless he draws a truly unfriendly set of neighbors. The least surprising name at the top of the list has to be Sitting Bull, who's been incompetent enough that we turned him into the AI Observer civ for this season's Apostolic Palace purposes. I think it's betweeen him or Frederick as the AI leader least likely to win a game. Even Montezuma could theoretically snowball off an early conquest.

At the other side of the finishing order, we saw many of our familiar names have success again in at least their opening round matches. Huayna Capac, Justinian, Julius Caesar, and Stalin all won their opening round matches in dominant fashion. They were joined in the playoffs by a series of unlikely first-time upset winners: Isabella and Tokugawa and Darius, none of whom had done much of anything in earlier seasons. Isabella in particular came into Season Four with zero points and managed to pick up a first place finish and three kills from a strong opening round game. The playoff round saw the upset trend go into overdrive, as Willem and Gilgamesh and Charlemagne were our three winners, none of whom had won a game previously. Somehow all of them made the Championship while Justinian and Huayna Capac and Caesar and Mao Zedong failed to advance. This was capped off by Charlemagne winning the title in completely ridiculous fashion after both Willem and Kublai Khan tossed away certain victories with idiotic decisions. As a result, Season Four will likely be remembered as a season of weird results despite many of the favorites holding serve in the opening round. Sometimes weird stuff happens in sports; Greece did win the Euro 2004 tournament after all.

The kills were heavily concentrated amongst the leaders who made the playoffs in the same pattern that we've seen in past years. The 30 leaders that fell in the opening round only managed 4 kills between them while the 6 leaders that made the Championship collected 18 trophies. This was actually less lopsided than what we saw in Season Three, where the Championship participants had a staggering 26 collective kills between them. The violence was more evenly distributed this year with a lot of different leaders having 2-3 kills apiece. This was likely due to the fact that we had so many different leaders come out on top in individual matches, with 12 different leaders winning the 13 total games. Season Three was much more lopsided with only 9 different leaders winning games, the minimum possible for the tournament. Gilgamesh ended up winning the Golden Spear award with 5 kills thanks to his last-second snipe of a kill in the Championship. If Kublai had landed that finishing blow, we would have had a three-way tie between Gilgamesh and Kublai and Charlemagne at four apiece.

The AI ranking list has also been updated to reflect an additional year of competition. As a quick reminder of how the ranking system works, leaders score 5 points for a first place finish, 2 points for a second place finish, and 1 point for each kill. This is a completely arbitrary system, of course, but it seems to do a decent job of recognizing AI performance across different games and different seasons. I included an additional column listing the points scored solely in Season Four to help indicate which leaders were the biggest movers in this season. One of the things that stood out to me was a strong performance from almost everyone at the top of the table. With the exceptions of Mansa Musa and Cyrus, 9 of the top 11 leaders scored at least 4 points this year. Out of that group, every leader won at least one game with the exception of Mao who had a second place finish. Now this is circular reasoning to a certain extent because the very act of winning a game grants points that put a leader closer to the top of the scoreboard, and yet we've done enough seasons by now that it can't be a total coincidence. It does feel to me that we're getting closer to identifying a core set of leaders that consistently perform well under the game settings that we use for AI Survivor.

The single biggest mover in the AI rankings was Charlemagne, predictably so since he was the overall winner of the competition. His 16 points elevated him from complete nonentity into the top eight where he'll get a Pool 1 seed for the next season. (I think there are a lot of other leaders licking their chops at the prospect of getting to face Charlemagne rather than Justinian or Mansa Musa in the Pool 1 category.) The other big surprises in Season Four are clustered together in the middle of the table, the area that I like to think of as the "had one good game" region. This is where we find Darius, Gilgamesh, and Tokugawa, as well as Willem and Isabella a little bit further down the list. All of these leaders won a single game this year and picked up the vast majority of their points from that single game, much like De Gaulle's one good game from Season Three. This puts them above the leaders who have never done much of anything further below, while also having a clear separation from the true heavyweights at the top of the list who have repeatedly won their matches. Season Four essentially expanded this version of the AI Survivor middle class.

Then there's the bottom of the list where points from Season Four were rare indeed. Among the 26 leaders in the bottom half of the list, there were a grand total of 7 points scored in this season; by way of contrast, the 26 leaders in the top half of the list scored a whopping 128 points. Roosevelt was the only one in this group to achieve anything of note, and even that was due to a very distance second place finish followed by a facepalming waste of a great starting position in his playoff game. None of the 23 leaders listed lower than Joao have ever won a game at all (and Joao's sole victory came against the weak opposition of a Wildcard game). Their scant points come from various runner up finishes and a handful of kills. Six leaders still have no points whatsoever and are currently being outscored by the Apostolic Palace thanks to its elimination of Tokugawa in this year's playoff round. We've done enough seasons at this point to have a median AI score of 7 points, which means that anyone still sitting around at 1 or 2 points has been a sorry specimen indeed. That's not to say that their luck can't change, as we saw Gilgamesh and Willem do much better this year, but it's definitely an uphill struggle for these individuals.

Another year's worth of data has allowed us to update the Trait rankings for Season Four. This list mostly looks similar to what we had following Season Three, with Imperialistic and Financial once again grading out at the top two traits. There is a difference here, however, as those two traits stood alone far out in front of the rest of the pack last year, whereas now the Creative trait has almost caught up thanks to good seasons from Kublai Khan and Gilgamesh and Willem. I do think that Creative is a helpful trait for these AI Survivor games and it's not surprising to me that it would grade out as among the best options. Aggressive and Spiritual are essentially situated in the same place as they were previously, average traits that are not especially good or especially bad for AI Survivor purposes. Expansive and Organized are both a clear tier lower for these games, just as they were following Season Three. The AI leaders don't seem to struggle with health or do enough to prioritize workers/granaries to get a lot of value out of Expansive, and their maintenance costs are heavily discounted on Deity, enough to make Organized less useful for them.

The biggest surprise was the Protective trait moving from being clearly the worst trait in the game to merely below-average status. This was due to much better than normal seasons from Charlemagne, Gilgamesh, and Tokugawa along with another solid performance from Mao. I suspect that this was a coincidence and the Protective trait will drop over time back down towards the bottom of the rankings; needless to say, I don't think Charlemagne and Tokugawa are particularly good AI leaders and unlikely to be repeat performers. The Industrious trait continues to be overvalued due to the presence of Huayna Capac and Stalin, with all of the other leaders having poor seasons but the trait grading out as average instead of terrible due to the two of them. With that said, I predicted that Stalin would have a poor Season Four and he made it back to the Championship game a second time, so perhaps there's something going on there. Charismatic dropped into next-to-last status this season, and that was a bit of a surprise because it seems to be helpful for the AIs in terms of managing happiness. Unfortunately none of the leaders with the Charismatic trait did especially well this season, only Boudica and Hannibal making the playoffs and neither one of them doing so in impressive fashion. That trait needs Cyrus to go on another run to boost its ranking.

This year the Philosophical trait graded out as the worst in the competition after being the second-worst trait last season. It's been a precipitous fall from the Season One version of these rankings which had Philosophical as the best trait overall. That was under the old Finish Ranking system that heavily valued silly stuff like not getting eliminated in the Wildcard game so it's not a direct comparison, but it's still noteworthy how the trait has collapsed in value over the following seasons. Philosophical is a poor trait for the AI leaders because they inevitably run so many specialists that they always get lots of Great People regardless. Plus the AI has no idea how to leverage individual Great People for timed Golden Ages, and of course we turn off Tech Trading to remove the value of lightbulbing followed by tech brokering. My feeling is that Philosophical and Protective are likely the worst traits for these games, while Imperialistic and Financial are probably the best. Somehow Victoria continues to have zero points despite being the leader that combines Financial and Imperialistic together - it's quite a feat!

We also continue with our tradition of looking at the average score of the leaders making up every past match. This allows us to see which games had particularly strong or weak fields, as well as revise our opinions about some matchups from past seasons that might have looked different at the time. Keep in mind that the average leader score after four seasons is 10 points, and therefore games above that have stronger fields and those below that have weaker fields. (For the statistically inclined, the standard deviation of the opening round matches is roughly 2.50.) The weakest field in AI Survivor history is now Season Three Game Two, a total snooze-fest won by Gandhi which came the closest we've ever seen to having no one eliminated. The score of 5.29 is almost two standard deviations below the median, something that we would expect to appear only about 5% of the time in a purely random setup. That game felt like a poor group of competitors at the time and the ranking system backs up that impression. Season Two Game Five was previously the worst-scoring game in AI Survivor history, however thanks to good seasons from Willem and Isabella and Shaka, it now grades out as merely below average instead of historically terrible. Other weak games include Boudica's win in Season One Game Three (6.50) and another Mansa Musa stomp in Season Two Game Six (6.71).

As far as the best collection of opening game talent, we have two games that grade out almost the same. The highest score comes from Season Two Game Two at 14.57, largely due to the presence of Huayna Capac and Stalin. The same Russian leader also appears in Season One Game Seven (14.50) along with Catherine and Charlemagne and Gandhi. That game probably wasn't as impressive as it looks due to Charlemagne's inflated score making him appear to be a fearsome competitor. For more well-balanced fields of top competitors, we also had the "Group of Death" matches from Season One Game Two (Mansa, Justinian, Zara, Willem) and Season Three Game Six (Caesar, Cyrus, Zara, Pericles). Those were likely tougher draws since they didn't rely on a couple of outlier leaders with super high scores to pull up the overall average.

The opening round games from Season Four didn't score especially high or especially low in comparison to past seasons. This was by design since we introduced the seeding system for Season Four, and it leveled out the playing field to prevent a bunch of top talent showing up all at once. The strongest group turned out to be Game Four, although again this is only a tough game if you believe that Charlemagne is one of the game's best leaders. The weakest group was Gandhi's competitors in Game Two, which actually graded out lower than the Wildcard game. (All three of Gandhi's wins have taken place in games with fields that were well below average, which might be a reason to be skeptical of him in future seasons.)

With regards to the playoffs, the perception was that Playoff One had the strongest field by far, and the ranking system again confirms that impression. Season Four Playoff One was the highest scoring playoff game in all four seasons of AI Survivor, and in fact had a stronger collection of talent than the Season Four Championship! It also outscored the Season One Championship and was just barely lower than the Season Two Championship. The other two playoff games from Season Four were pretty typical, with the average playoff score across all four seasons coming in at about 16.50 points. Season One has the lowest scoring playoff game of all time, a dismally boring affair won by Mansa Musa with a score of just 11.67. I wrote at the time that I thought some of the leaders in the Season One Championship weren't particularly great competitors, and additional seasons have largely confirmed that impression. By way of contrast, the Season Three Championship was ridiculously stacked with talent, an average score of 29.33 points! That game had five of our top six leaders in the total ranking, missing only Huayna Capac to have the strongest field possible by our current metric. I still think that some of those leaders would have mopped the floor with the Season Four Championship entrants.

OK, let's talk about Charlemagne for a minute here. I've been savaging him pretty good throughout this Conclusions section so let me explain in some detail why I regard him as a total fluke winner. Charlemagne's traits are roughly average for AI Survivor purposes, with Imperialistic being good and Protective being bad. His Mysticism/Hunting starting techs are well below average and put him in a poor position out of the gate; in the Championship game, Charlemagne took ages to research The Wheel tech and sat around with no resources connected long into the game. He is not an individual likely to get out to a dynamic start, largely because he lacks the economy to keep pushing expansion with his Imperialistic trait. Charlemagne's personality is a standard combination of religion + militarism, and his best path to a strong performance comes from founding a religion and then making a bunch of diplomatic allies through spreading it around. Unfortunately this religious focus will also cause him to research multiple secondary religions that don't help him, and it can also make serious enemies if Charlemagne finds himself on the wrong side of the religious divide. He's basically an average leader for AI Survivor purposes, a slightly better version of Saladin. Charlemagne had achieved almost nothing of consequence coming into this season, and was best known for throwing away a game in epic fashion to Wang Kon in Season Three.

So what exactly did Charlemagne do this season to emerge with the overall crown, and how did he accumulate those points that put him into Pool 1 status for Season Five? Back in Season Three, Charlemagne picked up 2 points from finishing in second place to Wang Kon by tying for a Spaceship victory on Turn 371. This was not an impressive performance and any halfway decent economic leader would have launched the spaceship decades earlier. In Season Four, Charlemagne made it to the playoffs by riding Justinian's coattails as the runner up of his opening round game. It was a distant second-place performance that largely consisted of being part of the winning religious bloc and watching while other leaders who had played better games outside of that bloc were picked off by Justinian. Charlemagne played his best game in the playoff round, leveraging a highly unlikely early conquest of Suryavarman (who ended up lacking all metal resources) into a powerhouse role. However, even in that game Charlemagne had no business winning and would have easily been eclipsed by Darius if the Persian leader hadn't lost his mind and started chasing after a Cultural victory. Charlemagne "won" the game with a late Turn 348 Spaceship victory while still sitting in second place on the scoreboard.

Then we had the Championship, where Charlemagne struggled out of the gate, spent most of the game sitting in third or fourth place in score, only to see everyone in front of him repeatedly self-sabotage themselves en route to backing his way into the title. Have you ever seen the clip of the speed skaters who all wipe out, leaving the distant trailing athelete as the winner of the race? Yeah, this was kind of like that. Willem would have won the title easily if he had researched Rifling tech, and then Kublai would have won the game easily if he hadn't stopped all research to pursue a completely crazy cultural plan. Charlemagne's Spaceship victory didn't arrive until the exceedingly late date of Turn 388, long after someone would have finished in most other games. In fact, Charlemagne's two victories in Season Four had two of the three latest finishing dates throughout the season, and both of them only occurred after another stronger leader had tossed away a certain victory. The most likely ending to this season would have seen Charlemagne finish the year with two second place finishes and a third place finish in the Championship game, scoring 6 or 7 points with his kills added but nothing more. Yes, Charlemagne was the winner of this season, but the Burger King has to be regarded as a bit of a fluke performer. Our other three champions were all highly deserving of their crowns - not so much this guy.

Anyway, I'd like to give thanks here to everyone here who took part in Season Four of AI Survivor. Even if it was nothing more than spending two minutes to submit a prediction, the competition wouldn't have been the same without all of your contributions. We had our best turnout yet this season, with about 160 predictions on average for each game and more than 100 viewers (!) watching along on the Livestream. We peaked at 181 contest entries and 148 live viewers, the most that I've ever had watching outside of my IPL casting days for League of Legends. This event simply would not be what it is without the lively banter in the chat taking place alongside the action. I am surprised again and again at the brilliant insights that the viewers come up with in their predictions, along with the hilarious and inane running commentary that makes it such a pleasure to put these games together. I think that the format we used this year was a huge success, and we'll think about further small tweaks (especially what to do about the Apostolic Palace) in the months to come.

Finally, this season is dedicated to Rich and Karla. I hope that watching these games helped to spark joy at a time of great loss. Thank you for being a part of this experience.