|
Just posting an article linked to on the Den. Wanted to see what people here thought of it. Full article here.
In 1981, a computer scientist from Stanford University named Doug Lenat entered the Traveller Trillion Credit Squadron tournament, in San Mateo, California. It was a war game. The contestants had been given several volumes of rules, well beforehand, and had been asked to design their own fleet of warships with a mythical budget of a trillion dollars. The fleets then squared off against one another in the course of a weekend. “Imagine this enormous auditorium area with tables, and at each table people are paired off,” Lenat said. “The winners go on and advance. The losers get eliminated, and the field gets smaller and smaller, and the audience gets larger and larger.”
Lenat had developed an artificial-intelligence program that he called Eurisko, and he decided to feed his program the rules of the tournament. Lenat did not give Eurisko any advice or steer the program in any particular strategic direction. He was not a war-gamer. He simply let Eurisko figure things out for itself. For about a month, for ten hours every night on a hundred computers at Xerox PARC, in Palo Alto, Eurisko ground away at the problem, until it came out with an answer. Most teams fielded some version of a traditional naval fleet—an array of ships of various sizes, each well defended against enemy attack. Eurisko thought differently. “The program came up with a strategy of spending the trillion on an astronomical number of small ships like P.T. boats, with powerful weapons but absolutely no defense and no mobility,” Lenat said. “They just sat there. Basically, if they were hit once they would sink. And what happened is that the enemy would take its shots, and every one of those shots would sink our ships. But it didn’t matter, because we had so many.” Lenat won the tournament in a runaway.
The next year, Lenat entered once more, only this time the rules had changed. Fleets could no longer just sit there. Now one of the criteria of success in battle was fleet “agility.” Eurisko went back to work. “What Eurisko did was say that if any of our ships got damaged it would sink itself—and that would raise fleet agility back up again,” Lenat said. Eurisko won again.
Eurisko was an underdog. The other gamers were people steeped in military strategy and history. They were the sort who could tell you how Wellington had outfoxed Napoleon at Waterloo, or what exactly happened at Antietam. They had been raised on Dungeons and Dragons. They were insiders. Eurisko, on the other hand, knew nothing but the rule book. It had no common sense. As Lenat points out, a human being understands the meaning of the sentences “Johnny robbed a bank. He is now serving twenty years in prison,” but Eurisko could not, because as a computer it was perfectly literal; it could not fill in the missing step—“Johnny was caught, tried, and convicted.” Eurisko was an outsider. But it was precisely that outsiderness that led to Eurisko’s victory: not knowing the conventions of the game turned out to be an advantage.
“Eurisko was exposing the fact that any finite set of rules is going to be a very incomplete approximation of reality,” Lenat explained. “What the other entrants were doing was filling in the holes in the rules with real-world, realistic answers. But Eurisko didn’t have that kind of preconception, partly because it didn’t know enough about the world.” So it found solutions that were, as Lenat freely admits, “socially horrifying”: send a thousand defenseless and immobile ships into battle; sink your own ships the moment they get damaged.
...
“In the beginning, everyone laughed at our fleet,” Lenat said. “It was really embarrassing. People felt sorry for us. But somewhere around the third round they stopped laughing, and some time around the fourth round they started complaining to the judges. When we won again, some people got very angry, and the tournament directors basically said that it was not really in the spirit of the tournament to have these weird computer-designed fleets winning. They said that if we entered again they would stop having the tournament. I decided the best thing to do was to graciously bow out.”
It isn’t surprising that the tournament directors found Eurisko’s strategies beyond the pale. It’s wrong to sink your own ships, they believed. And they were right. But let’s remember who made that rule: Goliath. And let’s remember why Goliath made that rule: when the world has to play on Goliath’s terms, Goliath wins.
|
I'm waving the BS flag.
We don't have an AI that you can simply insert the manual and have it design a fleet for you. Doug had to do a lot of work to turn the rules into something that the AI could work with. That alone would impart some sort of bias.
***
The fact that he was able to make a stationary torpedo mine field win is just showing the game designers the weakness of their game. If the objective was changed to "get past the opponents fleet" the enemy would just go around, or blow a path through his stationary fleet. Doug and his AI would lose.
***
As an old Battletech player, it was often the guy that loaded up on the little fast hover tank that won the game. Not pretty when the big walking bots are supposed to rule. But it would work and make die-hard players pull their hair out.
I'm waving the BS flag.
We don't have an AI that you can simply insert the manual and have it design a fleet for you. Doug had to do a lot of work to turn the rules into something that the AI could work with. That alone would impart some sort of bias.***
The fact that he was able to make a stationary torpedo mine field win is just showing the game designers the weakness of their game. If the objective was changed to "get past the opponents fleet" the enemy would just go around, or blow a path through his stationary fleet. Doug and his AI would lose.
***
As an old Battletech player, it was often the guy that loaded up on the little fast hover tank that won the game. Not pretty when the big walking bots are supposed to rule. But it would work and make die-hard players pull their hair out.
Two links:
http://www.aliciapatterson.org/APF0704/Johnson/Johnson.html (more detailed version of the story, giving insight into the programming methods)
http://portal.acm.org/citation.cfm?id=1056665 Published paper describing tournament, program. I don't have an ACM account, but the abstract is:
"During the month of June 1981, the EURISKO program was set the task of exploring the design of naval fleets conforming to a body of (several hundreds of) rules and constraints as set forward in Traveller: The Trillion Credit Squadron. EURISKO designed a fleet of ships suitable for entry in the 1981 Origins national wargame tournament, held at Dunfey's Hotel, in San Mateo, Ca., over July 4 weekend. The Traveller tournament, run by Game Designers Workshop (based in Normal, Illinois), was single elimination, six rounds. EURISKO's fleet won that tournament, thereby becoming the ranking player in the United States (and also an honorary Admiral in the Traveller navy). This win is made more significant by the fact that the program's creator, Professor Douglas Lenat of Stanford University's Heuristic Programming Project, had never played this game before, nor any miniatures battle game of this type."
|
|
I'm waving the BS flag.
We don't have an AI that you can simply insert the manual and have it design a fleet for you. Doug had to do a lot of work to turn the rules into something that the AI could work with. That alone would impart some sort of bias.
You mean the same way we put the rules of chess into a computer and can have it play at different difficulty levels? I'm not sure what you're throwing the BS flag over.
The fact that he was able to make a stationary torpedo mine field win is just showing the game designers the weakness of their game. If the objective was changed to "get past the opponents fleet" the enemy would just go around, or blow a path through his stationary fleet. Doug and his AI would lose.
Or it would adjust to the new rules, such as by making a fleet of nothing but fast, tough ships that could take the hits and move beyond the enemy fleet? Or wall one super tough ship with an a shield of weaker ships so it could not be attacked, and leave a wall of other ships to prevent the enemy fleet from getting past before it does? Maybe hundreds of one-hit-k-o ships if only one can be targeted at a time?
As an old Battletech player, it was often the guy that loaded up on the little fast hover tank that won the game. Not pretty when the big walking bots are supposed to rule. But it would work and make die-hard players pull their hair out.
And that is ultimately the point of the article I think. Once you take away how you THINK the rules SHOULD work, you can analyze how they DO work and change them to fit your preconcieved conventions. This is, after all, why we playtest rules.
Two links:
Thanks for the help!
When Microsoft started selling the idea that all you needed to know how to do was slap pieces of code together, somebody should have taken them out behind the shed and shot them.
Right, because everyone should re-invent the wheel every time they start a new project.
I'm all for RAD. I'm not at all for encouraging ignorance of what your RAD components are doing.
Fair enough. I withdraw my protest and apologize.
|
This is an excellent example of the inherent limitations of rules.
There are several names for it, "the liars paradox", "Gödel's First Incompleteness Theorem", etc. It's a phenomenon that has been studied extensively for decades (mostly since computers were invented), and really quite interesting.
Basically, it boils down to the following: "it is impossible to have a complete set of rules for any sufficiently complicated environment or ruleset".
There will always be situations where the rules either conflict or are inapplicable, and there will always be ways to follow the letter of the rules whilst violating their spirit.
It is literally impossible for the rules of a large system to cover everything. Small programs and simple board games can be "complete", but anything larger, anything freeform, is going to be incomplete and require people in authority to make judgment calls.
It's not a failure of the Contest Rules, it's not a failure of the Traveler Rules in general, it's simply a fact of the universe.
And that's why GMs are so great.
That's not what I said. But if you don't understand the wheel, there are some pretty major problems going on.
Get out of your server rooms, people ;-P
The world isn't digital.
There are quite a few states between "I can reinvent the wheel with but a thought" and "weal? wat u mean?"
Computers have become quite pervasive in our civilisation. That means not just that many people use them, but also that many people teach them new tricks.
And just like the level of proficiency needed to use a computer has decreased in general, the level of proficiency needed to create programmes has decreased. Again, in general.
I personally consider it a good thing. If computers were still those cryptic things where you had to memorise a lot of commands and type them in to do simple stuff, they'd still be a toy for a small minority. They probably wouldn't be nearly as powerful or cheap as they are.
And the fact that you're able to create programmes without knowing all the intricacies of the more gruesome programming languages is good, too. Increases the pool of potential workers who can do the work.
And it's not as if the classical computer scientist were completely gone, either. Just like you still need sufficiently competent users for some computer programmes, you still need sufficiently competent developers for certain tasks.
So "everyone" (for a given value of everyone) can get the formulae for calculating interest into a computer and make it tell them how long they'll have to pay back that loan, just as you can put a merely moderately intelligent secretary in front of a computer and have her type up a letter and save it.
You still cannot put that secretary in front of, say, a CAX solution and make her design a power plant. And in the same vein you still cannot put everyone in front of the computer and tell them to write the physics engine for the next Quake game, or write a compiler, or create a search engine.
I'm personally happy for every trivial change I don't have to do myself.