Need Help Evolving a Decision-Making Machine

[dbabbitt]

Hi Guys!

The chromosomes in my genome have 1 gene for a decision based on the trinary-like possibility genes that compose the rest of the chromosome. (These chromosomes can easily be represented as records in a SQL database table.) I want to keep a whole "ecosystem" of these chromosomes going, as they represent little bits of wisdom for what to do next, given the state of the game. Then I could just pole them as a group, and take the majority's advice:

    -- Should I take the 5 card or pass? The game has just started.
    SELECT TOP 1 decision_string
    FROM decisions
    WHERE
        (player_up = '1' OR player_up = 'Don''t Care') AND
        (deck_chip_count = '0' OR deck_chip_count = 'Don''t Care') AND
        (discard_pile_1_chip_count = '11' OR discard_pile_1_chip_count = 'Don''t Care') AND
        (discard_pile_2_chip_count = '11' OR discard_pile_2_chip_count = 'Don''t Care') AND
        (discard_pile_3_chip_count = '11' OR discard_pile_3_chip_count = 'Don''t Care') AND
        (discard_pile_4_chip_count = '11' OR discard_pile_4_chip_count = 'Don''t Care') AND
        (card_05_location = 'Top Card' OR card_05_location = 'Don''t Care')
    GROUP BY decision_string
    ORDER BY COUNT(decision_string) DESC

In JGAP I am using integers instead of the text strings shown above to take advantage of the IntegerGene class:
   
    Gene[] sampleGenes = new Gene[40];
   
    /* command_possibilities for decision_string */
    sampleGenes[0] = new IntegerGene(conf, 1, 2);
   
    /* play_possibilities for player_up */
    sampleGenes[1] = new IntegerGene(conf, 1, 5);
   
    /* chip_possibilities for various chip counts */
    for(int i=2; i<7; i++) {
        sampleGenes[i] = new IntegerGene(conf, 1, 46);
    }
   
    /* card_possibilities for various card locations */
    for(int i=7; i<40; i++) {
        sampleGenes[i] = new IntegerGene(conf, 1, 7);
    }

For instance, sampleGenes[2] to sampleGenes[7] represent having 1 to all 44 chips, having zero chips, and not caring about how many you have, for a total of 46 possibilities.

I guess it goes without saying that I should keep track of every chromosome involved in each decision and wait until the end of the game to evaluate the fitness of each. What is the best way to do that? You can't determine whether a player_up = "Don't Care" chromosome was used, for instance, just by looking at it. And the game is of such that trying to get the best score in any particular decision almost guarantees that you will not get the best score at the end.

I want to design my generational flow for continuous adaptation to the abilities of the player. I want each player to have their own AI that optimizes to get the scores of its 3 players as close to the final score of the human player as possible, improves with the players improvement, etc. How do I package this so that the ecosystem is maintained and becomes a permanent part of the user's experience?

Any Ideas would be greatly appreciated.

Thanx

Dave Babbitt

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
jgap-users mailing list
jgap-users@...
https://lists.sourceforge.net/lists/listinfo/jgap-users

[dbabbitt]

I take it I should extend the abstract BulkFitnessFunction class and:

1) play one complete game by poling the whole population for each decision,
2) track each decision reached by each chromosome,
3) give each chromosome a fitness value based on what part it played in winning or losing the game.

Is there a better way to do this than copying each chromosome into a database and running SQL against it? How do I set up the equivalent of a vote tracker table? It looks something like this:

tracker_id    player_id    is_carried   decision_id_list
1             1            0            1,23,53,69,79,81,131,147,151,155,159
2             1            1            2,24,54,70,80,82,94,132,148,152,156,160
3             2            0            23,53,69,79,131,147,155,159
4             2            1            24,54,70,80,94,132,148,156,160
5             3            0            23,53,69,79,131,147,155,159
6             3            1            24,54,70,80,94,132,148,156,160
7             4            0            23,53,69,79,131,147,155,159
8             4            1            24,54,70,80,94,132,148,156,160
9             1            0            1,23,53,69,79,81,131,147,155,159
10           1            1            2,24,54,70,80,82,94,132,148,156,160
11           2            0            23,53,69,79,131,147,155,159
12           2            1            24,54,70,80,94,132,148,156,160
13           3            0            23,53,69,79,131,147,155,159
14           3            1            24,54,70,80,94,132,148,156,160
15           4            0            23,53,69,79,131,147,155,159
16           4            1            24,54,70,80,94,132,148,156,160

As each player makes a decision, the decision ids of the yay and nay votes are kept track of so we know who to purge later. At the end of the game, I give the chromosomes (that voted with the majority to help the player in 4th place lose) the losing score of the 4th player, then the chromosomes that helped the player in 3rd place the 3rd place score (overwriting scores as necessary), then 2nd place, and finally 1st place. The majority of score changes turn out to be 1st place, with a few outliers left over for the others.

[Klaus Meffert-5]

David,

 

although it may be possible using GA for rules that are static regarding their structure, the better and more flexible choice is using GP (Genetic Programming).

GP is also possible via JGAP, please see the examples (e.g. package examples.gp).

 

Best

Klaus
www.klaus-meffert.com

 

---

From: Dave Babbitt [mailto:dave@...]
Sent: Thursday, April 23, 2009 3:02 PM
To: jgap-users@...
Subject: [jgap-users] Need Help Evolving a Decision-Making Machine

Hi Guys!

The chromosomes in my genome have 1 gene for a decision based on the trinary-like possibility genes that compose the rest of the chromosome. (These chromosomes can easily be represented as records in a SQL database table.) I want to keep a whole "ecosystem" of these chromosomes going, as they represent little bits of wisdom for what to do next, given the state of the game. Then I could just pole them as a group, and take the majority's advice:

    -- Should I take the 5 card or pass? The game has just started.
    SELECT TOP 1 decision_string
    FROM decisions
    WHERE
        (player_up = '1' OR player_up = 'Don''t Care') AND
        (deck_chip_count = '0' OR deck_chip_count = 'Don''t Care') AND
        (discard_pile_1_chip_count = '11' OR discard_pile_1_chip_count = 'Don''t Care') AND
        (discard_pile_2_chip_count = '11' OR discard_pile_2_chip_count = 'Don''t Care') AND
        (discard_pile_3_chip_count = '11' OR discard_pile_3_chip_count = 'Don''t Care') AND
        (discard_pile_4_chip_count = '11' OR discard_pile_4_chip_count = 'Don''t Care') AND
        (card_05_location = 'Top Card' OR card_05_location = 'Don''t Care')
    GROUP BY decision_string
    ORDER BY COUNT(decision_string) DESC

In JGAP I am using integers instead of the text strings shown above to take advantage of the IntegerGene class:
   
    Gene[] sampleGenes = new Gene[40];
   
    /* command_possibilities for decision_string */
    sampleGenes[0] = new IntegerGene(conf, 1, 2);
   
    /* play_possibilities for player_up */
    sampleGenes[1] = new IntegerGene(conf, 1, 5);
   
    /* chip_possibilities for various chip counts */
    for(int i=2; i<7; i++) {
        sampleGenes[i] = new IntegerGene(conf, 1, 46);
    }
   
    /* card_possibilities for various card locations */
    for(int i=7; i<40; i++) {
        sampleGenes[i] = new IntegerGene(conf, 1, 7);
    }

For instance, sampleGenes[2] to sampleGenes[7] represent having 1 to all 44 chips, having zero chips, and not caring about how many you have, for a total of 46 possibilities.

I guess it goes without saying that I should keep track of every chromosome involved in each decision and wait until the end of the game to evaluate the fitness of each. What is the best way to do that? You can't determine whether a player_up = "Don't Care" chromosome was used, for instance, just by looking at it. And the game is of such that trying to get the best score in any particular decision almost guarantees that you will not get the best score at the end.

I want to design my generational flow for continuous adaptation to the abilities of the player. I want each player to have their own AI that optimizes to get the scores of its 3 players as close to the final score of the human player as possible, improves with the players improvement, etc. How do I package this so that the ecosystem is maintained and becomes a permanent part of the user's experience?

Any Ideas would be greatly appreciated.

Thanx

Dave Babbitt

Eingehende eMail ist virenfrei.
Von AVG überprüft - www.avg.de
Version: 8.5.287 / Virendatenbank: 270.12.4/2077 - Ausgabedatum: 04/23/09 19:21:00

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
jgap-users mailing list
jgap-users@...
https://lists.sourceforge.net/lists/listinfo/jgap-users

[Klaus Meffert-5]

Dave,

 

regarding your SQL statements, I don't know the exact architecture of your proposed solution. It seems to have two levels: one GA and one database level. I cannot make any qualified comment about that as I do not know the details.

 

Regarding GP: It allows to construct rules, or SQL statements, natively by using commands like SELECT or operators like AND and OR . In opposite to that, with a GA you have to codify everything within the fitness function, i.e. you have to interpret an otherwise senseless stream of data into something meanignful.

 

Best

Klaus
www.klaus-meffert.com

 

---

From: dave.babbitt@... [mailto:dave.babbitt@...] On Behalf Of Dave Babbitt
Sent: Sunday, May 03, 2009 8:20 PM
To: Klaus Meffert
Subject: Re: [jgap-users] Need Help Evolving a Decision-Making Machine

It is true that GP would be more flexible in this sense: I wouldn't have to deal with "I Don't Care" choices because a smaller structure would just be a decision to ignore them. But in what way is it better? Is there a better way to decide what move to make than to get the majority vote of the whole population? I imagine I would do that in either case.

 

Right now, with a population of 1000, I can get only two games played before I get a heap error, so I am looking for a better way to do this than converting chromosomes to SQL records and back again.

On Sun, May 3, 2009 at 10:54 AM, Klaus Meffert <jgap@...> wrote:

David,

 

although it may be possible using GA for rules that are static regarding their structure, the better and more flexible choice is using GP (Genetic Programming).

GP is also possible via JGAP, please see the examples (e.g. package examples.gp).

 

Best

Klaus
www.klaus-meffert.com

 

---

From: Dave Babbitt [mailto:dave@...]
Sent: Thursday, April 23, 2009 3:02 PM
To: jgap-users@...
Subject: [jgap-users] Need Help Evolving a Decision-Making Machine

Hi Guys!

The chromosomes in my genome have 1 gene for a decision based on the trinary-like possibility genes that compose the rest of the chromosome. (These chromosomes can easily be represented as records in a SQL database table.) I want to keep a whole "ecosystem" of these chromosomes going, as they represent little bits of wisdom for what to do next, given the state of the game. Then I could just pole them as a group, and take the majority's advice:

    -- Should I take the 5 card or pass? The game has just started.
    SELECT TOP 1 decision_string
    FROM decisions
    WHERE
        (player_up = '1' OR player_up = 'Don''t Care') AND
        (deck_chip_count = '0' OR deck_chip_count = 'Don''t Care') AND
        (discard_pile_1_chip_count = '11' OR discard_pile_1_chip_count = 'Don''t Care') AND
        (discard_pile_2_chip_count = '11' OR discard_pile_2_chip_count = 'Don''t Care') AND
        (discard_pile_3_chip_count = '11' OR discard_pile_3_chip_count = 'Don''t Care') AND
        (discard_pile_4_chip_count = '11' OR discard_pile_4_chip_count = 'Don''t Care') AND
        (card_05_location = 'Top Card' OR card_05_location = 'Don''t Care')
    GROUP BY decision_string
    ORDER BY COUNT(decision_string) DESC

In JGAP I am using integers instead of the text strings shown above to take advantage of the IntegerGene class:
   
    Gene[] sampleGenes = new Gene[40];
   
    /* command_possibilities for decision_string */
    sampleGenes[0] = new IntegerGene(conf, 1, 2);
   
    /* play_possibilities for player_up */
    sampleGenes[1] = new IntegerGene(conf, 1, 5);
   
    /* chip_possibilities for various chip counts */
    for(int i=2; i<7; i++) {
        sampleGenes[i] = new IntegerGene(conf, 1, 46);
    }
   
    /* card_possibilities for various card locations */
    for(int i=7; i<40; i++) {
        sampleGenes[i] = new IntegerGene(conf, 1, 7);
    }

For instance, sampleGenes[2] to sampleGenes[7] represent having 1 to all 44 chips, having zero chips, and not caring about how many you have, for a total of 46 possibilities.

I guess it goes without saying that I should keep track of every chromosome involved in each decision and wait until the end of the game to evaluate the fitness of each. What is the best way to do that? You can't determine whether a player_up = "Don't Care" chromosome was used, for instance, just by looking at it. And the game is of such that trying to get the best score in any particular decision almost guarantees that you will not get the best score at the end.

I want to design my generational flow for continuous adaptation to the abilities of the player. I want each player to have their own AI that optimizes to get the scores of its 3 players as close to the final score of the human player as possible, improves with the players improvement, etc. How do I package this so that the ecosystem is maintained and becomes a permanent part of the user's experience?

Any Ideas would be greatly appreciated.

Thanx

Dave Babbitt

Eingehende eMail ist virenfrei.
Von AVG überprüft - www.avg.de
Version: 8.5.287 / Virendatenbank: 270.12.4/2077 - Ausgabedatum: 04/23/09 19:21:00

--
Dave Babbitt
617-413-8326
98 Ashley ST
W Springfield, MA 01089

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
jgap-users mailing list
jgap-users@...
https://lists.sourceforge.net/lists/listinfo/jgap-users