学习连接四（译文）

原文地址：https://www.codeproject.com/Articles/7787/Learning-Connect-Four

原文作者：pseudonym67

译文由本站 robot-v1.0 翻译

前言

A Connect Four Game that learns from experience.

可以从经验中学到的四通游戏.

• 下载源文件-221 Kb(Download source files - 221 Kb)
• 下载演示项目-28.2 Kb(Download demo project - 28.2 Kb)

介绍(Introduction)

首先,应该从一开始就注意到,这是一个四连环游戏,这实际上是该项目附带的事实.创建该项目的背后有两个不同的原因,第一个原因是我想要比以前的应用程序中使用的更好的板子控制,并且我需要一个框架来在内部进行开发,而不会被其他项目所困扰开发董事会的过程.另外,我需要能够开发板控件,以便可以在不同的应用程序中使用它,因此我想尝试避免过分牢固地将其连接到任何特定项目的危险.我目前对此的个人感觉是,董事会控制仍需要另一个项目才能将其放置到我想要的位置,但是它在这里完成了它所需要的工作.(First of all, it should be noted right from the start that the fact that this is a connect four game is pretty much incidental to the project. There were two distinct reasons behind the creation of this project, the first being that I wanted a better board control than the one that I had been using in previous applications, and I needed a framework to develop that within, without other projects becoming sidetracked by the process of developing the board. Also, I needed to be able to develop the board control so that I could use it in different applications, and so I wanted to try and avoid the dangers of connecting it too strongly to any particular project. My current personal feeling on this is that the board control still needs another project to get it where I want it to be, but it does the job required of it here.) 该项目的第二个原因是与我一直在阅读的有关人工智能和学习理念的书籍有关.我在某处读到的论点大致是这样的:如果我们要开发智能的计算机软件,则该程序必须能够访问人类的全部知识,以便能够解决任何问题.在很多层面上,这使我感到不对劲,很难知道从哪里开始.首先,尽管地球上没有人能够获得人类的全部知识,或者能够记住其中的一小部分,即使他们确实以某种方式设法将其泵入大脑彻底的神经衰弱.然后便是启发式方法,并在信息不完整的情况下做出决策,更不用说考虑个人和政治偏见了.(The second reason for the project is to do with the reading I have been doing about Artificial Intelligence and the idea of learning. The argument I read somewhere goes something along the lines that if we ever want to develop computer software that is intelligent then the program must have access to the full knowledge of the human race in order to be able to work anything out. This strikes me as wrong on so many levels, it’s kind of hard to know where to start. Firstly, though there is no one on the planet that either has access to the full knowledge of the human race, or would be capable of remembering more than a fraction of it, even if they did somehow manage to pump it into their brains without having a total nervous breakdown. Then there is the point of heuristics and making decisions with incomplete information, let alone taking into account personal and political bias.) 但是,这分散了这个特定项目的注意力,让我们回到它.计算机完全可以访问人类知识总和的想法引起的问题是一回事,但是只需说一秒钟,您就编写了一个计算机程序,它是地球上最智能的软件.唯一的问题是它绝对不知道.首先,您如何将所有人类知识的总和输入计算机？一秒钟忘掉所有人类知识的总和,如何将任何信息输入程序？最明显的答案是使用某种数据库,而数据库的唯一问题是它们最初是空的.数据库还用于特定目的,其中包含与手头任务或业务相关的数据.甚至构想一个可以用于所有事物的通用数据库将是一项艰巨的任务.最后,尽管我们到达了即使拥有这样一个数据库的地步,我们仍然需要将信息放入其中,这使我进入了导致该项目的主要思路.(But this is distracting from this particular project, so let’s get back to it. The question that arises from the idea of a computer having complete access to sum of human knowledge is one thing, but just say for one second that you have written a computer program that is the most intelligent piece of software on the planet. The only problem is that it knows absolutely nothing. How do you get the sum of all human knowledge into the computer in the first place? Forget the sum of all human knowledge for a second, how do you get any information into the program? The most obvious answer to this is with a database of some sort, and the only problem with databases is that they are initially empty. Databases are also used for specific purposes, containing data that is pertinent to the task or business at hand. It would be quite a daunting task to even conceive of a general database that could be used for all things. Finally, though we get to the point where even if we had such a database, we would still need to put the information into it, which brings me to the main thread of thinking that led to this project.) 假设我们拥有虚构的智能软件和虚构的全知数据库模式来提供信息,那么他们将编写程序来填充数据库,或者坐在那里输入信息.我可以向你保证,我不会.我发现最好的时候数据库编程很乏味,我最后要做的最后一件事就是将信息输入到我的余生中,因为即使我个人还不了解所有人类知识的总和,可能会花费我的余生,只是将我确实拥有的少量知识以一种程序可以有意义地提取信息的形式输入数据库.这使我得出一个显而易见的结论,即如果一个软件要学习所有人类知识的总和,那么它就必须自己去做.(Given that we have the imaginary intelligent piece of software and the imaginary all knowing database schema to feed it information, who is going to either write the program/s to fill the database, or sit there typing in the information. I can assure you, I wouldn’t. I find database programming tedious at the best of times, the last thing I would be interested in doing would be entering information into a database for the rest of my life because even though I am personally far from knowing the sum of all human knowledge, it would probably take the rest of my life just to type in the small amount of knowledge I do have into a database in a form where a program could meaningfully extract the information. This brings me to the obvious conclusion that if a piece of software is going to learn the sum of all human knowledge, then it is just going to have to do it on its own.) 一个孩子可以自己学习,那么为什么不买一个软件呢？当然,与其给软件提供一个信息数据库,不如让它自己获取信息并希望它知道如何处理它,更有意义的是开发一种允许计算机自行学习的方法.这很方便地将我们带入模式.(A child can learn on its own, so why not a piece of software? Rather than hand the software a database of information to use, surely it makes more sense to develop a way of allowing the computer to learn for itself, than to just chuck information at it and hope that it knows how to deal with it. Which rather conveniently brings us to patterns.)

记忆模式理论(Memory Pattern Theory)

一切都有模式.目前您所坐的房间与您生活中的所有其他房间完全一样.问题是"你怎么知道自己在房间里？“您当前的环境是什么,它使您知道自己在一个房间里,而答案是您的环境模式满足您对房间的公认定义的标准.也就是说,它具有进入通道,通常为四个或更多的墙壁,地板和某种形式的屋顶.通常有某种类型的窗口,但这不是必需的.当我们开始谈论房间类型时,我们可以参与其中,但是它们遵循相同的基本模式,并增加了额外的功能,例如书架,图书馆的书籍,厨房的火炉,浴室的浴缸或淋浴器或两者.(Everything has patterns. The room you are sitting in at the moment follows exactly the same patterns as all the other rooms you have ever been in your life. The question is “How do you know you’re in a room?” What is it about your current environment that lets you know you are in a room, and the answer is that the pattern of your environment fulfils your criteria for the accepted definition of a room. That is, it has an entry way, walls, usually four or more, a floor, and a roof of some description. There are usually windows of some sort but these aren’t necessary. We can get more involved when we start talking about types of rooms but they follow the same basic pattern with added extras such as shelves, and books for a library, or a stove for a kitchen, or a Bath or shower or both for the bathroom.) 然后的问题是,我们如何将模式的思想应用到软件可以以某种方式使用的形式,从而可以了解其环境,这使我们回到"联系四人"计划,因为各部分的形式代表了游戏模式.如果您从一个非常基本的角度考虑它,那么"连接​​四人"的游戏就是一组玩家响应以做出自己的动作的颜色模式的集合.某些模式意味着游戏中的某些事物,玩家根据这些模式做出决定,尤其是如果他们以前已经看到过这些模式并且导致游戏获胜或失败.(The question then is how do we apply the idea of patterns to a form that the software can use in such a way that it can learn about its environment, which brings us back to the Connect Four program in that the formations of the pieces represent the patterns of the game. If you think about it on a very basic level, the game of Connect Four is simply a collection of color patterns that the players respond to in order to make their moves. Certain patterns mean certain things within the game, and the players make their decisions based on these patterns, especially if they have seen those patterns before and they have caused the game to be either won or lost.) 至少在三个级别上进行四人连接游戏.有内存级别.玩家知道的"我以前见过这种模式"级别是否可能构成威胁.在某种程度上,玩家会识别出三种相同颜色的图案,这可能会导致玩家或其对手赢得比赛;在这种安全级别中,当前举手既不会赢也不会输游戏.(A game of Connect Four takes place on at least three levels. There is the level of memory. The ‘have I seen this pattern before’ level that the player knows could be a threat or not. There is the level of threat in which the player recognizes a pattern of three pieces of the same color that could result in either the player or their opponent winning the game, and there is the safe level in which the current move will neither win nor lose the game.)

学习连接四(Learning Connect Four)

连接四人游戏使用四种不同类型的模式来确定移动.这些是攻击性,防御性,警告性和战术性的.(The Connect Four game uses four distinct types of patterns to determine a move. These are Aggressive, Defensive, Warning, and Tactical.)

• 好斗的(Aggressive):-侵略性模式是软件知道或认为可以采取行动赢得比赛的地方.(:- An Aggressive Pattern is where the software either knows or thinks that it can make a move that will win the game.)
• 防守型(Defensive):-防御模式是软件知道或认为需要采取某些措施以避免丢失游戏的地方.(:- A Defensive Pattern is where the software either knows or thinks that a certain move is needed to be made in order to avoid losing the game.)
• 警告(Warning):-警告模式是软件知道或认为棋盘内的某个模式可能导致对手玩家赢得游戏的地方.(:- A Warning Pattern is where the software either knows or thinks that a certain pattern within the board can result in the opponent player winning the game.)
• 战术性(Tactical):-战术模式是软件知道或认为董事会内的某个模式可以使软件赢得比赛的地方.(:- A Tactical Pattern is where the software either knows or thinks that a certain pattern within the board can result in the software winning the game.) 每次软件移动时,它都会为开发板创建快照.此时,碎片是什么颜色或它们属于哪个玩家都没有关系.一旦确定了板上的零件,计算机就会开始搜索零件以寻找图案,其中图案被定义为两个或更多相同颜色的零件.自然,零件的颜色决定了该零件组最终定义为哪种类型的图案,因此,由两台或三台计算机组成的一组颜色将被定义为一种激进的图案,软件可以使用该图案来赢得游戏.尽管在此阶段,代码仅涉及确定任何颜色的有效图案.(Every time that the software makes a move, it takes a snapshot of the board. At this point, it doesn’t matter what color the pieces are or which player they belong to. Once the pieces within the board have been determined then the computer starts searching through the pieces looking for patterns, where a pattern is defined as two or more pieces of the same color. Naturally, the color of the pieces determines what type of pattern the group of pieces is eventually defined as, so a group of either two or three pieces of computers color will be defined as being an aggressive pattern that can be used by the software to win the game. Although, at this stage, the code is simply concerned with determining valid patterns of whatever color.) 查找模式的代码示例如下:(An example of the code finding a pattern is:)

/// check top
/// 
patternSquareInfo = connectFourGame.GetSquareInfo( 
    GetIdentifierAbove( square.Identifier ) );
    
if( patternSquareInfo != null )
{
    bool bTwoPieces = false;
    bool bThreePieces = false;

    if( patternSquareInfo.IsOccupied == true )
    {
        nextInfo = connectFourGame.GetSquareInfo( 
            GetIdentifierAbove( patternSquareInfo.SquareIdentifier ) );
        if( nextInfo != null && nextInfo.IsOccupied == false )
        {
            /// allow this square
            /// 
            bTwoPieces = true;
        }
        else 
        {
            if( nextInfo != null )
            {
                bTwoPieces = true;
                bThreePieces = true;
            }
        }
    }

    if( bTwoPieces == true )
    {
        ConnectFourPiece abovePiece = new 
            ConnectFourPiece( false, patternSquareInfo.SquareIdentifier );

        if( piece.IsPieceRed == patternSquareInfo.IsRed )
        {
            abovePiece.IsPieceRed = patternSquareInfo.IsRed;
            abovePiece.Position = PIECEPOSITION.ABOVE;
            abovePiece.Level = 1;
                            
            if( connectFourGame.PlayerIsRed == patternSquareInfo.IsRed )
                abovePiece .IsEnemy = false;
            else
                abovePiece.IsEnemy = true;

            pattern.AddGamePiece( abovePiece );

            if( bThreePieces == false )
            {
                if( patternCollection.IsIn( pattern ) == false )
                    patternCollection.AddPattern( pattern );
            }
        }
    }

    if( bThreePieces == true )
    {
        patternSquareInfo = nextInfo;
        nextInfo = connectFourGame.GetSquareInfo( 
            GetIdentifierAbove( patternSquareInfo.SquareIdentifier ) );
        if( nextInfo != null && nextInfo.IsOccupied == false )
        {
            ConnectFourPiece abovePiece = new 
                ConnectFourPiece( false, patternSquareInfo.SquareIdentifier );

            if( piece.IsPieceRed == patternSquareInfo.IsRed )
            {
                abovePiece.IsPieceRed = patternSquareInfo.IsRed;
                abovePiece.Position = PIECEPOSITION.ABOVE;
                abovePiece.Level = 2;

                if( connectFourGame.PlayerIsRed == patternSquareInfo.IsRed )
                    abovePiece.IsEnemy = false;
                else
                    abovePiece.IsEnemy = true;

                pattern.AddGamePiece( abovePiece );

                if( patternCollection.IsIn( pattern ) == false )
                    patternCollection.AddPattern( pattern );
            }
        }
    }

    pattern.Clear();
    pattern.AddGamePiece( piece );
}

代码从正方形中包含的当前棋子信息开始,在代码示例中,它检查棋盘以查看正方形上方是否有棋子.如果存在,它将检查上方的正方形,以查看上方的正方形是否为有效正方形,并且是否也包含一块.如果原始图块上方的正方形有效,则这是两段式模式,并且只要颜色匹配,就会存储该图块,但是除非其上方的正方形不等于null,否则不会将图案添加到集合中.(The code starts with the current piece information that is contained in the square, and in the code example, it checks the board to see if there is a piece on the square above it. If there is, it then checks the square above that to see if the square above is a valid square and if it also contains a piece. If the square above the original piece is valid then this is a two piece pattern, and as long as the colors match, then the piece is stored, but the pattern is not added to the collection unless the square above it is not equal to null.)

如果上方的正方形是有效的,则这是三块式样,只要颜色相同且上方的正方形有效且未被占用,就会添加剩余的一块.这样做是因为,如果模式包含三段,那么下一个要添加到该模式的段对于某人将是一个成功的举动,并且如果没有位置可移动,那么它根本就与模式无关.(If the square above that is valid then this is a three piece pattern, and the remaining piece is added as long as the colors are the same and the square above is valid and unoccupied. This is done because if the pattern contains three pieces then the next piece to be added to that pattern is going to be a winning move for someone, and if there is no where to move to, then it is simply not relevant as a pattern.) 现在,我们可以对板进行快照,以处理板包含的所有模式.(We are now in the position where we have a snapshot of the board to work on with all the patterns that the board contains being held within the) patternCollection 对象,它只是"连接四个模式"的集合.下一步是确定我们将如何处理这些模式.(object which is just a collection of Connect Four Patterns. The next step is to determine what we are going to do with the patterns.) 以下代码是查找防御模式的代码的摘录.(The following code is a snippet from the code that finds the defensive patterns.)

for( int i=0; i<patternCollection.Count; i++ )
{
    holdingPattern = patternCollection.GetPattern( i );

    if( historicalPatternCollection.IsIn( holdingPattern ) == true )
    {
        holdingPattern = 
            ( ConnectFourPattern )historicalPatternCollection.GetPattern( 
            holdingPattern );

        if( holdingPattern.NumberOfTimesSeenInLosingGame > 0 
            && IsValidPattern( holdingPattern ) == true )
        {
            holdingPattern.Weighting = nMemoryDefensiveWeight;
            defensivePatterns.AddPattern( holdingPattern );
        }
        else
        {
            if( connectFourGame.GetSquareInfo( 
                holdingPattern.GetStartsWith() ).IsRed 
                != connectFourGame.PlayerIsRed )
                defensivePatterns.AddPattern( holdingPattern );
        }
    }
    else /// determine if this is a defensive pattern
    {
        patternSquareInfo = connectFourGame.GetSquareInfo( 
            holdingPattern.GetStartsWith() );

        /// if the pattern square colour is NOT the same as the player colour
        /// it's a pattern that needs defending against
        /// 
        if( patternSquareInfo.IsRed != connectFourGame.PlayerIsRed )
        {
            leftSquareInfo = connectFourGame.GetSquareInfo( 
                GetIdentifierToLeft( patternSquareInfo.SquareIdentifier ) );
            leftSquareNextInfo = connectFourGame.GetSquareInfo( 
                GetIdentifierToLeft( 
                GetIdentifierToLeft( patternSquareInfo.SquareIdentifier ) ) );
            rightSquareInfo = connectFourGame.GetSquareInfo( 
                GetIdentifierToRight( patternSquareInfo.SquareIdentifier ) );
            rightSquareNextInfo = connectFourGame.GetSquareInfo( 
                GetIdentifierToRight( 
                GetIdentifierToRight( patternSquareInfo.SquareIdentifier ) ) );

            if( leftSquareNextInfo != null )
                nextSquareInfo = connectFourGame.GetSquareInfo( 
                GetIdentifierToLeft( leftSquareNextInfo.SquareIdentifier ) );
            else
                nextSquareInfo = null;

            if( leftSquareInfo != null && leftSquareNextInfo != null )
            {
                if( leftSquareInfo.IsOccupied == true 
                    && leftSquareInfo.IsRed != connectFourGame.PlayerIsRed ) 
                {
                    if( leftSquareNextInfo.IsOccupied == false )
                        defensivePatterns.AddPattern( holdingPattern );
                    else
                    {
                        if( nextSquareInfo != null 
                        && nextSquareInfo.IsOccupied == false
                        && leftSquareNextInfo.IsRed 
                        != connectFourGame.PlayerIsRed )
                        {
                        defensivePatterns.AddPattern( holdingPattern );
                        }
                    }
                }
            }

该代码首先循环浏览从快照收集的模式,然后检查该模式是否为历史模式.也就是说,如果代码在先前的游戏中遇到了该特定模式.这是由保存每个游戏结束时从棋盘的最后一个快照收集的所有模式的代码确定的.如果该模式是以前见过的模式,并且已经成为输赢游戏的一部分,则该模式的权重将赋予内存防御权重. (下面将讨论权重在游戏中的使用方式.)然后将模式添加到防御模式列表中.但是,如果该图案以前曾见过,但尚未成为输家游戏的一部分,则只要颜色标准匹配,该图案仍会添加到防御性图案列表中,但不会赋予任何特殊权重.(The code starts by cycling through the patterns that were collected from the snapshot, and then checks to see if the pattern is a historical pattern. That is, if the code has encountered that specific pattern in previous games. This is determined by the code saving all the patterns collected from the last snapshot of the board at the end of each game. If the pattern is a pattern that has been seen before, and has been part of a losing game, then the weighting for the pattern is given a memory defensive weighting. (The way weights are used in the game is discussed below.) The pattern is then added to the defensive patterns list. If, however, the pattern has been seen before but has not been part of a losing game, then the pattern is still added to the defensive patterns list, as long as the color criteria matches, but it does not get given any special weighting.) 产生大量代码的地方是以前从未见过这种模式时,我们必须努力解决.这是通过获取沿我们要检查的轴的正方形,然后进行测试以查看在防御模式下沿轴的正方形是否包含玩家的棋子,以使他们赢得比赛.(The bit that generates a lot of code is when the pattern has not been seen before, and we have to work it out the hard way. This is done by getting the squares along the axis that we want to check, and then testing to see if the squares along the axis in the case of the defensive pattern contain the player’s pieces in such a formation that could lead them to win the game.) 每当计算机进行移动时,对于所有模式类型都将重复此过程,以便计算机在当前快照中构建模式的完整图片.处理完快照中的模式后,计算机将到达以下代码:(This procedure is repeated for all the pattern types whenever the computer makes a move in order for the computer to build a complete picture of the patterns within the current snapshot. Once the patterns within the snapshot have been processed then the computer reaches this code:)

if( aggressivePatterns.Count == 0 && defensivePatterns.Count == 0 
    && warningPatterns.Count == 0 && tacticalPatterns.Count == 0 )
{
    MakeRandomMove();
    return;
}

这基本上意味着,如果在当前快照中未找到任何模式,则只需随机移动即可.请注意,尽管由于至少需要两个相邻的相同颜色的正方形才能形成图案,所以这不只是第一步.一旦有了单独的模式集合,就可以使用流程模式函数来处理所有四个模式集合.(Which basically means that if no patterns have been found within the current snapshot then just make a random move. Note though that as there needs to be at least two adjoining squares of the same color to form a pattern, this will not just be called for the first move. Once we have the separate pattern collections, then we get to the process patterns function which is used to deal with all four of the pattern collections.) 过程模式通过集合中的模式运行,并尝试为各个模式分配权重,以确定对模式做出反应的重要性.(The Process Patterns function through the patterns in the collections, and attempts to assign weights to individual patterns in order to determine how important it is to react to the pattern.) 该函数开始于:(The function starts by:)

if( pattern.NumberOfTimesSeen > 1 || pattern.Weighting == nAggressiveWeight )
{

    /// ensures that a pattern of three pieces will have a higher priority
    /// than a pattern with two pieces
    /// 

    if( pattern.Count == 3 )
    {
        pattern.Weighting = patternWeight * 2;

        /// A winning pattern can be ignored 
        /// if a defensive pattern from memory is used 
        /// and the winning pattern has not been seen before. 
        /// 
 
        if( patternWeight == nAggressiveWeight )
            pattern.Weighting += nMemoryAggressiveWeight;
    }
    else 
        pattern.Weighting = 0;

    /// give added weight if pattern is part of an ending move
    /// 

    if( pattern.IsEndingPattern == true )
        pattern.Weighting += 2;

    if( pattern.ResponsePresent == true )
    {
        if( pattern.NumberOfTimesSeenInWinningGame 
            > pattern.NumberOfTimesSeenInLosingGame )
        {
            pattern.Weighting += pattern.NumberOfTimesSeenInWinningGame * 3;
            bFindMove = false;
        }
        else
            bFindMove = true;
    }

    /// try to increase the warning level in an attempt to detect 
    /// two way tricks
    /// 

    if( patternWeight == nWarningWeight 
        && pattern.NumberOfTimesSeenInLosingGame 
            > pattern.NumberOfTimesSeenInWinningGame )
        pattern.Weighting += pattern.NumberOfTimesSeenInLosingGame;

首先,检查是否已多次查看该模式,除非该模式是始终允许通过的积极模式.这样做是为了减少程序丢失以前从未出现过的获胜动作的情况,因为没有人真正想玩一款能让对手感到获胜的感觉的游戏.然后检查模式是否计数为3.这样做是为了使包含三块的图案比只有两块的图案具有更高的权重,以及在被给予最高评分之前已经看到的三块激进的图案.(It starts by checking that the pattern has been seen more than once unless it is an aggressive pattern in which it is allowed through at all times. This is done to reduce the occurrences of the program missing winning moves that it hasn’t seen before, as no one really wants to play a game where they get the feeling that their opponent is letting them win. It then checks to see if the pattern has a count of three. This is done so that patterns containing three pieces can be given a higher weighting than patterns with only two pieces, and an aggressive pattern of three pieces that has been seen before being given the highest rating of all.) 然后,我们检查看看该模式是否被识别为先前已结束四人制游戏的模式,如果已经识别,则在检查该模式是否已具有上一游戏的编程响应之前,对其权重会稍有增加.如果前一场比赛有回应,并且如果是获胜回应,那么该模式的权重就会增加,因此我们意识到我们需要找到一个办法.(We then check to see if the pattern is recognized as a pattern that has previously ended a game of Connect Four, and if it has, it is given a slightly increased weighting before checking if the pattern already has a programmed response from a previous game. If there is a response from a previous game, and if it is a winning response, then the weighting for the pattern is increased, so we recognize that we need to find a move to make.) 然后,我们检查该模式是否为警告,并尝试增加预防双向技巧的几率.一种双向技巧,即对手有两个相邻的棋子,并且两边的正方形都为空白,这意味着如果对手在两个棋子的两端放置棋子,他们将赢得比赛,并且对此无能为力.完成此操作后,该函数将检查是否需要查找移动.如果存在,它将检查所有可用选项并相应地分配权重.(We then check to see if the pattern is a warning and try to increase the odds for preventing two way tricks. A two way trick being where the opponent has two pieces next to each other with both squares on either side being blank, which means that if the opponent places a piece on either end of the two pieces, they are going to win the game, and nothing can be done about it. Once this is done, the function then checks to see if there is a need to find a move. If there is, it checks all the available options and assigns weights accordingly.) 处理完所有模式之后,代码将提取权重最高的模式并尝试进行移动,尽管在这一点上,如果移动对计算机不利,仍然可以阻止该移动.它将为玩家提供赢得比赛的机会.由于这些条件,计算机在决定随机移动之前只会进行几次尝试移动.(When all the patterns have been processed, the code then extracts the pattern with the highest weighting and attempts to make the move, though at this point, a move can still be blocked if it is not to the computer’s advantage to make it, in that it will give the player a chance to win the game. It is because of these conditions that the computer will only make a few attempts to make a move before deciding to just move randomly.)

记住动作(Remembering the Moves)

在每个游戏结束时初始化程序并将其保存时,将加载游戏的历史模式.在每个游戏结束时,将上次快照收集的模式与历史模式收集中的模式进行比较.如果模式存在,则将其更新,如果不存在,则将其添加到集合中,然后再将整个历史模式集合保存到磁盘.(The historical patterns for the game are loaded when the program is initialized and saved at the end of every game. At the end of each game, the patterns collected by the last snapshot are compared to the patterns in the historical pattern collection. If the pattern exists, it is updated, and if it doesn’t, it is added to the collection before the entire historical pattern collection is saved to disk.) 这是保存的图案数据的示例:(Here is an example of the saved pattern data:)

<ConnectFourPattern>
    <BasicGamePatternSet>
        <PatternID>0</PatternID>
        <BasicGamePiece>
            <PieceID>0</PieceID>
            <IsStartForPattern>True</IsStartForPattern>
            <Square>CF</Square>
            <IsEnemy>False</IsEnemy>
            <PiecePosition>START</PiecePosition>
            <Level>0</Level>
        </BasicGamePiece>
        <BasicGamePiece>
            <PieceID>1</PieceID>
            <IsStartForPattern>False</IsStartForPattern>
            <Square>DF</Square>
            <IsEnemy>False</IsEnemy>
            <PiecePosition>RIGHT</PiecePosition>
            <Level>1</Level>
        </BasicGamePiece>
        <NumberOfTimesSeen>80</NumberOfTimesSeen>
        <NumberOfTimesSeenInWinningGame>38</NumberOfTimesSeenInWinningGame>
        <NumberOfTimesSeenInLosingGame>42</NumberOfTimesSeenInLosingGame>
        <EndingPattern>False</EndingPattern>
        <Weighting>103</Weighting>
        <Response>
            <ResponsePresent>0</ResponsePresent>
        </Response>
    </BasicGamePatternSet>
</ConnectFourPattern>

应当注意,图案和片段的ID在当前程序中均未使用,可能会从该系列的下一个程序中完全删除.(It should be noted that the IDs for both the pattern and the pieces are not used in the current program, and will probably be removed completely from the next program in the series.)

连接四中的权重(Weighting in Connect Four)

代码中有许多预定义的权重常数,它们可以帮助代码准确地确定要对某种模式进行的处理.这些是:(There are a number of predefined weight constants within the code that help the code decide exactly what it is going to do about a certain pattern. These are:)

const int nAggressiveWeight = 10;
const int nDefensiveWeight = 7;
const int nTacticalWeight = 3;
const int nWarningWeight = 5;
const int nMemoryAggressiveWeight = 105;
const int nMemoryDefensiveWeight = 104;
const int nMemoryTacticalWeight = 103;
const int nMemoryWarningWeight = 102;

这些权重分为两类:这些是每种模式类型的标准权重,如果是来自内存的则是该类型的标准权重.这里的想法是,当从内存中识别出一个模式时,使用该程序的可能性要比程序以前从未见过的模式中的权重大得多.当代码决定使用哪种模式时,它会分配初始权重.(These weights are split into two categories; these being the standard weighting for each pattern type, and the standard weighting for the type if it comes from memory. The idea here is that when a pattern is recognized from memory, then it is much more likely to be used than a weight from a pattern that the program hasn’t seen before. When the code is deciding what patterns to use, it assigns the initial weights.)

holdingPattern = ( ConnectFourPattern )historicalPatternCollection.GetPattern( 
    holdingPattern );


if( holdingPattern.NumberOfTimesSeenInLosingGame > 0 
    && IsValidPattern( holdingPattern ) == true )

{
    holdingPattern.Weighting = nMemoryDefensiveWeight;
    defensivePatterns.AddPattern( holdingPattern );
}
else
{
    if( connectFourGame.GetSquareInfo( holdingPattern.GetStartsWith() ).IsRed 
        != connectFourGame.PlayerIsRed )
        defensivePatterns.AddPattern( holdingPattern );
}

上面的代码显示了寻找防御模式时使用的循环.在这一点上,我们知道该模式是历史模式,如果它是输赢游戏中常见的模式,则可以为它分配防御模式的内存权重.但是,如果在输掉的游戏中通常看不到该模式,则留给后面的代码为其分配权重.(The above code shows the loop that is used when finding defensive patterns. At this point, we know that the pattern is in the historical patterns, and if it is a pattern that is commonly seen in a losing game, then we assign it the memory weight for a defensive pattern. If, however, the pattern is not usually seen in a losing game, it is left to the later code to assign a weight to it.) 后面的代码在"处理模式"功能中,该代码计算要放置响应片段的模式的哪一端.这是通过简单的权重来完成的,其中,根据方格的可用性,为每侧分配一个权重,该权重会增加和减少,而获胜的位置将被授予权值.应当注意,“处理模式"功能处理所有模式,并将每种模式类型的权重作为传入的参数.(This later code is in the Process Patterns function where the code calculates which end of the pattern to place the response piece. This is done using simple weights where each side is given a weight that is incremented and decremented according to the availability of the squares, with the winning position being awarded the passed in weight. It should be noted that the Process Patterns function processes all patterns and takes the weight for each pattern type as the passed in parameter.)

平衡问题(A Question of Balance)

程序学习部分的诀窍不仅是使程序在进行过程中不断改进,而且看起来好像也在不断改进.因此,学习不是立即进行的.该代码需要至少看到两次动作,然后才能开始对此作出反应.这意味着,在开始玩游戏时,玩家可能会故意以愚蠢的明显获胜举动赶上计算机,并在计算机识别出该举动之前将其撤下几次.例如,假设您正在董事会的最下面进行直行;首先,计算机应该让您摆脱它.但是一旦发现并记住了移动,程序就应该每次捕获该移动.(The trick to the learning part of the program is not only to get the program to improve as it goes along, but to appear as if it is improving as well. For this reason, the learning does not take place instantly. The code needs to see a move at least twice before it will start to react to it. This means that when playing the game at the start, it is possible for the player to deliberately catch out the computer with what should be a stupidly obvious winning move, and pull off the move a couple of times before the computer recognizes it. For example, say you were doing a straight run across the bottom of the board; at first, the computer should let you get away with it. But once it spots the move and remembers it, the program should trap that move every time.) 虽然,应该指出的是,某些代码在更传统的意义上可以立即计算出最佳动作.这样做是因为,如果游戏要完全依赖于所记忆的内容,那就太容易打败了,而且我个人认为学习的印象不会那么有效.在允许移动计算机之前,还会执行一些更高级别的代码,这样做是为了防止出现真正的明显错误.(Although, it should be pointed out that some of the code works in the more traditional sense of calculating the best move immediately. This is done because if the game was to rely entirely on what it remembered, it would be too easy to beat, and I personally don’t think that the impression of the learning would be as effective. There is also some higher level code that is executed before the computer is allowed to move, this is done to prevent really obvious mistakes.) 这实际上意味着代码在三个单独的级别上运行,这些级别是它记住的东西的初始和基本级别.普通"四连环"游戏所能发挥的中等确定性级别将直接根据棋盘的当前状态计算出一招,而上控制层则可以否决任何举动,如果该举动未能通过规则来满足规则,则该否决权可以被否决,例如,允许玩家在下一步行动中获胜.(This effectively means that the code runs on three separate levels, these being the initial and basic level of things that it remembers. The middle deterministic level that a normal Connect Four game would work on calculates a move directly based on the current status of the board, and the upper control level that can veto any move if it fails to fulfill its rules by making a move that will, for example, allow the player to win with their next move.) 该软件确实具有战术模式,尽管在程序中可能未充分使用该策略模式,因为在开发"四人连线"游戏的智能程度问题上出现了两次问题.鉴于Connect Four游戏的本质,我试图在代码的可玩性和复杂性之间做出折衷,我并不完全相信有可能编写出每次都能获胜的杀手级游戏,即使您可以,玩起来不会很有趣.因此,由于这个原因,尽管有一些关于使用浮动策略以备将来使用的想法,但这些策略并不是该代码的主要部分.因此,任何看似出色的举动都是偶然的,这些举动都是通过让玩家阻止获胜的举动而吸引玩家的,这些获胜的举动允许软件使用玩家刚刚丢下的棋子来赢得比赛.(The software does have a tactics pattern although this is probably under-used within the program as the question arose a couple of times in the development of just how intelligent do you need a Connect Four game to be. I’ve tried to reach a compromise between playability and complication within the code, as given the nature of a Connect Four game, I’m not entirely convinced that it is possible to write a killer game that would win every time, and even if you could, it wouldn’t be any fun to play. So, for this reason, the tactics aren’t a major part of this code though there are some ideas about using tactics floating about for future use. Therefore, any seemingly brilliant moves that catch the player out by getting them to block a winning move that allows the software to use the piece the player just dropped to win the game, are purely incidental.)

参考文献注释(A Note on the References)

下面列出的书是自我开始研究人工智能以来我已阅读的书.所有这些都或多或少地影响了我的思想,在我有意使用书中某些内容的地方,我会在使用它的文章中明确指出.文本中未提及一本书或多本书的事实并不意味着它没有影响力,因为它们都是核心研究的一部分.(The books listed below are the books that I have read since I started researching artificial intelligence. All of them have influenced my thinking to a greater or lesser degree, and where I knowingly use something from a book, I will make it clear in the article that uses it. The fact that a specific book or books are not mentioned in the text does not mean it has not had an influence as they are all part of the core research.)

参考文献(References)

• Tom Archer(2001)Inside C#,Microsoft Press.(Tom Archer (2001) Inside C#, Microsoft Press.)
• Jeffery Richter(2002)应用Microsoft .NET Framework编程,Microsoft Press.(Jeffery Richter (2002) Applied Microsoft .NET Framework Programming, Microsoft Press.)
• Charles Peltzold(2002)使用C#对Microsoft Windows进行编程,Microsoft Press.(Charles Peltzold (2002) Programming Microsoft Windows With C#, Microsoft Press.)
• Robinson等(2001)Professional C#,Wrox.(Robinson et al (2001) Professional C#, Wrox.)
• William R. Staneck(1997),Web发布的专业参考版,Sams.net.(William R. Staneck (1997) Web Publishing Unleashed Professional Reference Edition, Sams.net.)
• Robert Callan,(1999年),<神经网络的本质>,Prentice Hall.(Robert Callan, (1999) The Essence Of Neural Networks, Prentice Hall.)
• Timothy Masters(1993)C ++中的实用神经网络食谱,Morgan Kaufmann(Academic Press).(Timothy Masters (1993) Practical Neural Network Recipes In C++, Morgan Kaufmann (Academic Press).)
• Melanie Mitchell(1999),<遗传算法导论>,麻省理工学院出版社.(Melanie Mitchell (1999) An Introduction To Genetic Algorithms, MIT Press.)
• Joey Rogers(1997)C ++中的面向对象的神经网络,学术出版社.(Joey Rogers (1997) Object-Oriented Neural Networks in C++, Academic Press.)
• 西蒙海金(Simon Haykin)(1999)神经网络综合基金会,普伦蒂斯霍尔(Prentice Hall).(Simon Haykin (1999) Neural Networks A Comprehensive Foundation, Prentice Hall.)
• Bernd Oestereich(2002)在实践中使用UML进行面向对象分析和设计的软件开发,Addison Wesley.(Bernd Oestereich (2002) Developing Software With UML Object-Orientated Analysis And Design In Practice, Addison Wesley.)
• R Beale和T Jackson(1990年),<神经计算入门>,物理研究所出版.(R Beale & T Jackson (1990) Neural Computing An Introduction, Institute Of Physics Publishing.)
• 巴特`科斯科(Bart Kosko,1994年),<模糊思维>,火烈鸟.(Bart Kosko (1994) Fuzzy Thinking, Flamingo.)
• Buckley＆Eslami(2002)模糊逻辑和模糊集简介,物理学出版社.(Buckley & Eslami (2002) An Introduction To Fuzzy Logic And Fuzzy Sets, Physica-Verlag.)
• 史蒂文`约翰逊(Steven Johnson)(2001)出现,企鹅出版社.(Steven Johnson (2001) Emergence, The Penguin Press.)
• 约翰H荷兰(John H. Holland)(1998)<从混乱到有序的出现>,牛津大学出版社.(John H. Holland (1998) Emergence From Chaos To Order, Oxford University Press.)
• 厄尔`考克斯(Earl Cox)(1999),<模糊系统手册>,AP Professional.(Earl Cox (1999) The Fuzzy Systems Handbook, AP Professional.)
• 马克`沃德(1999):虚拟生物,潘.(Mark Ward (1999) Virtual Organisms, Pan.)
• Bonabeau,Dorigo,Theraulaz(1999)从自然系统到人工系统的群智能,牛津大学出版社.(Bonabeau, Dorigo, Theraulaz (1999) Swarm Intelligence From Natural To Artificial Systems, Oxford University Press.)

许可

本文以及所有相关的源代码和文件均已获得The Code Project Open License (CPOL)的许可。