diff --git a/SBGames_2015/Article.pdf b/SBGames_2015/Article.pdf deleted file mode 100644 index a01ba82..0000000 Binary files a/SBGames_2015/Article.pdf and /dev/null differ diff --git a/SBGames_2015/Article.tex b/SBGames_2015/Article.tex index dac6afa..a2211f1 100644 --- a/SBGames_2015/Article.tex +++ b/SBGames_2015/Article.tex @@ -13,28 +13,28 @@ \usepackage{xcolor} \newcommand{\toDo}[1]{\textcolor{red}{#1}} +\newcommand{\gnramos}[1]{\textcolor{blue}{#1}} %% Paper title. \title{Comparing the Performance of Finite-State Machines with Different Numbers of States on TORCS} %% Author and Affiliation (multiple authors). Use: and between authors -\author{Bruno H. F. Macedo\\Gabriel F. P. Araujo\\ \textit{University of Brasília}\\ \textit{Technogy Faculty} (FT)\\ - \and Gabriel S. Silva\\Matheus C. Crestani\\Yuri B. Galli\\ \textit{University of Brasília}\\ - \textit{Technogy Faculty} (FT)\\ - \and Guilherme N. Ramos\\ \textit{University of Brasília}\\Professor at - \textit{Department of Computer Science} (CIC)\\ +\author{Bruno H. F. Macedo\\Gabriel F. P. Araujo\\ + \and Gabriel S. Silva\\Matheus C. Crestani\\\textit{University of Bras\'{i}lia} + \and Yuri B. Galli\\ Guilherme N. Ramos } + \contactinfo{\{bhfmacedo,yurigalli\}@gmail.com \\ - gnramos@cic.unb.br \\ + gnramos@unb.br \\ \gnramos{so um contato - do autor correspondente} } %% Keywords that describe your work. -\keywords{finite-state machines, computer games, TORCS, SCRC, artificial intelligence, genetic algorithm, self-driving car} +\keywords{finite-state machine, computer games, TORCS, SCRC, artificial intelligence, genetic algorithm, self-driving car} %% Start of the paper -% Attention: As you need to insert EPS images in Postscript, -% you need to insert PDF images into PDFs. -% In the text, extensions cancbe omitted (latex use .eps, pdflatex get .pdf) +% Attention: As you need to insert EPS images in Postscript, +% you need to insert PDF images into PDFs. +% In the text, extensions cancbe omitted (latex use .eps, pdflatex get .pdf) % To convert them: epstopdf myimage.eps \begin{document} @@ -51,36 +51,38 @@ %% Abstract section. \begin{abstract} - + \toDo{Alterar cabecalho do programa para aceitar portugues} - + \gnramos{para que? (o texto é em Ingles - e poderia ser em inglês mais simples/objetivo) - to-do: \textbf{escrever 10 paginas}} + + \gnramos{reescrever. abstract deveria ter $\pm$ 5 frases objetivas: 1) problema 2) proposta 3) mérito 4) validação 5) resultados} This work presents two different approaches developed to control a self-driving car in the racing environment simulated by TORCS. The problematic revolves around the representation of the complex behaviour that - describes a pilot during a race, whose proposed solutions are two different finite-state machines; while one + describes a pilot during a race, whose \gnramos{refere-se ao comportamento?} proposed solutions are two different finite-state machines; while one method treats the problem with a more partitioned approach, the other tries to generalize the possible - situations. Furthermore, in order for the final controller to attain competitive status, i.e. parameter fine - tuning, the incorporation of a genetic algorithm was performed. Evolving the driver using this method - initially required the definition of a set of parameters, where the optimization lied effectively. Validation - occurred through exhaustively testing the configurations of the two approaches, comparing them along with - artificial intelligences provided by the platform and also State of the Art implementations. - + situations. Furthermore, in order for the final controller to attain competitive status, i.e. parameter fine + tuning \gnramos{explicacao desnecessaria}, the incorporation of a genetic algorithm was performed\gnramos{``foi feita a incorporação de GA''? melhor ser objetivo: ``utilizou-se GA para'' }. Evolving the driver using this method + initially required the definition of a set of parameters, where the optimization lied effectively\gnramos{o que isso quer dizer? (lied effectively)}. Validation + occurred through exhaustively \gnramos{não foi exaustivo} testing the configurations of the two approaches, comparing them along with + artificial intelligences provided by the platform and also State of the Art implementations\gnramos{tantos detalhes não são necessários}. + \end{abstract} %% The ``\keywordlist'' command prints out the keywords. \keywordlist \contactlist - + % \section{Topics}% -% +% % Talk about: -% +% % \begin{itemize} % \item How the problem appeared; % \item The approach chosen to solve it; % \item How we evolved it by applying a genetic algorithm; % \item How we improved the process of evolving through virus infection. % \end{itemize} -% +% % Include: % \item State-of-art exposure; % \item Model definition (theoretical outline); @@ -100,8 +102,8 @@ %\input{Future_Works} %\section*{Acknowledgements} - + \bibliographystyle{sbgames} \bibliography{Bibliography} - + \end{document} diff --git a/SBGames_2015/Introduction.tex b/SBGames_2015/Introduction.tex index 22fbcae..bc7b8d6 100644 --- a/SBGames_2015/Introduction.tex +++ b/SBGames_2015/Introduction.tex @@ -1,30 +1,32 @@ \section{\textbf{Introduction}} \label{sec:Intro} - + Automation of day to day tasks is an endeavour that has moved a large amount of scientific resources in the - recent history. One specific example is target of studies around the globe by a lot of universities, companies - and industries, which is the automation of vehicles, more specifically, automobiles. The objective of such + recent history\gnramos{referencia?}. One specific example is target of studies around the globe by a lot of universities, companies + and industries\gnramos{trocar por target of research}, which is the automation of vehicles, more specifically, automobiles. The objective of such attempts is the development of artificial intelligences capable of driving a car safely, with traffic law enforcement, real-time decision making, efficiency and, in addition, resource economy - as with gas or even - time. The practical applications of such controllers in autonomous vehicles are truly numerous, for example, the - researches pursued by DARPA~\cite{DARPA}. + time\gnramos{referencia?}. The practical applications of such controllers in autonomous vehicles are truly numerous, for example, the + researches\gnramos{quais?} pursued by DARPA~\cite{DARPA}\gnramos{site nao e referencia! se quiser por o link, use nota de rodape}. + + \gnramos{melhorar a contextualizacao do problema. acrescentar justificativas para pesquisa da solucao} The Simulated Car Racing Championship (SCRC), using the platform TORCS (The Open Racing Car Simulator), has - brought an excellent environment for benchmarking AI approaches for the problem of autonomous car controllers. - Notwithstanding, the optimum behaviour of a controller is a complex matter in its full extent; for this purpose, - the strategy adopted to deal with it was to divide the problem into smaller portions, i.e., less complicated - subproblems, in order to implement a finite-state machine that admittedly covers all necessary behaviours. Later + brought an excellent environment for benchmarking AI approaches for the problem of autonomous car controllers\gnramos{referencia?}. + Notwithstanding, the optimum behaviour of a controller is a complex matter in its full extent\gnramos{confuso. encontrar o comportamento?}; for this purpose, + the strategy adopted to deal with it \gnramos{apresentar o que tem sido feito antes de mostrar a proposta} was to divide the problem into smaller portions, i.e., less complicated + subproblems, in order to implement a finite-state machine that admittedly covers all necessary behaviours \gnramos{referencia?}. Later on this paper, each of those subproblems are treated as the states of the referenced finite-state machine, which individually incorporate different but complementary parts of the integral behaviour. - - One way to enhance the performance of the created controllers is by comparing each and every possible set of + + One way to enhance the performance of the created controllers is by comparing \gnramos{evaluating} each and every possible set of parameters or configurations it could assume, but that choice is not always possible due to time and space - complexities. For that reason, although hand-coded methods might present satisfactory outcomes in questions of - such intricacy similar to the one discussed in this paper, they will hardly ever outperform the ones that are + complexities. For that reason, although hand-coded methods \gnramos{o que são?} might present satisfactory outcomes in questions of + such intricacy \gnramos{quais?} similar to the one discussed in this paper, they will hardly ever outperform the ones that are computer-aided on account of the incisive and indefatigable pace that the latter perform the task, which results - in more possibilities tested. Considering this, after an initial structure of the controller was designed, a + in more possibilities tested\gnramos{referência? por que nao?}. Considering this, after an initial structure of the controller was designed, a method of computer-aided fine tuning was assimilated to it, which was a genetic algorithm. - - Apart from this Introduction, this paper is structured as follows: Section~\ref{sec:Environment} introduces + + Apart from this Introduction, \gnramos{trocar por The rest of} this paper is structured as follows: Section~\ref{sec:Environment} introduces TORCS, the working environment used in the context of this task, along with the competition, SCR Championship, that currently represents the utmost metric to evaluate the performance of the controller proposed, and what is already being done at this context in related works, highlighting the strategies that are standing out - State of @@ -34,4 +36,4 @@ \section{\textbf{Introduction}} \label{sec:Intro} validation process occurred, through the methodology, the experiments and the results achieved, including their correspondent analysis; Section~\ref{sec:Conclusions} provides conclusions about the acquired results, which establish the comparison between the finite state machines with few and with moderate number of states, pointing - out prospects about what is yet to be done in future works to improve those results. + out prospects about what is yet to be done in future works to improve those results. \gnramos{excessivamente detalhado} diff --git a/SBGames_2015/TORCS_Environment.tex b/SBGames_2015/TORCS_Environment.tex index e7f58ea..7f3c836 100644 --- a/SBGames_2015/TORCS_Environment.tex +++ b/SBGames_2015/TORCS_Environment.tex @@ -1,116 +1,115 @@ -\section{\textbf{TORCS Environment and SCRC}} \label{sec:Environment} +\section{The Simulation Environment} \label{sec:Environment} - The Open Racing Car Simulator (TORCS)~\cite{TORCS} is a widely used platform for benchmarking AI, renowned for - its highly credible physics modeling engine and yet user-friendly interface for car racing simulation. One of + The Open Racing Car Simulator (TORCS) is a widely used platform for benchmarking AI, renowned for + its highly credible physics modeling engine and yet user-friendly interface for car racing simulation~\cite{TORCS}. One of the many other qualities of this open-source simulator is its portability, concerning multi-platform environments - - such as different operational systems and architectures - and multi-programming language ones alike. + - such as different operational systems and architectures - and multi-programming language ones alike.\gnramos{relevancia neste contexto?} -\subsection{TORCS as platform, interface and environment} \label{subsec:TORCS} - - TORCS deals with a robotic simulation in the format of car racing. It serves both as an ordinary AI game and a - research platform~\cite{2009}, providing through the interface with its competitions a complete sensor-based +\subsection{TORCS} \label{subsec:TORCS} + + TORCS deals with a robotic simulation in the format of car racing\gnramos{???}. It serves both as an ordinary AI game\gnramos{o que é um jogo de IA?} and a + research platform~\cite{2009}, providing through the interface with its competitions\gnramos{que competicoes?} a complete sensor-based interaction system in which the developer is able to interpret received parameters of the car - such as speed in X, Y and even Z axes - and construe them in order to control the car through programming its actuators, some of - which are acceleration and steering. This behaviour is widely embraced when it comes to artificial intelligence - methods - a set of inputs to result in a set of outputs. - + which are acceleration and steering. This behavior is widely embraced when it comes to artificial intelligence + methods - a set of inputs to result in a set of outputs\gnramos{nao e um comportamento}. + Another credibility factor for this platform is its non-punctual cars, each of which possess a body with sensors - and actuators, as mentioned earlier, and interact with other cars in the environment by a life-like collision + and actuators, as mentioned earlier\gnramos{repetitivo}, and interact with other cars in the environment by a life-like collision system. Nevertheless, TORCS is still a simulator, and its limitations, along with the defined racing environment - and the modeled robotic car, are more than likely to affect any results obtained. This is an inherited + and the modeled car, are more than likely to affect any results obtained. This is an inherited characteristic of any real-life problem simulation, what in academia is denominated \emph{reality gap}~\cite{RG}, and it stems from the simplifications attained concerning the car models, the technical features of the tracks, and so forth. \subsection{The SCR Championship} \label{subsec:SCRC} - The Simulated Car Racing Championship (SCRC)~\cite{SCR} is an example of a well-known competition which utilizes - TORCS as interface. Being an event joining three competitions held at major scientific conferences, such as + The Simulated Car Racing Championship (SCRC) is an example of a well-known competition which utilizes + TORCS as interface~\cite{SCR}. Being an event joining three competitions held at major scientific conferences, such as \emph{IEEE Congress on Evolutionary Computation}~\cite{CEC}, \emph{Genetic and Evolutionary Computation - Conference}~\cite{GECCO} and \emph{IEEE Conference on Computational Intelligence and Games}~\cite{CIG}, it is + Conference}~\cite{GECCO}, and \emph{IEEE Conference on Computational Intelligence and Games}~\cite{CIG}\gnramos{conferencias nao sao referencias}, it is an accepted metric of evaluation in the fields of Evolutionary Computation and Computational Intelligence - regarding Games. - + regarding Games\gnramos{referencia?}. + The SCRC features encapsulation with its interface towards TORCS, meaning there is an abstraction layer between - the two and, in addition, some of the information about the racing execution remains hidden from the controllers, + the two and\gnramos{desnecessario explicar isso}, in addition, some of the information about the racing execution remains hidden from the controllers, such as the geometrical format of the track and its category. SCRC and TORCS communicate through UDP packages, and each player receives information regarding the sensors of the car, some of which were already presented - earlier on this section. The figure 1 ilustrate the data avaliable at the torcs and client(controller) layer + earlier on this section\gnramos{desnecessario repetir}. Figure 1 ilustrates the data avaliable at the TORCS and client (controller) layer of abstraction. - \begin{figure}[h] + \begin{figure}[h] \centering \includegraphics[width=250pt]{Figure1} - \caption{\label{fig:Figure1}Avaliable data inside TORCS and at the client.} + \caption{\label{fig:Figure1}Avaliable data.} \end{figure} - + A brief description of the sensors is as follows: the driver can use a set of sensors to acquire relative position, speed, and car condition such as rpm, damage and gas. Useful information about the car during a race can be mined from the data provided by them; for example, there are two sets of sensors labeled \emph{track} and \emph{trackpos} that inform the position of the racer according to a desired direction and according to the track track axis, respectively, and these two sets can be combined to inform a approximated - location of the car. - + location of the car.\gnramos{paragrafo desnecessario} + The complete sensorial input information can be found at the Simulated Car Racing Championship Competition - Software Manual~\cite{SCRC}, within section 7.7. Noise can be incorporated to the aforementioned sensors, + Software Manual~\cite{SCRC}. Noise can be incorporated to the aforementioned sensors, option that is inexorably present during the actual competition and will be dealt with in the section Future - Works. + Works.\gnramos{Bastaria ``noise can be introduced in the sensors''. o restante e desnecessario, e subjetivo (``inexorably''?!) informacoes trabalhos futuros aparecem apenas no final...} - Race tracks differ from each other concerning sensorial input if they are part of different group types, which - are categorized into \emph{Road}, \emph{Dirt} and \emph{Oval}. The races from the SCRC select the track types + Race tracks %differ from each other concerning sensorial input if they are part of different group types, which + are categorized into \emph{Road}, \emph{Dirt} and \emph{Oval}\gnramos{referencia?}. The races from the SCRC select the track types randomly, and adopt the following structure: - + \begin{itemize} - + \item \emph{Warm-up} - when each driver is able to explore the track and deduce information from it at will, for a limited time on their own. This encourages the application of online learning techniques that can be useful in a race on the same track; - + \item \emph{Qualifiers} - a stage placing each driver in a race alone against the clock to select the best ones to compete in the \emph{Actual Race}; - + \item \emph{Actual Race} - when finally the eight best drivers from the \emph{Qualifiers} race against each other. - + \end{itemize} - + \gnramos{desnecessario, mandar ler a referencia para detalhes} + The reason why TORCS presents itself as a satisfactory AI benchmark, in combination with SCRC, is because even - though there is an infinity of possibilities on how the sensorial input received from the server can be - translated into the behaviour of the actuators, they can all be compared in a race, which has a robust and steady + though there is an infinity \gnramos{palavra com significado muito especifico, melhor usar outra} of possibilities on how the sensorial input received from the server can be + translated into the behavior of the actuators, they can all be compared in a race, which has a robust and steady scoring and evaluational system. In other words, there are many different approaches concerning how to teach the racer encoded by the developers to drive in a racing competition only with the information given by the sensors, and the metric to that issue is the performance on the race itself. \subsection{Related Works and State of the Art} \label{subsec:Related} - By controller, let it be understood that the subject is the programming code that in fact controls the car/driver/racer within racing environment. Some examples of awarded controllers and their driving methods - will be presented in this subsection. They are what can be called the State of the Art among TORCS, and it is + will be presented in this subsection\gnramos{frase desnecessaria}. They are what can be called the State of the Art among TORCS, and it is very common among them the incorporation of machine learning methods, along with other evolving techniques - using artificial intelligence. Instinctively, as the nature of the problem comprises evolution by experience, - learning procedures tend to enhance performance and competitiveness. Essentially, there are two ways of + using artificial intelligence\gnramos{frase desnecessaria}. Instinctively, as the nature of the problem comprises evolution by experience, + learning procedures tend to enhance performance and competitiveness\gnramos{os procedimentos nao tem instintos}. Essentially, there are two ways of evolving controllers: Online Learning and Offline Learning, the first meaning that improvements are achieved during the actual race execution time and the latter that it is done before the competition, on the account of - the developers themselves and with their own resources. - - The current champion of the SCR Championship is the controller \emph{Mr. Racer}~\cite{MrRacer}, and it has - proven to be the State of the Art by winning at least the last three competitions that happened. The authors - of this implementation learn parameters offline through Covariance Matrix Adaptation Evolution Strategy + the developers themselves and with their own resources\gnramos{aprendizado online pode ser feito antes da competição, a diferença é outra}. + + The current champion\gnramos{que ano?} of the SCR Championship is the controller \emph{Mr. Racer}~\cite{MrRacer}, and it has + proven to be the State of the Art by winning at least the last three competitions that happened\gnramos{melhor falar apenas que ganhou as ultimas 3. mas foram etapas ou campeonato?}. The authors + of this implementation \gnramos{quem aprender não é o controlador?}learn parameters offline through Covariance Matrix Adaptation Evolution Strategy (CMA-ES), use regression and low-pass filtering to reduce noise impact, distinguish normal asphalted roads from dirt-based ones for behavioral separation and implement an authentic opponent-handling method. Their Online Learning consists on the track model selection to categorize into dirt or asphalt, choice of databased - sets of parameters that best fit the track and the tuning of a target speed for all its corners. - + sets of parameters that best fit the track and the tuning of a target speed for all its corners.\gnramos{enfase no aprendizado, mas o que interessa nesse artigo é o modelo do controlador} + Another renowned controller is \emph{AUTOPIA}~\cite{AUTOPIA}. According to the founders of the - competition~\cite{SCRC} and the authors of \emph{Mr. Racer} themselves, it is a competitive match, with the - potential to even be the best one available, but since no entries were received from them in a while, their - means of winning a competition were somewhat restrained. Nevertheless, assessing its performance is - worthwhile, and its description is the implementation of a modular Fuzzy Architecture, whose division contains + competition~\cite{SCRC} and the authors of \emph{Mr. Racer} themselves, it is a competitive match\gnramos{a quem?}, with the + potential to even be the best one available, but since no entries were received from them in a while\gnramos{muito subjetivo, especificar a quantidade de tempo}, their + means of winning a competition were somewhat restrained\gnramos{??? se não se inscrevem tem restrições para ganhar?!}. Nevertheless, assessing its performance is + worthwhile\gnramos{por que?}, and it implements of a modular Fuzzy Architecture, whose division contains gear, steering and speed control. Their controller is optimized by means of a genetic algorithm for Offline Learning, and by means of landmarking the lane exit points for further speed reduction for Online Learning. - - These and other controller exemplifications served as criteria for the analysis and development of the - approach presented in this paper. Aspects incorporated and adapted from them feature modularity, offline learning - through genetic algorithms, online learning through landmarking and choosing sets of parameters for different - categories of tracks, etc. Aspiring to design a controller capable of incorporating these features, the design of - a model was proposed and is presented in the succeeding section. - \ No newline at end of file + + These and other controller exemplifications\gnramos{referências?} served as criteria for the analysis and development of the + approach presented in this paper. Aspects incorporated and adapted from them feature modularity, Offline Learning + through genetic algorithms, Online Learning through landmarking and choosing sets of parameters for different + categories of tracks, among others. Aspiring to design a controller capable of incorporating these features, the design of + a model was proposed and is presented in the following section.