Monthly Archives: September 2021

New 1.2.0 release

A new version 1.2.0 of EvoSuite has now been released. The main changes are:

  • Java 9 support for DSE (@ilebrero)
  • Updated Docker configuration (@mitchellolsthoorn)
  • Support for JUnit 5 output (@VoglSebastian)
  • Refactoring of the generics-related code (@VoglSebastian)
  • Java 9 support for Eclipse plugin
  • General refactorings and reformatting
  • Bug fixes for various crashes and minor issues

Release data is available 1.2.0 release results.

As usual, the source code release can be found on GitHub at https://github.com/EvoSuite/evosuite/releases/tag/v1.2.0.

Recent work on improving the fitness landscape

The fitness landscape faced by EvoSuite is challenging. Here are a couple of recent investigations to address this problem:

  • [PDF] S. Vogl, S. Schweikl, and G. Fraser, “Encoding the certainty of boolean variables to improve the guidance for search-based test generation,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2021, pp. 1088-1096.
    [Bibtex]
    @inproceedings{GECCO21_Fitness,
      title={Encoding the certainty of boolean variables to improve the guidance for search-based test generation},
      author={Vogl, Sebastian and Schweikl, Sebastian and Fraser, Gordon},
      booktitle={Proceedings of the Genetic and Evolutionary Computation Conference},
      pages={1088--1096},
      year={2021}
    }
  • [PDF] [DOI] Y. Lin, Y. S. Ong, J. Sun, G. Fraser, and J. S. Dong, “Graph-Based Seed Object Synthesis for Search-Based Unit Testing,” in Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, New York, NY, USA, 2021, p. 1068–1080.
    [Bibtex]
    @inproceedings{FSE21_ObjectSeed,
    author = {Lin, Yun and Ong, You Sheng and Sun, Jun and Fraser, Gordon and Dong, Jin Song},
    title = {Graph-Based Seed Object Synthesis for Search-Based Unit Testing},
    year = {2021},
    isbn = {9781450385626},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3468264.3468619},
    doi = {10.1145/3468264.3468619},
    abstract = {Search-based software testing (SBST) generates tests using search algorithms guided
    by measurements gauging how far a test case is away from exercising a coverage goal.
    The effectiveness of SBST largely depends on the continuity and monotonicity of the
    fitness landscape decided by these measurements and the search operators. Unfortunately,
    the fitness landscape is challenging when the function under test takes object inputs,
    as classical measurement hardly provide guidance for constructing legitimate object
    inputs. To overcome this problem, we propose test seeds, i.e., test code skeletons
    of legitimate objects which enable the use of classical measurements. Given a target
    branch in a function under test, we first statically analyze the function to build
    an object construction graph that captures the relation between the operands of the
    target method and the states of their relevant object inputs. Based on the graph,
    we synthesize test template code where each "slot" is a mutation point for the search
    algorithm. This approach can be seamlessly integrated with existing SBST algorithms,
    and we implemented EvoObj on top of EvoSuite. Our experiments show that EvoObj outperforms
    EvoSuite with statistical significance on 2750 methods over 103 open source Java projects
    using state-of-the-art SBST algorithms.},
    booktitle = {Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
    pages = {1068–1080},
    numpages = {13},
    keywords = {object oriented, code synthesis, software testing, search-based},
    location = {Athens, Greece},
    series = {ESEC/FSE 2021}
    }
  • [PDF] Y. Lin, J. Sun, G. Fraser, Z. Xiu, T. Liu, and J. S. Dong, “Recovering fitness gradients for interprocedural Boolean flags in search-based testing,” in Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2020, pp. 440-451.
    [Bibtex]
    @inproceedings{ISSTA20_Gradient,
      title={Recovering fitness gradients for interprocedural Boolean flags in search-based testing},
      author={Lin, Yun and Sun, Jun and Fraser, Gordon and Xiu, Ziheng and Liu, Ting and Dong, Jin Song},
      booktitle={Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis},
      pages={440--451},
      year={2020}
    }

Recent SBST tool competition results

EvoSuite participated in the various instances of the SBST tool competition (https://sbst21.github.io/tools/) over the recent years, here is a list of all the results that have been missing on the webpage so far. In all these cases, EvoSuite achieved the highest overall score in the competition:

  • [PDF] S. Vogl, S. Schweikl, G. Fraser, A. Arcuri, J. Campos, and A. Panichella, “EVOSUITE at the SBST 2021 Tool Competition,” in 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST), 2021, pp. 28-29.
    [Bibtex]
    @inproceedings{SBST21_competition,
      title={EVOSUITE at the SBST 2021 Tool Competition},
      author={Vogl, Sebastian and Schweikl, Sebastian and Fraser, Gordon and Arcuri, Andrea and Campos, Jose and Panichella, Annibale},
      booktitle={2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST)},
      pages={28--29},
      year={2021},
      organization={IEEE}
    }
  • [PDF] A. Panichella, J. Campos, and G. Fraser, “EvoSuite at the SBST 2020 Tool Competition,” in Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, 2020, pp. 549-552.
    [Bibtex]
    @inproceedings{SBST20_competition,
      title={EvoSuite at the SBST 2020 Tool Competition},
      author={Panichella, Annibale and Campos, Jos{\'e} and Fraser, Gordon},
      booktitle={Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops},
      pages={549--552},
      year={2020}
    }
  • [PDF] J. Campos, A. Panichella, and G. Fraser, “EvoSuiTE at the SBST 2019 tool competition,” in 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST), 2019, pp. 29-32.
    [Bibtex]
    @inproceedings{SBST19_competition,
      title={EvoSuiTE at the SBST 2019 tool competition},
      author={Campos, Jos{\'e} and Panichella, Annibale and Fraser, Gordon},
      booktitle={2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST)},
      pages={29--32},
      year={2019},
      organization={IEEE}
    }
  • [PDF] [DOI] G. Fraser, J. M. Rojas, and A. Arcuri, “EvoSuite at the SBST 2018 Tool Competition,” in Proceedings of the 11th International Workshop on Search-Based Software Testing, New York, NY, USA, 2018, p. 34–37.
    [Bibtex]
    @inproceedings{SBST18_competition,
    author = {Fraser, Gordon and Rojas, Jos\'{e} Miguel and Arcuri, Andrea},
    title = {EvoSuite at the SBST 2018 Tool Competition},
    year = {2018},
    isbn = {9781450357418},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3194718.3194729},
    doi = {10.1145/3194718.3194729},
    abstract = {EvoSuite is a search-based tool that automatically generates executable unit tests
    for Java code (JUnit tests). This paper summarises the results and experiences of
    EvoSuite's participation at the sixth unit testing competition at SBST 2018, where
    EvoSuite achieved the highest overall score (687 points) for the fifth time in six
    editions of the competition.},
    booktitle = {Proceedings of the 11th International Workshop on Search-Based Software Testing},
    pages = {34–37},
    numpages = {4},
    location = {Gothenburg, Sweden},
    series = {SBST '18}
    }