Using EvoSuite on the Commandline
EvoSuite comes as an executable jar file which you can be called as follows:
java -jar evosuite.jar <options>
Running EvoSuite
To generate a test suite using the whole test suite generation method (chromosome = testsuite, fitness = covering all goals), use the following command:
java -jar evosuite.jar <target> [options]
The target can either be a class using:
-class <ClassName>
or a package prefix, in which case EvoSuite tries to generate a test suite for each class in the classpath that match the prefix:
-prefix <PrefixName>
or a classpath entry, in which case EvoSuite tries to generate a test suite for each class in the given classpath entry:
-target <jar file or directory>
The most important options are:
- -projectCP <classpath> sets the classpath for test generation.
- -seed: Seed to use for the random number generator (when not specified, EvoSuite will use a new random seed every time it is called)
Alternatively, to create a test suite by addressing each test goal by itself (chromosome = testcase, fitness = coverage of one goal, e.g. a branch), use the following command:
java -jar evosuite.jar -generateTests <target> [options]
Configuring EvoSuite
All parameters can be set on the command line. To see the available command line options, use:
java -jar evosuite.jar -help
In addition, EvoSuite has many parameters which can be set on the command line by passing properties using the common
-Dkey=value
syntax. For example, to run EvoSuite for only 1 minute instead of its default of 10 minutes, run:
java -jar evosuite.jar -generateSuite -Dsearch_budget=60 -Dstopping_condition=MaxTime
Other common parameters are to turn off assertion generation (-Dassertions=false) or to deactivate the sandbox in which tests are executed with a restrictive security manager (-Dsandbox=false).
To see all available parameters, use:
java -jar evosuite.jar -listParameters
Optionally, it is possible to set up EvoSuite in a project directory to save parameters and the classpath. This is done using the following command:
java -jar evosuite.jar -setup <target> <classpath> [<classpath> ...]
- The target can be a class name, a package name, or a classpath entry (e.g. path or jar file)
- The classpath is a list of path entries.
This will create a directory evosuite-files containing the file evosuite-files/evosuite.properties which mainly contains the settings for the genetic algorithm, but also other settings such as whether to create HTML reports. To find out which parameters EvoSuite supports, please check the evosuite.properties file until the documentation is complete; each property in evosuite.properties contains a description.
For example, to setup EvoSuite for a project test which contains its sources in test/build/classes and requires library test/lib/foo.jar the setup would look like the following:
$ cd test $ java -jar /path/to/evosuite.jar -setup build/classes lib/foo.jar
As an alternative to setting properties on the commandline using the -Dkey=value syntax, they can also be changed by editing the property file
evosuite-files/evosuite.properties
in the project under test. To change a value, simply uncomment the entry and enter the new desired value.
EvoSuite Output
Depending on the configuration, EvoSuite creates two types of outputs:
- Test suites are put in the directory evosuite-tests
- Reports are put in the directory evosuite-report
A report consists of CSV data files (evosuite-report/statistics.csv contains the data collected after each run, and evosuite-report/data ontains one CSV file for each run with detailed information), and optionally an HTML report (evosuite-report/report_generation.html). The HTML report contains one entry for each run (i.e. one for each run on a class), listing all the parameters, the test cases, the coverage, and if plot=true in evosuite.properties it also contains plots of the fitness, length, and size histories.
How to run the generated JUnit tests
The generated JUnit tests rely on the EvoSuite framework. Such framework needs to be on the classpath when the tests cases are compiled and run. You need to add the evosuite-standalone-runtime.jar jar file to the project. Alternatively, you can directly use the evosuite.jar executable as runtime dependency (it is a superset of the standalone runtime), but it would bring in many non-necessary third-party libraries into the build.
Classpath on Windows
The Windows operating systems have limitations when it comes to pass long inputs to spawn processes. On large projects with complex classpaths, using -projectCP might not work. You can however write the classpath on a single line text file, and then rather use -DCP_file_path=<pathToThatFile>