About SF100
Several promising techniques have been proposed to automate different tasks in software testing, such as test data generation for object-oriented software. However, reported studies in the literature only show the feasibility of the proposed techniques, because the choice of the employed artifacts in the case studies (e.g., software applications) is usually done in a non-systematic way. The chosen case study might be biased, and so it might not be a valid representative of the addressed type of software (e.g., internet applications and embedded systems). The common trend seems to be to accept this fact and get over it by simply discussing it in a threats to validity section. To achieve sound empirical results, we randomly selected 100 Java projects from SourceForge, which is the most popular open source repository (more than 300,000 projects with more than two million registered users). The resulting benchmark not only is very large, but more importantly it is statistically sound and representative for open source projects.
For more details about SF100 please consider the following paper, in which it was introduced:
-
![[PDF]](http://www.evosuite.org/wp-content/plugins/papercite/img/pdf.png)
G. Fraser and A. Arcuri, “Sound Empirical Evidence in Software Testing,” in 34th International Conference on Software Engineering, ICSE 2012, June 2-9, 2012, Zurich, Switzerland, 2012, pp. 178-188.
[Bibtex]@inproceedings{ICSE12, author = {Gordon Fraser and Andrea Arcuri}, TITLE="Sound Empirical Evidence in Software Testing", YEAR={2012}, pages = {178--188}, ee = {http://dx.doi.org/10.1109/ICSE.2012.6227195}, booktitle = {34th International Conference on Software Engineering, ICSE 2012, June 2-9, 2012, Zurich, Switzerland}, publisher = {IEEE}, isbn = {978-1-4673-1067-3}, }
Update
2012/03/16: We have resolved some dependency issues and updated the SF100 archives.
Download
All projects selected for the case study are open source projects. To check the individual licenses please check the projects’ websites (see table below). A download containing bytecode of all 100 projects together with their dependencies (check webpages for licenses) is available here:
- SF100: SF100-20120316.tar.gz
- SF100 set up for EvoSuite: SF100-EvoSuite-20120316.tar.gz
Project details
| Project Name | Short Name | Website | Source |
|---|---|---|---|
| Tullibee | tullibee |  |
 |
| A4J (Amazon AWS 4 Java) | a4j | |
|
| Jigen – Java Installer Generator | jigen | |
|
| Remote Invocation Framework (RIF) | rif | |
|
| templateIt | templateit | |
|
| jNFe – Java NFe | jnfe | |
|
| sfm | sfmis | |
|
| gfarceGestionFac | gfarcegestionfa | |
|
| FalseLight | falselight | |
|
| DAWN | water-simulator | |
|
| imSMART Migration | imsmart | |
|
| DSA Chat Program | dsachat | |
|
| jdbacl | jdbacl | |
|
| JState | omjstate | |
|
| BeanBin | beanbin | |
|
| JoomlaTemplateDetails | templatedetails | |
|
| Inspirento | inspirento | |
|
| JSecurity | jsecurity | |
|
| JMCA – Java Method Cohesion Analyzer | jmca | |
|
| NekoMUD | nekomud | |
|
| Geo-Google – Free Address Standardizer | geo-google | |
|
| byuic | byuic | |
|
| Java Wiki Bot Framework | jwbf | |
|
| saxpath | saxpath | |
|
| JNI-InChI | jni-inchi | |
|
| JIPA | jipa | |
|
| Framework for Game Related Research | gangup | |
|
| greencow | greencow | |
|
| APBSmem | apbsmem | |
|
| BluePenguinMail | bpmail | |
|
| Xisemele | xisemele | |
|
| HTTP Analyzer | httpanalyzer | |
|
| JVC – Java View Controller | javaviewcontrol | |
|
| SBMLReader 2 Plugin for Cytoscape | sbmlreader2 | |
|
| Corina | corina | |
|
| SchemaSpy | schemaspy | |
|
| PetSoar – Open Source J2EE PetStore | petsoar | |
|
| javaBB | javabullboard | |
|
| Diffi | diffi | |
|
| cards24 | cards24 | |
|
| Follow | follow | |
|
| asphodel | asphodel | |
|
| lilith | lilith | |
|
| summa | summa | |
|
| Lotus | lotus | |
|
| Nutzenportfolioanalyse | nutzenportfolio | |
|
| DVD Home Video Project | dvd-homevideo | |
|
| resources4j | resources4j | |
|
| Die Bierse | diebierse | |
|
| Pascal interpreter written in Java | biff | |
|
| Java Interactive Profiler | jiprof | |
|
| Lagoon | lagoon | |
|
| Shp2KML | shp2kml | |
|
| DB-Everywhere | db-everywhere | |
|
| Lavalamp | lavalamp | |
|
| jHandballMoves | jhandballmoves | |
|
| hft-bomberman | hft-bomberman | |
|
| FPS370 – First Person Shooter Template | fps370 | |
|
| MyGrid – Distributed Grid | mygrid | |
|
| Sugar : making Java sweeter | sugar | |
|
| framework for dynamic analysis and test | noen | |
|
| dom4j: flexible XML framework for Java | dom4j | |
|
| ObjectExplorer4J | objectexplorer | |
|
| JTailPlus | jtailgui | |
|
| GoToServers Simple FTP Client | gsftp | |
|
| openjms | openjms | |
|
| Google App Engine Application Manager | gae-app-manager | |
|
| Bible Study Tool | biblestudytool | |
|
| macaw | lhamacaw | |
|
| EchoDep Hub and Spoke | echodep | |
|
| Ext4J | ext4j | |
|
| Battlecry | battlecry | |
|
| Free Instant Messenger Server/Client 1.0 | fim1 | |
|
| FixSuite | fixsuite | |
|
| OpenHRE | openhre | |
|
| dash-framework | dash-framework | |
|
| Io Project | io-project | |
|
| CalorieCount | caloriecount | |
|
| The West Fortbattle Player | twfbplayer | |
|
| Wheel | wheelwebtool | |
|
| JavAthena | javathena | |
|
| Internet Protocol Address Calculator | ipcalculator | |
|
| xBus | xbus | |
|
| IFX Framework | ifx-framework | |
|
| SHOP | shop | |
|
| Advanced T-Robots Version 2J | at-robots2-j | |
|
| JAW-BR | jaw-br | |
|
| JOpenChart Toolkit | jopenchart | |
|
| JIGL (Java Image and Graphics Library) | jiggler | |
|
| dcParseArgs | dcparseargs | |
|
| Class Viewer | classviewer | |
|
| JCVI Java Common | jcvi-javacommon | |
|
| QuickServer – TCP server framework | quickserver | |
|
| JCLO: A Java Command Line Option package | jclo | |
|
| CellWars2009 | celwars2009 | |
|
| HEAL: Health Education Assets Library | heal | |
|
| Feudalism Game | feudalismgame | |
|
| Transmitter Locator | trans-locator | |
|
| Newzgrabber | newzgrabber | |
|
| JGAAP | jgaap | |
|