Towards Property-Based Tests in Natural Language

Gordon, Colin S.

44th IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results, ICSE (NIER), May 2022, doi: 10.1145/3510455.3512781

pdf url youtube arXiv

Abstract

We consider a new approach to generate tests from natural language. Rather than relying on machine learning or templated extraction from structured comments, we propose to apply classic ideas from linguistics to translate natural-language sentences into executable tests. This paper explores the application of combinatory categorial grammars (CCGs) to generating property-based tests. Our prototype is able to generate tests from English descriptions for each example in a textbook chapter on property-based testing.

Bibtex

@inproceedings{icsenier22,
  author    = {Gordon, Colin S.},
  title     = {Towards Property-Based Tests in Natural Language},
  booktitle = {44th {IEEE/ACM} International Conference on Software Engineering: New Ideas and Emerging Results, {ICSE} {(NIER)}},
  publisher = {{IEEE}},
  year      = {2022},
  ieee = {https://ieeexplore.ieee.org/document/9793513},
  abbr = {ICSE NIER},
  bibtex_show = {true},
  url = {https://arxiv.org/abs/2202.03616},
  arxiv = {2202.03616},
  doi = {10.1145/3510455.3512781},
  youtube = {https://youtu.be/uQWuQoR5nbM},
  selected={true},
  pdf = {papers/icsenier22.pdf},
  note = {Acceptance Rate 27\% (26/97).},
  abstract = {We consider a new approach to generate tests from natural language. Rather than relying on machine learning or templated extraction from structured comments, we propose to apply classic ideas from linguistics to translate natural-language sentences into executable tests. This paper explores the application of combinatory categorial grammars (CCGs) to generating property-based tests. Our prototype is able to generate tests from English descriptions for each example in a textbook chapter on property-based testing.},
}