REPL test: copy-paste REPL interactions to define tests
Source code: https://github.com/jsmaniac/repltest
#lang repltest racket (define x 3) > (+ x 1) 4
The first part of the file is kept inside the top-level module. This module uses the language indicated just after #lang repltest, for example:
After the first occurrence of the prompt (by default "> ", later versions of this package will allow customizing this) is encountered, all the remaining contents of the file are understood as a REPL transcript. The prompt is only recognized if it is outside of any s-expression, which means that the > function can be used normally.
#lang repltest racket (define x (> 3 4)) > x #f
1 The test submodule
This language injects a test submodule using module*. When the test module is run, the expression after each prompt is read and evaluated as if it had been typed inside a real REPL, using read-eval-print-loop. The result, as printed on the standard output and standard error, is compared with the text read until the next prompt. The next prompt will only be recognized if it is not part of an s-expression, which means that occurrences of > inside an expression in the output are correctly handled:
#lang repltest racket (define x '(> 3 4)) > x ’(> 3 4) > '(> 5 6) ’(> 5 6)
The fact that a real REPL is used means that any language-specific output will be produced as expected. For example typed/racket prints the type of the result before the result itself, so it must be included in the expected output:
#lang repltest typed/racket (define x 0) > x - : Integer [more precisely: Zero] 0
2 Warning concerning comments
Comments are not currently supported inside the REPL transcript. Also, the current version does not recognise the first prompt if it is preceded by a comment.
3 Warning concerning spaces and newlines
The tests are space-sensitive, so care should be taken to include a newline at the end of the file. This is due to the fact that in most languages, the REPL prints a newline after the result. Furthermore, extra spacing like blank lines should not be added in the transcript part of the file.
4 Future improvements
Later versions of this package will allow customizing the following aspects:
Flexibility of whitespace comparisons (strip leading and trailing whitespace, or ignore all whitespace differences).
Support comments before and inside the REPL transcript.
Specifying a regexp matching the prompt, and a regexp for characters preceding the prompt which are not part of it (and therefore will be part of the preceding result or main module’s code).
Disable calling read on the output expressions, which can be useful when the output contains unbalanced parenthesis, or do not otherwise match the language’s syntax, for example:
#lang repltest racket > (displayln "(unbalanced") (unbalanced > (displayln "#invalid (syntax . too . many . dots)") #invalid (syntax . too . many . dots) This will also have the side-effect of allowing the prompt to be matched inside s-expressions.
Distinguish standard output (purple font in DrRacket), printed result (blue font) and standard error (red font).