doc(BUD-2,test,cli): topiary test subcommand#10
Conversation
github-project-automation
Bot
moved this to Draft
in Blueprints for upcoming development
| ## Unresolved questions | ||
|
|
||
| * Do we want to apply a specific naming schema for our tests? | ||
| * How do we group multiple tests? |
There was a problem hiding this comment.
Namespaces, perhaps:
foo::bar::baz
Co-authored-by: Christopher Harrison <Xophmeister@users.noreply.github.com>
mkatychev
changed the title
topiary test subcommand
Xophmeister
left a comment
Xophmeister
left a comment
There was a problem hiding this comment.
Looks good 👍 I've added a few more suggestions to flesh it out a bit.
|
|
||
| ## Motivation | ||
|
|
||
| * There's value to having test cases be in one huge file for side effects. |
There was a problem hiding this comment.
Doesn't delimiting the tests is one huge file ostensibly remove side-effects (unless the test was specifically structured to have side-effects)?
| ## Motivation | ||
|
|
||
| * There's value to having test cases be in one huge file for side effects. | ||
| * One should not have to recompile the `topiary` binary to test query changes for built-in languages |
There was a problem hiding this comment.
This is already true, isn't it? You can just give Topiary the path to a query file to test any query changes.
|
|
||
| * There's value to having test cases be in one huge file for side effects. | ||
| * One should not have to recompile the `topiary` binary to test query changes for built-in languages | ||
| * external grammar repositories such as [topiary-nushell](https://github.com/blindFS/topiary-nushell) should have comparable DX to what is happening in topiary |
There was a problem hiding this comment.
I think this point is probably the strongest, but it needs expanding on to understand the shortfalls and what/how topiary-nushell does to workaround them.
|
|
||
| ## Alternatives considered | ||
|
|
||
| Instead of comments, one could use [`query_name!`](https://topiary.tweag.io/book/reference/capture-names/general.html#query_name) but DX may be different. |
There was a problem hiding this comment.
I think there are plenty of times when multiple queries need to be used to adequately test something, so query_name! becomes too limiting.
| `topiary test` is a new subcommand and adds no changes to existing | ||
| Topiary behaviour, so there are no backwards-compatibility concerns. |
There was a problem hiding this comment.
This isn't a backwards compatibility concern, but I could see this also superseding the current input/expected tests. It seems like a more disciplined approach, rather than dumping some random source file into the input and then shoehorning regression tests into it 👍
|
|
||
| Test report could look like this in stdout: | ||
| ```sh-session | ||
| $ topiary test --dir ./my_lang/test --query ./my_lang/query.scm |
There was a problem hiding this comment.
This isn't stated, but I suspect you imply this with your final unresolved questions, but you'd presumably expect something like:
topiary test [OPTIONS...] [TESTS...]
Where TESTS... is an optional list of tests, either fully qualified or partially by namespace, which the harness will then run explicitly.
I think this is worth writing down because I think that's pretty nice DX that we don't have right now.
2 participants
Topiary BUD-2
Checklist
I have discussed this BUD proposal at:
Implement
topiary testtopiary#1142 (issue preceded BUDs)I have followed the guidance for new BUDs
Description
This proposal proposes introducing a new
topiary testsubcommand for formatting query validation.