Feature request: Support using `Pure text` as test data in testing

[akphi]

Similar requests

• I have searched and found no existing similar requests

How are you using Studio?

No response

What problems are you trying to solve?

It's a nice capability to have engine server to test against or even do very simple stuffs like translating test case in Pure to protocol JSON format to keep test data more succinct. Also it's nicer to be able to use this strategy for each package rather than collate all of them into legend-manual-tests, so we would like to run the engine server in the background always (this takes just a little bit of time in the pipeline hopefully), and automatically give all tests the ability to use test data in Pure grammar.

Describe the solution you would like to see implemented

• Setup manual tests to use snapshot Docker image for legned-engine
• Move the test cases in legend-manual-test to the respective packages
• Delete the package legend-manual-test
• Move test step from node-ci.yml to a separate test.yml workflow in Github action workflow and add the engine Docker step as the first step before running tests
• CONSIDER stopping Docker afterwards
• Create a Jest utility function pure_grammar_test = process.env.DISABLE_PURE_GRAMMAR_TEST ? test.skip : test;
• Create function to load grammar file and parse them to JSON/entities that checks for the flag and do a try/catch to throw some meaningful error to the developers

Describe alternatives you have considered

We considered the alternative approach to separate these grammar test out to a different set of tests, e.g. something.server-integration-test.js, where we can use projects config to run the two set of tests independently. In terms of setup, this complicates the way we compute coverage, and also the way developer run these tests (we will need separate scripts for running tests, e.g. test, test:server-integration, test:watch, test:server-integration:watch).

Also personally, I feel that calling these tests server-integration is a bit misleading for 2 reasons:

1. It implies that we are depending on the backend(s) for some operations, that's the job of e2e tests, while these tests are strictly unit/integration tests for studio
2. The use of backend server in this case is only to equip Studio with some capabilities that it does no have, such as to convert Pure grammar text to protocol JSON, nothing more than that, imagine in the future where we can port this functionality to Studio/JS, the intention behind these tests won't change anyhow, just their method of acquiring the test data will be.

Documentation, Design, Adoption, Migration Strategy

No response

Contribution

• I would like to work on this feature