Running tests with a bundler should be easier

[connor4312]

Currently our standard test setup is using mocha and globbing for files. This doesn't work well with bundlers like esbuild or webpack. There are hacks to allow star imports in both these bundlers, but ideally we could figure some less-intrusive way to deal with these.

Webpack removes the need to glob for files, but a bundled file that requires test env CLI cannot be used with this package:

• This package uses runTests  to execute the code CLI with special arguments and expose require('vscode').
• Jest Mocha and all their buddies do a similar thing.
• Trying to do both at the same time results in Jest/Mocha/etc exiting with 1 (no tests found) at the end of the runTests.ts file.

Ideally to support the test CLI (Jest Mocha etc) the core repo needs to add CLI argument for an absolute bin path to wrap the harness JS.

[connor4312]

Yea, webpack would, though we would dislike 'hardcoding' to a single bundler. And I myself prefer esbuild for my extensions 🙂 There are solutions there too, of course, but the question is whether we want to maintain a list of instructions/plugins needed for each bundler (which is sure to change over time)

Fwiw there is now bleeding edge BDD test closures shipped with node - Test runner (NodeJS).
It's not on the same version that VS Code ships with unfortunately, but yeah.

[remcohaszing]

I’m trying to understand this problem. Is this correct?

There are integration tests. These tests run test actual behaviour based on the generated bundle that ships with the VSCode extension. These do not require access to the source code. These integration tests are often written in TypeScript, but compiled and run as JavaScript. There’s no need to bundle tests for this.

There are unittests. These test the source code on a lower level. Both the unittests and source code are often written in TypeScript, but compiled and run as JavaScript.

This means tests need a compilation step which is different from the build step.

So the real problem is extension tests can’t run TypeScript code, right?

---

I imagine the ideal situation would be that the user just needs to write a standard test file, run code-test, and everything just works.

I think this is possible by:

• Providing a CLI (feat: add CLI #161)
• Using node:test (still experimental) and removing support for third party test runners. Use run to support TypeScript extensions.
• Compiling TypeScript on the fly like ts-node or tsx.

I have a working test suite that scales to Node and Web in a platform-independent manner:

https://github.com/vscode-toolroom/vscode-textmate-languageservice/tree/11223728567c20d66e0966b96290f9e3328f9a4d

• webpack library bundling - webpack.config.js
• separate tsconfig for node test compilation tsconfig.test.json
• also webpack web test bundling - webpack.config.web.test.js
• harness (runTests pattern) and runner (suite index) for node+web - see ./test
• Mocha and VS Code tests in any way you wish! ./test/suite