Add lint rule to validate JSDoc codeblocks using TS compiler

[som-sm]

This PR introduces a lint rule that extracts example code blocks from JSDoc comments and validates them using @typescript/vfs. It then reports any errors at their corresponding locations.

The errors and their locations are identical to normal TS errors, so the experience feels just like editing a regular TS file.

Screen.Recording.2025-10-10.at.6.10.33.PM.mov

---

Working

Linting a single file involves the following steps:

1. Setting up the environment
   A virtual TypeScript environment with a single virtual file is created on top of the existing file system. And the virtual environment is created such that it has access to the files on disk.

   Location of virtual files
   While setting up the virtual environment, we specify the root directory. The virtual environment then creates a virtual directory called vfs within the root directory, and all virtual files are created relative to this vfs directory.

   For example, the following snippet:

   // type-fest/lint-rules/validate-jsdoc-codeblocks.js
   import ts from "typescript";
   import {createFSBackedSystem, createVirtualTypeScriptEnvironment} from "@typescript/vfs";
   
   const FILENAME = 'example-codeblock.ts';
   
   const compilerOptions = {...};
   
   const virtualFsMap = new Map();
   virtualFsMap.set(FILENAME, 'Some code here...');
   
   const rootDir = path.join(import.meta.dirname, '..'); // We go back one level from `lint-rules` dir
   const system = createFSBackedSystem(virtualFsMap, rootDir, ts);
   const env = createVirtualTypeScriptEnvironment(system, [FILENAME], ts, compilerOptions);

   creates the following directory structure:

   type-fest/
   ├── package.json
   ├── source/
   │   ├── all-extend.d.ts
   │   ├── ...
   │   └── xor.d.ts
   └── vfs/ (virtual)
       └── example-codeblock.ts  (virtual file)
   

   So, if we want to import the Or type from our example-codeblock.ts virtual file, we'd have to use '../source/or.d.ts' as the import path, like:

   import type {Or} from '../source/or.d.ts';
   
   const tt: Or<true, true> = true;
   const tf: Or<true, false> = true;
   const ft: Or<false, true> = true;
   const ff: Or<false, false> = false;

   The location of the virtual file might seem problematic, but in practice it’s not, because our JSDoc example codeblocks never use relative imports. Instead, they always import from 'type-fest':

   import type {Or} from 'type-fest';
   
   const tt: Or<true, true> = true;
   const tf: Or<true, false> = true;
   const ft: Or<false, true> = true;
   const ff: Or<false, false> = false;

   Note: Importing directly from 'type-fest' works fine because Node.js allows self referencing a package using its name.

2. Running the lint rule on each exported type node

   • The rule extracts the JSDoc comment corresponding to that node and then extracts the codeblocks within the JSDoc.
   • It iterates over the codeblocks, replaces the contents of the virtual file with the codeblock, and runs TypeScript diagnostics on the virtual file.
   • Finally, it collects any diagnostic errors and reports them back at their exact locations.

---

The lint rule surfaced around ~250 errors (on publicly accessible docs), so this PR also fixes/improves multiple JSDoc code examples. Some common errors that it catches:

• Missing import statements
  

• Floating examples
  

• References to hypothetical functions/variables
  

• Duplicate identifiers
  

• Syntax errors
  

• Typos
  
  

[som-sm]

@sindresorhus This PR is still a WIP, because there are still a lot of code blocks that have errors in them, but would appreciate if you could give an early feedback on this.

Couple of questions:

• At several places we have code examples for internal types, like:
  

  type-fest/source/get.d.ts

  Lines 82 to 93 in 72f491f

  	/**
  	Returns true if `LongString` is made up out of `Substring` repeated 0 or more times.
  	@example
  	```
  	ConsistsOnlyOf<'aaa', 'a'> //=> true
  	ConsistsOnlyOf<'ababab', 'ab'> //=> true
  	ConsistsOnlyOf<'aBa', 'a'> //=> false
  	ConsistsOnlyOf<'', 'a'> //=> true
  	```
  	*/
  	type ConsistsOnlyOf<LongString extends string, Substring extends string> =

  
  Now, in such cases, we can't have an import statement because internal types are not publicly available.

  So, we can either make this rule work only for exported types, or inline the examples like I've done in this change:
  

  Or, maybe we can do something even better, wdyt?

• At times we highlight errors in our codeblocks, but with this lint rule in place, we can't have errors in codeblocks, so I guess it's fine to add an @ts-expect-error directive in such cases. This would silence the error and also clearly state the intent that an error is indeed expected. Refer this change:

  

• I think this rule makes sense and is useful, but unfortunately it takes around min and a half to run, which might be a problem. Although, the in editor DX is pretty fast, it's just when it runs on all files together.

  Screen.Recording.2025-10-10.at.6.10.33.PM.mov
  
  

  Performance Consideration: Line 68 creates a new virtual TypeScript environment for each code block. For files with many examples, this could be slow. Consider reusing the environment or batching:

  // Current approach creates new env per codeblock
  const env = createVirtualTypeScriptEnvironment(system, [FILENAME], ts, compilerOptions);

  Suggestion: Create one environment per file and update it with new content, rather than recreating for each block.

  This suggestion isn't possible because you can't update the virtual file once the virtual environment is created.

  I'm looking into ways for improving this, but let me know if you have some thoughts around this.

[som-sm]

• but unfortunately it takes around min and a half to run, which might be a problem.

Should be fairly fast now, the env creation happens only once now!

Now, the only task left is to fix the remaining 214 errors 😪

[som-sm]

the env creation happens only once now!

No, it can't run just once.

• but unfortunately it takes around min and a half to run, which might be a problem.

The env creation now happens per file instead of the initial per file per code block setup. So, it now takes around a min. Refer #d974efa.

[sindresorhus]

This is a very useful rule 🙌

[sindresorhus]

Now, in such cases, we can't have an import statement because internal types are not publicly available.

I think we should just ignore the internal types. Many will eventually be exposed and then they can be fixed then.

At times we highlight errors in our codeblocks, but with this lint rule in place, we can't have errors in codeblocks, so I guess it's fine to add an @ts-expect-error directive in such cases. This would silence the error and also clearly state the intent that an error is indeed expected. Refer this change:

👍

I think this rule makes sense and is useful, but unfortunately it takes around min and a half to run, which might be a problem. Although, the in editor DX is pretty fast, it's just when it runs on all files together.

Doesn't ESLint cache things? Maybe something is not working with the cache.

Alternatively, we could only run this rule in CI.

[som-sm]

I've added comments wherever I felt it needed some explanation. Hope it makes the review process a tad bit easier.

[som-sm]

Changed this example because these three points have nothing to do with the type.

[som-sm]

Removed this example because the primary focus here wasn't on IsNever. Added a better example (see below).

[som-sm]

Replaced this example because it had multiple issues, but the updated example conveys a similar idea as the original example.

[som-sm]

This example is almost similar to the previous one for this type, so simply removed it instead of replacing it with an alternative.

[som-sm]

The existing example had issues, and a simpler example seemed better in this case.

[som-sm]

There's a zero-width space character b/w * and / to prevent */ from being interpreted as the end of the JSDoc comment. Because of which there's a /* eslint-disable no-irregular-whitespace */ comment at the start of this file.

Umm...not sure if there's a better way of achieving this, I tried a couple different solutions but nothing seemed to work.

[som-sm]

Because of which there's a /* eslint-disable no-irregular-whitespace */ comment at the start of this file.

Now, the rule is disabled only for comments.

type-fest/xo.config.js

Lines 88 to 98 in 657dd4d

	{
	files: 'lint-rules/test-utils.js',
	rules: {
	'no-irregular-whitespace': [
	'error',
	{
	'skipComments': true,
	},
	],
	},
	},

[som-sm]

These utils help make the test cases terse, otherwise this already lengthy file would have been even lengthier.

For example, the following

exportTypeAndOption(jsdoc(fence(code1)), jsdoc(fence(code2)))

evaluates to:

/**
```
type A = string;
```
*/
export type T0 = string;

/**
```
type B = number;
```
*/
export type T1 = string;

export type TOptions = {
        /**
        ```
        type A = string;
        ```
        */
        p0: string;

        /**
        ```
        type B = number;
        ```
        */
        p1: string;
};

[som-sm]

For invalid tests, the code is not generated via some utility, as it is for valid cases, because here we want to validate the exact line and column numbers where the errors appear, and if the code were obfuscated behind some utility, then those numbers would feel magical/arbitrary, reducing the overall readability and maintainability of the test cases.

[som-sm]

For errors, we use a utility because manually typing the column and endColumn values is error-prone and also less readable.

For example, in the following test

{
	code: dedenter`
		/**
		@example
		\`\`\`ts
		type NoImport = Add<1, 2>;
		\`\`\`
		*/
		export type Add = number;
	`,
	errors: [
		{messageId: 'invalidCodeblock', line: 4, column: 17, endLine: 4, endColumn: 20},
	],
},

the numbers 17 and 20 feel magical/arbitrary and it's very easy to mess up those numbers and end up with false-positive tests.

It's much more readable if the column and endColumn values are generated like this:

errors: [
	{
		messageId: 'invalidCodeblock',
		line: 4,
		column: 'type NoImport = '.length + 1,
		endLine: 4,
		endColumn: 'type NoImport = '.length + 1 + 'Add'.length,
	},
],

But, this is still repetitive and has scope for simplification.

And, that's what the errorAt utility simplifies:

errors: [
	errorAt({
		line: 4,
		textBeforeStart: 'type NoImport = ',
		target: 'Add',
	}),
],

Note: The line and endLine values are manually entered because those are pretty straightforward. And errorAt uses the line value if an endLine is not explicitly passed, which also reduces the repetition a bit.

[som-sm]

The existing example had several issues, and it wasn’t really instantiating IsUnknown with unknown. Replaced it with a simpler example.

[som-sm]

@sindresorhus This PR is finally ready for review. It took some effort to get it to the finish line, but I’m happy with how it turned out. I’ve also updated the PR description to include a detailed explanation of how this lint rule works.

The only changes within the source directory are inside JSDoc example code blocks. I’ve fixed those examples using the following approach:

1. If an example was easily fixable (e.g., missing import, typo, or floating snippet), I simply made the fix without checking the logic or relevance of the example.

2. If an example contained errors that weren’t easily fixable, I improved it from the ground up. I’ve added comments wherever I made such changes.

[som-sm]

• At times we highlight errors in our codeblocks, but with this lint rule in place, we can't have errors in codeblocks, so I guess it's fine to add an @ts-expect-error directive in such cases. This would silence the error and also clearly state the intent that an error is indeed expected. Refer this change:

NOTE: Using the @ symbol (@ts-expect-error) inside JSDoc codeblocks breaks rendering. There's already an open issue for the same.

[sindresorhus]

This is super nice! 🎉 And seeing all the changes to code examples, it was really needed. It's hard to manually ensure code examples are correct.