Add a TypeScript declaration generator for a typed-function?

[gwhitney]

This is an idea that came up in the discussion for josdejong/mathjs#2448. Since in some sense, typed-function is a library that adds runtime type-checking and overloading to JavaScript that has a similar flavor to TypeScript's compile-time type-checking and overloading, it would be nice if there were a way to generate a TypeScript declaration for a typed-function that would compile (as close as possible to) exactly when the actual JavaScript call generated will not throw a TypeError. Because the type conversion and matching rules are fairly complex, I think this could only be done as a facility provided by this module, since it sees/knows everything that is going on with an individual function.

One potentially significant stumbling block is to get the sort of type inference that's actually of value to TypeScript users, the declaration should have some information about what the return types of functions are. So I think this would really only be doable in a useful way if typed-function added return-type annotations. So I will add an issue for that possibility as well.

[josdejong]

Thanks, that would be a good idea. It would be nicest if the types could be defined "on the fly" using generics, but I expect that to be too complex. Generating the type definitions statically would be a good alternative solution.

Anyone interested in doing an experiment with this?

[cshaa]

Here's a quick proof of concept!

[gwhitney]

Here's a quick proof of concept!

Well, that's pretty awesome. Thanks! I see that the type BaseTypes is defined at the top. I presume, though, that TypeScript types are not extensible? In other words, there isn't a way to update the BaseTypes type as someone calls typed.addType()? If we knew a static collection of all the types one ever wanted math.js, say, to deal with, then I would see no obstruction to simply rewriting it entirely in TypeScript from the ground up. It seems to me precisely that the ability to add types on the fly is what makes math.js/typed-function and TypeScript hard to reconcile. But I would be happy to be proved wrong. Until that happens, I despair of this automatically-from-the-TypeScript-compiler generation of .d.ts, and fear that a manual string-manipulating .d.ts generator from a typed-function instance is all we can hope for... But yes, I hope I am wrong.

[cshaa]

TypeScript does not allow types to be dynamically modified: if you, for example, define a type as string, you cannot retroactively redefine it to string | number elsewhere in the code. Such code would be impossible to statically analyze and much harder to reason about, and I don't think there is any type checker that would let you do this.

What you can do, however, is to have a typedFactory, to which you pass all type definitions and it gives you a typed function that correctly recognizes all of them. In the context of math.js, this would mean that each bundle would have its separate typed function which only recognizes the types present in such bundle.

You are right that “the ability to add types on the fly is what makes math.js/typed-function and TypeScript hard to reconcile”, because mutating the type of something is generally just a terrible idea and I'm not aware of any system that would let you do this. However, with typedFactory, no type is mutated – instead a new function with a new type is created.

Example usage of typedFactory here!

The part with the (x): x is something => x instanceof Something might be confusing for a non-TS user, so I'll quickly explain it. The TS syntax for an arrow function return type is (x): returnType => implementation(x). If you have a function which returns a boolean, you may declare that it's a so-called type guard using the syntax x is foo as the return type. If you do this, you tell TS that you guarantee the function will only return true if its argument is of type foo.

Therefore, the argument of typedFactory is an object which maps type names to their corresponding type guards. This is useful both for TS, as it can extract the necessary type-name pairs to use for static analysis, but also at runtime, as you can use these type guards to check what type something is.

[gwhitney]

Well that is excellent, in the sense that it looks really promising for writing a typescript declaration generator for mathjs more or less as it stands so we can avoid the drain of hand-maintenance of the index.d.ts. We could just run it twice, once for the main bundle and once for the number-only version. It still hasn't quite dawned on me yet how something like a "math.ts" would work, though, so pursuing a full rewrite of mathjs in TypeScript isn't really a priority for me. (I'm not in principle against such a rewrite, though, and am willing to help out if someone cleverer comes up with an architecture for it.)

[gwhitney]

(In particular based on your lovely proof of concept, my thinking is that we could write a little typescript program that loads in a math.js bundle holding its nose about its types, and then for each typed-function object, it extracts the map from string signatures to all of the (direct and converted) implementations of that typed-function, and passes that map to the typed-factory, producing a bona-fide correctly-typed TypeScript function but more importantly generates a .d.ts with the correct declarations which can then just be installed as the index.d.ts for the library, so that math.js can then be used from TypeScript without going through that "translation" process.)

Edit: or rather, upon some reflection, a javascript program that loads a mathjs instance, and quite trivially writes out a typescript program which defines the proper object assembling all the types and conversions, calls the typedFactory, and generates a proper index.d.ts for the mathjs instance. A bit more roundabout but I actually think this would work and cut through the treadmill of updating a manually-written index.d.ts. Seems well worth putting effort into.

[josdejong]

@m93a 😎 this POC in #123 (comment) looks really promising! I cannot think of a real show stopper right now. We will need to change the API though, and make it immutable again: have a typedFactory where you pass all types and conversions, instead of having methods addType and addConversion, maybe some difficulties will arise when working out this experiment.

so we can avoid the drain of hand-maintenance of the index.d.ts. We could just run it twice, once for the main bundle and once for the number-only version.

@gwhitney I'm not sure what you mean: I think this smart, generic TypeScript definition will replace the manual written out type definitions. I guess this solution will not have a spelled out type definition for each individual function anymore, that is derived by TypeScript based on the generic type definitions.

[gwhitney]

so we can avoid the drain of hand-maintenance of the index.d.ts. We could just run it twice, once for the main bundle and once for the number-only version.

@gwhitney I'm not sure what you mean: I think this smart, generic TypeScript definition will replace the manual written out type definitions. I guess this solution will not have a spelled out type definition for each individual function anymore, that is derived by TypeScript based on the generic type definitions.

Well, at the moment a typedFactory requires a fixed collection of types (in other words, all of the keys of the argument object to the typedFactory() function have to be known at compile time). So m93a's lovely proof of concept, as far as I can see, is not consistent with any version of typed-function in which you can add types at runtime. I think that's a feature that's definitely wanted. If so, then what you can do is in javascript, load all of mathjs with all the types you want (either the whole shebang as in the current main mathjs bundle, or just the number stuff as in the number bundle) and then have a utility program in javascript that writes out a little dummy TypeScript program that declares a static list all of the types actually used (and their conversions, which the proof of concept doesn't actually have yet) and then calls the resulting typedFactory for each of the mathjs operations. Then the build process compiles each of those dummy TypeScript programs created, producing .d.ts files which are then published with the actual javascript mathjs. Thereby, we do not have to undertake the daunting task of directly writing a typed-function -> TypeScript .d.ts translator, which seems like a lot more work than leveraging the TypeScript compiler.

Does this explanation clarify my comment?

[josdejong]

Hm, yes, I understand what you mean. Maybe we have to look a bit different to the problem: instead of dynamically importing new data types and functions into an existing mathjs instance (which is not possible to support by TypeScript if I understand it correctly), it may be possible to create a new mathjs instance from scratch with all the datatypes+functions that you want?

I've done some fiddling around with the second example of @m93a , trying to mold it into this model of the user importing the datatypes and functions that he/she wants (trying to grow it a bit towards the pocomath concept):

Dynamically construct an instance with data types and function signatures that you want

EDIT: updated the POC a bit, we're not there yet

[gwhitney]

Well, that is a lot more like Pocomath, that's true. Nice! Seems like a reasonably promising path. The main concern I see at the moment is that it seems to require (in much the same way as mathjs is organized now) that for each operator there is one place where every implementation is listed. One of the main ideas of Pocomath was trying to get away from that, as (pardon my saying so) it really breaks modularity and independence of different types.

The problem is this is getting beyond my experience/knowledge/comfort level with Typescript -- would there be a way to collect up the different implementations of 'add' from different imports, and then when one is done with all the imports, cook up the final "typed-function" version of 'add'? I don't really mind if you need to have a file for a given bundle that lists all of the types in that bundle -- since after all you have to import the modules for each type anyway, so they already need to be listed somewhere -- but having to reiterate all of the type-variants of each operator would be very verbose and redundant and would create otherwise unnecessary dependencies among different parts of the code. It seems like it would make custom bundling really prohibitively difficult, as you would have to track down for every single operator the exact list of implementations that you need/want.

Anyhow, I guess I will have to be mostly on the sidelines and cheer on the effort to find a TypeScript-y approach to typed-function.

[josdejong]

It's far from there, but I would like to see how far we can get. I'm not sure if it's possible to make it fully work with TypeScript. I'm not a Typescript expert either, but I'll do some more fiddling and see what I can learn 😁. @m93a more help would be welcome.

Anyhow, I guess I will have to be mostly on the sidelines and cheer on the effort to find a TypeScript-y approach to typed-function.

no problem at all👍