7.x typegen configuration (migration from 5.1.1)

[monitz87]

Hi. I'm trying to migrate some typescript projects to use version 7.4.1 (from 5.1.1). The types are generated without issue, but the compiler throws a ton of errors when attempting to build the projects:

• Module '"@polkadot/types/lookup"' has no exported member 'PalletImOnlineHeartbeat' is a good example. None of the stuff being imported from @polkadot/types/lookup can be found
• Cannot find name 'PolymeshPrimitivesDocument' where PolymeshPrimitivesDocument is (I assume) some sort of wrapper over our custom Document type. There is no indication that the augmented types are even trying to import the custom types from anywhere

Is there some special configuration required in order to make this work in 7.x?

[monitz87]

I managed to find a solution by looking at #4224 and #4213

If anyone else is running into this:

• Add an --endpoint flag to your generate:defs script (same as the one for generate:meta) pointing to the chain or a json file with the metadata
• Add "@polkadot/types/lookup": ["src/interfaces/types-lookup.ts"] to your tsconfig.json's paths

I noticed the new --endpoint flag is added to the example in this guide, but it's very easy to miss (especially if you are migrating from a previous version and it looks very similar to what you already have). The lookup path in the tsconfig isn't documented at all

[monitz87]

Question: considering the types are now being fetched from the chain, why does the generate:defs script require an --input?

[jacogr]

Happy you got it sorted, nice write-up above.

The input is still useful,

• RPC types are not defined in metadata, aka they still need manual definition (I believe Substrate has something logged to expose the types alongside the rpc_methods call, no ETA)
• for historic blocks, the type definitions are still useful
• the typegen itself doesn't require that you be on metadata v14, so it caters for old as well

I do see your point though, some teams may just supply an empty file here when they have no need for the above. If somebody would like to make it optional when --endpoint is specified and/or the metadata contains types, PRs welcome.

[monitz87]

I still noticed that if no input definitions are passed, the only types that are generated come in the form PalletPalletNameTypeName which might be a bit uncomfortable to work with. It might make sense as an improvement that the shorter aliases are also generated from the metadata if no manual types are passed. Wdyt?

[jacogr]

Yes, it is very Java-ish, actually on purpose since the "short names" actually got a lot of teams in trouble as soon as it starts overlapping between pallets. so it basically uses the full path as specified in the metadata.

As long as it remains the default and short-names are aliasses (assuming theye are no conflicts), I guess we could go crazy on aliasses. Example -

// actual path in the metadata (the type params are only added in _some_ case, e.g. when conflicts)
// polkadot_primitives::v1::signed::UncheckedSigned
export interface PolkadotPrimitivesV1SignedUncheckedSigned extends Struct {
// lots of stuff goes here
...
}

// all aliases, checked for conflicts

// v1::signed::UncheckedSigned
export interface V1SignedUncheckedSigned extends PolkadotPrimitivesV1SignedUncheckedSigned {}

// signed::UncheckedSigned
export interface SignedUncheckedSigned extends PolkadotPrimitivesV1SignedUncheckedSigned {}

// UncheckedSigned
export interface UncheckedSigned extends PolkadotPrimitivesV1SignedUncheckedSigned {}

If there are conflicts on a short path, then don't apply an alias. It can actually always replace what is passed as input - since all tx, query, events, ... work of the metadata types. (Only RPC is problematic, but in that case e.g SpCoreHeader or the equivalent should still map to Header)

EDIT: Here is a PR that would extract short aliases (to be used in the typegen, not included above) - #4488 (Would need to be careful since it is a horribly slow & nested operation, so for now it is there but may not go in anytime soon-ish)

[monitz87]

What about namespacing?

export namespace PolkadotPrimitives {
  export namespace V1 {
    export namespace Signed {
      export interface UncheckedSigner {
        // stufff
      }

      export interface TypeThatUsesTheOtherOne {
        unchecked_signer: UncheckedSigner;
      }
    }
  }
}

Or something like that. The definitions may look messier but usage should look a bit cleaner

[jacogr]

No namespaces.

[Tbaut]

Thanks for the write-up @monitz87

I do see your point though, some teams may just supply an empty file here when they have no need for the above.

The manual types are optional afaik, I just followed this here https://polkadot.js.org/docs/api/start/typescript.user#chain-modules and ran successfully:

yarn polkadot-types-from-chain --endpoint ws://localhost:9944 --output ./src/interfaces

Now as pointed out in the issue, I also have red squiggles form anything imported from @polkadot/types/lookup, and the hint to add "paths": {"@polkadot/types/lookup": ["./src/interfaces/types-lookup.ts"]} in tsconfig doesn't help in my case because I do not have such a types-lookup.ts. What am I missing?

[monitz87]

You need to add the --endpoint flag to the polkadot-types-from-defs script as well. Also, the discussion about manual types was related to that script

[Tbaut]

Oh thanks I see, I missed this part. I ended up doing the following:

• generate the output based on a node's metadata in ./src/interfaces
• run the type generation from the chain

yarn polkadot-types-from-chain --endpoint ws://localhost:9944 --output ./src/interfaces

• get the metadata

curl -H "Content-Type: application/json" -d '{"id":"1", "jsonrpc":"2.0", "method": "state_getMetadata", "params":[]}' http://localhost:9933 > ./custom-node-metadata.json

• create an empty type.ts in ./src/interfaces
• run the following to get the lookups (the endpoint pointing to my local node didn't work)

yarn polkadot-types-from-defs --endpoint ./custom-node-metadata.json --input ./src/interfaces

Now thanks to the paths in tsconfig, TS is happy.

[joelamouche]

Hi, I am having a similar problem when trying to migrate to 7..x
This is my PR: moonbeam-foundation/moonbeam#1219
I have the tsconfig setup but I have a similar error as @monitz87 :

  1) Uncaught error outside test suite:
     Uncaught TSError: ⨯ Unable to compile TypeScript:
../moonbeam-types-bundle/packages/api-augment/src/alphanet/interfaces/augment-api-consts.ts:316:7 - error TS2411: Property 'blockLength' of type 'FrameSystemLimitsBlockLength & AugmentedConst<ApiType>' is not assignable to 'string' index type 'Codec'.

316       blockLength: FrameSystemLimitsBlockLength & AugmentedConst<ApiType>;
          ~~~~~~~~~~~
../moonbeam-types-bundle/packages/api-augment/src/alphanet/interfaces/augment-api-consts.ts:320:7 - error TS2411: Property 'blockWeights' of type 'FrameSystemLimitsBlockWeights & AugmentedConst<ApiType>' is not assignable to 'string' index type 'Codec'.

320       blockWeights: FrameSystemLimitsBlockWeights & AugmentedConst<ApiType>;
          ~~~~~~~~~~~~
../moonbeam-types-bundle/packages/api-augment/src/alphanet/interfaces/augment-api-consts.ts:324:7 - error TS2411: Property 'dbWeight' of type 'FrameSupportWeightsRuntimeDbWeight & AugmentedConst<ApiType>' is not assignable to 'string' index type 'Codec'.

324       dbWeight: FrameSupportWeightsRuntimeDbWeight & AugmentedConst<ApiType>;
          ~~~~~~~~
../moonbeam-types-bundle/packages/api-augment/src/alphanet/interfaces/augment-api-consts.ts:336:7 - error TS2411: Property 'version' of type 'SpVersionRuntimeVersion & AugmentedConst<ApiType>' is not assignable to 'string' index type 'Codec'.

336       version: SpVersionRuntimeVersion & AugmentedConst<ApiType>;
          ~~~~~~~
../moonbeam-types-bundle/packages/api-augment/src/alphanet/interfaces/augment-api-consts.ts:387:24 - error TS2344: Type 'FrameSupportWeightsWeightToFeeCoefficient' does not satisfy the constraint 'Codec'.
  The types of 'hash.registry' are incompatible between these types.
    Type 'Registry' is missing the following properties from type 'Registry': createClassUnsafe, getUnsafe

387       weightToFee: Vec<FrameSupportWeightsWeightToFeeCoefficient> & AugmentedConst<ApiType>;
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../moonbeam-types-bundle/packages/api-augment/src/alphanet/interfaces/augment-api-consts.ts:405:7 - error TS2411: Property 'palletId' of type 'FrameSupportPalletId & AugmentedConst<ApiType>' is not assignable to 'string' index type 'Codec'.

405       palletId: FrameSupportPalletId & AugmentedConst<ApiType>;
          ~~~~~~~~
../moonbeam-types-bundle/packages/api-augment/src/alphanet/interfaces/augment-api-consts.ts:444:7 - error TS2411: Property 'selfLocation' of type 'XcmV1MultiLocation & AugmentedConst<ApiType>' is not assignable to 'string' index type 'Codec'.

444       selfLocation: XcmV1MultiLocation & AugmentedConst<ApiType>;
          ~~~~~~~~~~~~
../moonbeam-types-bundle/packages/api-augment/src/alphanet/interfaces/augment-api-consts.ts:461:7 - error TS2411: Property 'selfLocation' of type 'XcmV1MultiLocation & AugmentedConst<ApiType>' is not assignable to 'string' index type 'Codec'.

461       selfLocation: XcmV1MultiLocation & AugmentedConst<ApiType>;