feat(gatsby): Allow awaiting API run triggered by createNode action

[stefanprobst]

When the createNode action is called, we currently have no way of knowing when subsequent onCreateNode API calls have finished. This is e.g. a problem in custom field resolvers, where we cannot safely call createRemoteFileNode.

This PR makes createNode a thunk which still synchronously dispatches the actions, but returns a Promise of the triggered API run that can optionally be awaited.

[wardpeet]

In my opinion, this looks great!

I just have a question if you could elaborate on

When the createNode action is called, we currently have no way of knowing when subsequent onCreateNode API calls have finished. This is e.g. a problem in custom field resolvers, where we cannot safely call createRemoteFileNode.

why can't we safely call createRemoteFileNode?

[stefanprobst]

why can't we safely call createRemoteFileNode?

Currently the createNode action is re-emitted as an event, which is caught in plugin-runner, which calls api-runner-node. The event-emitter sort of cuts off the feedback to the callsite.

The problem is that when we call createRemoteFileNode in a resolver, and want to e.g. query a childImageSharp field on that file node. When createRemoteFileNode returns we can be sure that the File node has been created (all is synchronous), but we need to await the subsequent onCreateNode API call as well that creates the ImageSharp child node. Otherwise, the childImageSharp field will be null.

[stefanprobst]

Anything missing from this?

[DSchau]

Thinking this looks good--let's get this merged today?

[DSchau]

@pieh want to take a look at this soon?

[pieh]

@pieh want to take a look at this soon?

Yes, on it.

[pieh]

I just verified using this simple snippet:

---
title: Hello World
date: "2015-05-01T22:12:03.284Z"
remoteURL: "https://images.unsplash.com/photo-1556379092-dca659792591"
---

exports.sourceNodes = ({ actions, schema, store, cache, createNodeId }) => {
  actions.createTypes(`
  type MarkdownRemark implements Node {
    frontmatter: Frontmatter
  }
  `)
  actions.createTypes(
    schema.buildObjectType({
      name: "Frontmatter",
      fields: {
        remoteURL: {
          type: `File`,
          resolve: async (source, args, context, info) => {
            const fieldValue = source[info.fieldName]
            console.log({ fieldValue })

            const fileNode = await createRemoteFileNode({
              url: fieldValue,
              store,
              cache,
              createNode: actions.createNode,
              createNodeId,
              ext: `.jpg`,
            })

            return fileNode
          },
        },
      },
    })
  )
}

that this indeed works as expected. We will need to look at bumping/adjusting query running concurrency later on (because using above will limit downloading files to be just from 4 queries, and this can be bottleneck). Will run few more tests (against our .org and few other sites to verify there isn't any unintentional breakage somewhere, but code LGTM

[pieh]

Also additional note - (not a blocker) - right now, fact that createNode will return Promise is not documented. I'm not 100% sure where it should be documented - at least to action signature in API reference I think can be adjusted to note that Promise is returned, and when we have actually docs that cover using await SomethingThatUsesCreateNode in resolvers (so probably some advanced schema docs), then we could add cross linking

[stefanprobst]

We will need to look at bumping/adjusting query running concurrency later on

yes good point!

[stefanprobst]

fact that createNode will return Promise is not documented

Added short note about the returned Promise

[pieh]

LGTM