Filling all fields

[eunjae-lee]

When some fields are empty on a certain row, the api result from airtable doesn't include the field in that row.

{
  "records": [
    {
      "id": "recR4fJQsMlpWyy5A",
      "fields": {
        "Name": "John"
      },
      "createdTime": "2018-12-30T07:10:11.000Z"
    },
    {
      "id": "recoqit7jOBr8m6JK",
      "fields": {
        "Name": "Michael",
        "Notes": "Note1"
      },
      "createdTime": "2018-12-30T07:10:11.000Z"
    }
  ]
}

You can see "Notes" field is missing at the first row. Could we have a fields option?

    {
      resolve: 'gatsby-source-airtable',
      options: {
        apiKey: 'xxx',
        tables: [
          {
            baseId: 'xxx',
            tableName: 'xxx',
            tableView: 'xxx',
            fields: ['Name', 'Notes']
          },
        ],
      },
    }

When fields is specified, the result will always include all the fields with null as default value if none.
This is to prevent graphql query from failing. Refer to gatsbyjs/gatsby#2392

What do you think?

[jbolda]

👋

So I believe there are two layers of issues here. First it is Airtable not delivering all of the expected fields (which when Gatsby infers your data structure causes errors). Your proposition is to fix this. Secondly we have the ongoing issue with Gatsby inferring your data structure in a manner you don't expect/want. Couple other (I believe) relevant issues below:

• [Discussion] Feature: GraphQL schema snapshots for all data sources to solve undefined/empty data issues gatsbyjs/gatsby#3344
• Extending third-party schemas via directives gatsbyjs/rfcs#11
• [EPIC] Graphql schema refactor gatsbyjs/gatsby#4261

Obviously we aren't going to fix the second consideration here, but it's looking like the solution might be similar to your thoughts for the first consideration. I'm thinking we may not want to try to implement a list of fields as that might step on the toes Gatsby core's eventual solution. However, maybe we can adjust our solution slightly to something that would always be useful for this plugin. What if we instead of fields have an optional object defaultValues that just gets merged with each Airtable row prior to mapping and node creation? You would be able to literally add fields that don't exist in Airtable this way, and we don't need to try to decide a sensible default / null for the user. (I believe, but don't quote me, that the graphql infer skips/ignores nulls anyways.) What do you think of this approach? Usable in the long run, and fixes your issue in the short term.

[eunjae-lee]

Firstly, it's Airtable's decision not to fill all the fields when their values are null and it's pretty common situation. So we cannot do anything about it.
Secondly, as soon(?) as gatsby fix those issue, this fix will be useless.

So as you suggest, defaultValues sounds a good way to cover this problem and also provides something useful to other users as well. And actually that's a better fix than my previous suggestion because all fields might have different default values.

[jbolda]

Firstly, it's Airtable's decision not to fill all the fields when their values are null and it's pretty common situation. So we cannot do anything about it.

Agreed as well. (I didn't make that clear in my previous comment.)

Sounds perfect to me. Would you be available to work on a PR for this?

[eunjae-lee]

Sure. I'll try to work on it in a couple of days. Ah, by the way, happy new year 🎉

[jbolda]

Great! Thanks again for the help, and happy new year to you as well!