gatsby-plugin-sitemap: TypeError: Cannot use 'in' operator to search for 'nodes' in undefined

[janosh]

Description

plugin-sitemap crashes build:

⠹ onPostBuild

 ERROR #11321  PLUGIN

"gatsby-plugin-sitemap" threw an error while running the onPostBuild lifecycle:

Cannot use 'in' operator to search for 'nodes' in undefined

  40 |           case 9:
  41 |             queryRecords = _context.sent;
> 42 |             filteredRecords = (0, _internals.filterQuery)(queryRecords, excludeOptions, basePath, resolveSiteUrl);
     |                                                          ^
  43 |             urls = serialize(filteredRecords);
  44 | 
  45 |             if (!(!rest.sitemapSize || urls.length <= rest.sitemapSize)) {

File: node_modules/gatsby-plugin-sitemap/gatsby-node.js:42:58



  TypeError: Cannot use 'in' operator to search for 'nodes' in undefined
  
  - internals.js:113 getNodes
    [blog]/[gatsby-plugin-sitemap]/internals.js:113:15
  
  - internals.js:46 filterQuery
    [blog]/[gatsby-plugin-sitemap]/internals.js:46:19
  
  - gatsby-node.js:42 _callee$
    [blog]/[gatsby-plugin-sitemap]/gatsby-node.js:42:58
  
  - task_queues.js:97 processTicksAndRejections
    internal/process/task_queues.js:97:5

Steps to reproduce

My config (see in repo)

{
  resolve: `gatsby-plugin-sitemap`,
  options: {
    output: `/sitemap.xml`,
    query: `{
      site {
        meta: siteMetadata {
          url
        }
      }
      pages: allSitePage {
        nodes {
          path
        }
      }
    }`,
    resolveSiteUrl: ({ site }) => site.meta.url,
    serialize: ({ site, pages }) =>
      pages.nodes.map(node => ({ url: site.meta.url + node.path })),
  }
  ,
},

Environment

  System:
    OS: macOS 10.15.5
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 14.4.0 - /usr/local/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.14.4 - /usr/local/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 83.0.4103.97
    Firefox: 77.0.1
    Safari: 13.1.1
  npmPackages:
    gatsby: 2.23.1 => 2.23.1 
    gatsby-image: 2.4.6 => 2.4.6 
    gatsby-plugin-algolia: ^0.11.1 => 0.11.1 
    gatsby-plugin-catch-links: 2.3.4 => 2.3.4 
    gatsby-plugin-favicon: ^3.1.6 => 3.1.6 
    gatsby-plugin-feed: ^2.5.4 => 2.5.4 
    gatsby-plugin-google-analytics: 2.3.3 => 2.3.3 
    gatsby-plugin-lodash: ^3.3.3 => 3.3.3 
    gatsby-plugin-manifest: 2.4.10 => 2.4.10 
    gatsby-plugin-mdx: ^1.2.14 => 1.2.14 
    gatsby-plugin-netlify-cache: ^1.2.0 => 1.2.0 
    gatsby-plugin-offline: 3.2.8 => 3.2.8 
    gatsby-plugin-react-helmet: 3.3.3 => 3.3.3 
    gatsby-plugin-sharp: 2.6.10 => 2.6.10 
    gatsby-plugin-sitemap: ^2.4.5 => 2.4.5 
    gatsby-plugin-styled-components: ^3.3.3 => 3.3.3 
    gatsby-remark-autolink-headers: ^2.3.4 => 2.3.4 
    gatsby-remark-code-titles: ^1.1.0 => 1.1.0 
    gatsby-remark-copy-linked-files: ^2.3.4 => 2.3.4 
    gatsby-remark-embed-video: 3.0.10 => 3.0.10 
    gatsby-remark-emojis: ^0.4.3 => 0.4.3 
    gatsby-remark-images: 3.3.9 => 3.3.9 
    gatsby-remark-katex: ^3.3.3 => 3.3.3 
    gatsby-remark-responsive-iframe: 2.4.4 => 2.4.4 
    gatsby-remark-smartypants: 2.3.3 => 2.3.3 
    gatsby-remark-sub-sup: ^1.0.0 => 1.0.0 
    gatsby-remark-vscode: ^2.1.1 => 2.1.1 
    gatsby-source-filesystem: 2.3.10 => 2.3.10 
    gatsby-transformer-sharp: 2.5.4 => 2.5.4 
    gatsby-transformer-yaml: 2.4.3 => 2.4.3 

[vladar]

Thank you for opening this, @janosh

I think it happens because you've aliased allSitePage in your query. gatsby-plugin-sitemap relies on this key to build the site map:

gatsby/packages/gatsby-plugin-sitemap/src/internals.js

Lines 23 to 25 in 670f92f

	const { allSitePage, ...otherData } = data
	let { allPages, originalType } = getNodes(allSitePage)

And it is actually mentioned in the plugin docs:

Due to how this plugin was built it is currently expected/required to fetch the page paths from allSitePage, but you may use the allSitePage.edges.node or allSitePage.nodes query structure.

So the quick fix for you is to change pages: allSitePage back to allSitePage.

We're marking this issue as answered and closing it for now but please feel free to reopen this and comment if you would like to continue this discussion. We hope we managed to help and thank you for using Gatsby! 💜

[janosh]

@vladar Good catch! Should have noticed that.

But now I'm getting

ERROR #11321  PLUGIN

"gatsby-plugin-sitemap" threw an error while running the onPostBuild lifecycle:

Error: Protocol is required

with the following config

options: {
  output: `/sitemap.xml`,
  query: `{
    site {
      siteMetadata {
        url
      }
    }
    allSitePage {
      nodes {
        path
      }
    }
  }`,
  resolveSiteUrl: ({ site }) => site.siteMetadata.url,
  serialize: ({ site, allSitePage }) =>
    allSitePage.nodes.map(node => ({ url: site.siteMetadata.url + node.path })),
}

Both error messages weren't that helpful so far.

[vladar]

Seems similar to #12912

[vladar]

Oh well, this seems weird for me but this plugin does this:

gatsby/packages/gatsby-plugin-sitemap/src/internals.js

Lines 57 to 69 in 6a9ffcc

	return {
	...otherData,
	allSitePage: {
	[originalType]:
	originalType === `nodes`
	? allPages
	: allPages.map(page => {
	return { node: page }
	}),
	},
	site: { siteMetadata: { siteUrl } },
	}
	}

as you see it forces url key to be siteUrl. In your case it is just url. I assume if you change it in the gatsby-config from siteMetadata.url to siteMetadata.siteUrl it should work.

But I do agree that this is super confusing. Feel free to submit a PR with additional validation that will provide better error messages in these cases!

[janosh]

But I do agree that this is super confusing. Feel free to submit a PR with additional validation that will provide better error messages in these cases!

I was just about to say that. Why have a resolveSiteUrl option if the plugin throws anyway as soon as the site's url is named anything other than siteUrl.

[vladar]

@moonmeister Do you have any context on this? (sorry if tagging inappropriately)

[janosh]

I think exports.onPostBuild is doing things in the wrong order: first excluding things matching exclude and then serializing the results.

gatsby/packages/gatsby-plugin-sitemap/src/gatsby-node.js

Lines 40 to 60 in 6a9ffcc

	const filteredRecords = filterQuery(
	queryRecords,
	excludeOptions,
	basePath,
	resolveSiteUrl
	)
	const urls = serialize(filteredRecords)
	if (!rest.sitemapSize || urls.length <= rest.sitemapSize) {
	const map = sitemap.createSitemap(rest)
	urls.forEach(u => map.add(u))
	return writeFile(saved, map.toString())
	}
	const {
	site: {
	siteMetadata: { siteUrl },
	},
	} = filteredRecords
	return new Promise(resolve => {

If you do it this way around you first have to make all sorts of assumptions about the structure of the data returned by query during the exclusion. Is there any reason not to serialize first and then perform the exclusion on data in a normalized format? If not I'd happy to submit a PR to reverse the order.

[vladar]

Maybe because exclude relies on page.path which is unavailable after serialization. So we'll have to do some data juggling anyway? But I have zero context on this plugin, so just guessing.