Add source code links to documentation on webstie.

[rockchalkwushock]

Description

Add generation of source code links to API documentation for the project.

Example

## createNodeId [(source)](github link to specific line in docs)

createNodeId (source)

[rockchalkwushock]

@m-allanson

Yesterday I spoke with @KyleAMathews about this and using the AST to achieve the goal of this issue. He referenced your PR #4057 and I understand the resolve-module-exports code. I'm not understanding though where exactly to integrate something similar in the gatsby build process. It would have to be after the page is built I would think; unless it's possible to stream the AST while searching for the function name and line number.

The very rough thought process in my head:

1. gatsby builds page
2. intercept AST
3. search for function names
4. retrieve line number via retainLines babel property
5. create links using the line number
6. rebuild page???

Looking for some advice or guidance. Might have bit off more than I can chew with this.

[KyleAMathews]

Oh it just occurred to me that https://www.npmjs.com/package/gatsby-transformer-documentationjs might already be exposing the line numbers for functions — first thing to do would be to check what it's outputting by running gatsby develop on gatsbyjs.org (in the www directory) and then try querying JS files and exploring the schema gatsby-transformer-documentationjs creates — see the queries used on those pages already https://github.com/gatsbyjs/gatsby/blob/master/www/src/pages/docs/node-apis.js

If line numbers aren't exposed by documentationjs, I'd add this functionality as a new local plugin for gatsbyjs.org since it's super custom to us. You'd use onCreateNode to watch for new File nodes being created. Then for the specific files we care about, you'd then do the babel parse and extract out line numbers and then add that data back to the File node with createNodeField. You'd then query that data in the different reference page components and use that to construct links back to the source on Github.

Make sense? :-)

[rockchalkwushock]

@KyleAMatthews

I think I understand. Will be easier to do so when I’m back in front of my computer to start digging around. Hopefully I can get back to work on it tonight. Thank you for the guidance.

[KyleAMathews]

Thanks for jumping on this! It'll be sweet once it's going. Keep asking questions if you get stuck at all!

[rockchalkwushock]

@KyleAMathews

Looks like here we are removing the line number through the transformer.

[rockchalkwushock]

Update

Right now this is what I have and I'm a little perplexed as to why I get a whole lot more than one console.log():

// Helper function
// Will need to make this handle the other doc case.
const isMemberOf = str => str === `api-node-docs`

exports.onCreateNode = ({ node, boundActionCreators, getNode }) => {
  const { createNodeField } = boundActionCreators
  // ... other code for site
  } else if (
    node.internal.type === `DocumentationJs` &&
    isMemberOf(node.memberof)
  ) {
    const babylon = require(`babylon`)
    const traverse = require(`babel-traverse`).default

    // query returns parent with '"path to file" absPath .....' trim that off
    const absPath = node.parent.substring(0, node.parent.search(`absPath`) - 1)
    const code = fs.readFileSync(absPath, `utf-8`)

    const babylonOpts = {
      sourceType: `module`,
      allowImportExportEverywhere: true,
      plugins: [
        `jsx`,
        `doExpressions`,
        `objectRestSpread`,
        `decorators`,
        `classProperties`,
        `exportExtensions`,
        `asyncGenerators`,
        `functionBind`,
        `functionSent`,
        `dynamicImport`,
        `flow`,
      ],
      retainLines: true, // @see https://babeljs.io/docs/usage/api/#options
    }

    const ast = babylon.parse(code, babylonOpts)
    traverse(ast, {
      AssignmentExpression: function AssignmentExpression(astPath) {
        if (
          astPath.node.left.property.type === `Identifier` &&
          astPath.node.left.property.name === `onPostBuild`
        ) {
          // Perhaps it's just early but should this not only log once?
          console.log(astPath.node.left.property.loc)
        }
        return null
      },
    })
  }
}

The output in my terminal is:

SourceLocation {
  start: Position { line: 199, column: 8 },
  end: Position { line: 199, column: 19 },
  identifierName: 'onPostBuild' }
# repeats 17 more times
SourceLocation {
  start: Position { line: 197, column: 8 },
  end: Position { line: 197, column: 19 },
  identifierName: 'onPostBuild' }
# ditto

I feel like I'm headed in the right direction for getting the line numbers. I'm just about there and then can tweak the code to work with the other documents and create the new node field. I will unfortunately be out the rest of the day. I might get a chance to take a crack at this more this evening, but if not will spend a good amount of Monday working on it.

[m-allanson]

@rockchalkwushock Can you post a link to your code here? Maybe I can try running it to see what's happening. Although I'm very new to using babylon too, so I might not have any answers :)

[rockchalkwushock]

@m-allanson

This is the branch with the code. I didn't get time to look at it more last night. Planning on looking at it more at 1300 today. I think I'm close to having the AST part figured out, then it will be a matter of creating the node and updating the link generation to add the source code link.

[KyleAMathews]

Huh, wonder why we remove the line number. What happens if you delete that line?

[rockchalkwushock]

@KyleAMathews

DM'ing you on Discord

[KyleAMathews]

Due to the high volume of issues, we're closing out older ones without recent activity. Please open a new issue if you need help!