firestore-bigquery-export 0.1.3 still have same issue with #236.

[ight-reco]

[REQUIRED] Step 2: Describe your configuration

• Extension name: firestore-bigquery-export (storage-resize-images, firestore-send-email, etc)
• Extension version: 0.1.3

(I'm sorry for a Japanese screenshot)

[REQUIRED] Step 3: Describe the problem

This issue is related to #236.
Simply put, the issue still occurs in my projects.

Steps to reproduce:

After installing firestore-bigquery-export 0.1.3 and create a document that has a DocumentReference field in firestore console, the following error occurred.

ext-firestore-bigquery-export-fsexportbigquery
Error when mirroring data to BigQuery TypeError: Converting circular structure to JSON
    at JSON.stringify (<anonymous>)
    at events.map (/srv/node_modules/@firebaseextensions/firestore-bigquery-change-tracker/lib/bigquery/index.js:63:36)
    at Array.map (<anonymous>)
    at FirestoreBigQueryEventHistoryTracker.<anonymous> (/srv/node_modules/@firebaseextensions/firestore-bigquery-change-tracker/lib/bigquery/index.js:55:33)
    at Generator.next (<anonymous>)
    at fulfilled (/srv/node_modules/@firebaseextensions/firestore-bigquery-change-tracker/lib/bigquery/index.js:20:58)
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:229:7)

I think it's same issue with #236, but the line number of source code is different just a little bit (firestore-bigquery-change-tracker/lib/bigquery/index.js:63:36)

I uninstalled and reinstall the extension, but it has not changed, and reconfigured the extension, but it has not changed. (#236 (comment))
I tried in another project, but that occured.

Expected result

The issue (#236) don't occur.

Actual result

The issue (#236) occurs.

Supplementations

I don't know that these informations help you, but I added just in case.

I downloaded the source code of ext-firestore-bigquery-export-fsexportbigquery from GCP console (not firebase), the CHANGELOG.md and extension.yaml is updated, but package.json and firestore-bigquery-change-tracker/src/bigquery/index.ts is still old.

CHANGELOG.md

## Version 0.1.3
feature - Interpret data more easily with column descriptions in the exported BigQuery data (#138).

- The raw changelog now includes column descriptions.
- The schema-views script allows you to specify custom column descriptions.

fixed - Updated `@firebaseextensions/firestore-bigquery-change-tracker` dependency (fixes issues #235 and #225).

extension.yaml

name: firestore-bigquery-export
version: 0.1.3
specVersion: v1beta

package.json

{
  "name": "firestore-bigquery-export",
  "description": "Export a Cloud Firestore collection to BigQuery",
  "main": "functions/lib/index.js",
  "scripts": {
    "build": "npm run clean && npm run compile",
    "clean": "rimraf functions/lib",
    "compile": "tsc",
    "prepare": "npm run build",
    "test": "jest",
    "generate-readme": "firebase ext:info . --markdown > README.md"
  },
  "author": "Jan Wyszynski <wyszynski@google.com>",
  "license": "Apache-2.0",
  "dependencies": {
    "@firebaseextensions/firestore-bigquery-change-tracker": "^1.1.2",
    "@google-cloud/bigquery": "^2.1.0",
    "@types/chai": "^4.1.6",
    "chai": "^4.2.0",
    "firebase-admin": "^8.0.0",
    "firebase-functions": "^3.3.0",
    "generate-schema": "^2.6.0",
    "inquirer": "^6.4.0",
    "lodash": "^4.17.14",
    "nyc": "^14.0.0",
    "rimraf": "^2.6.3",
    "sql-formatter": "^2.3.3",
    "ts-node": "^7.0.1",
    "typescript": "^3.4.5"
  },
  "private": true
}

firestore-bigquery-change-tracker/src/bigquery/index.ts

  async record(events: FirestoreDocumentChangeEvent[]) {
    await this.initialize();
    const rows = events.map((event) => {
      return {
        insertId: event.eventId,
        json: {
          timestamp: event.timestamp,
          event_id: event.eventId,
          document_name: event.documentName,
          operation: ChangeType[event.operation],
          data: JSON.stringify(event.data),
        },
      };
    });
    await this.insertData(rows);
  }

I couldn't check the running firestore-bigquery-change-tracker version because I couldn't find package-lock.json from the code, but it might be old in my case?
It's public in npm, so I guess my installed version is latest, but I have no idea except for caching case.

[laurenzlong]

Hi @ight-reco did you follow the steps here to ensure that you do receive the dependency update? #236 (comment)

[ight-reco]

@laurenzlong Thank you for your reply!
Yes, I did both!
In addition, I tried initial install in new project that has not used any Firebase extensions, but the error occurs in the project.

[russellwheatley]

firestore-bigquery-export & firestore-bigquery-change-tracker modules were not sharing the same firestore DocumentReference anymore. I guess the recent changes to firestore-bigquery-change-tracker meant they had their own node module albeit with different versions.

I've created a PR (#270) that now uses the class name (DocumentReference) as a more robust solution. Thanks for raising the issue @ight-reco 👍

[laurenzlong]

The fix is now live in v0.1.4, you can update your extension by following the instructions in https://firebase.google.com/docs/extensions/manage-installed-extensions?platform=console#update-version. Please create a new issue if you still experience problems. Thanks for filing!