fix(firestore-bigquery-export): add package & update exports

[russellwheatley]

fixes #250

• The latest commit for the gen-schema-view uses the firestore-bigquery-change-tracker package script which is not specified in the package.json.

fixes #196

• The RawChangelogViewSchema was not exported out of the firestore-bigquery-change-tracker package. This caused the fields property to be undefined at this point in the code.

Here's a screenshot of the gen-schema-view script working again:

Please note that the gen-schema-view package will still not work until this PR has been merged and the firestore-bigquery-change-tracker & gen-schema-view packages have been published again.

[newadays]

Thanks @russellwheatley Do we have update on merging the PR?

[stari4ek]

firestore-bigquery-change-tracker:^1.1.2 does not have RawChangelogViewSchema

[stari4ek]

And since there is new dependency for firestore-bigquery-change-tracker DEVSETUP.md should be also updated

[laurenzlong]

Thanks @russellwheatley ! Can you please

• Increment version of firestore-bigquery-change-tracker (it should be 1.1.4 since I just published 1.1.3)
• Increment version of gen-schema-view
• Mark this PR as "ready for review" if it is ready to be merged and the packages are ready to be published?

[kiendt07]

Pls merge this PR soon 🙇‍♂️

[laurenzlong]

firestore-bigquery-change-tracker and gen-schema-view have both been updated on NPM.

[sommes54]

{ "fields": [ { "name": "main", "type": "string" }, { "name": "medium", "type": "string" }, { "name": "small", "type": "string" }, { "name": "thumbnail", "type": "string" } ], "name": "mainImage", "type": "map" },

ERROR
firestore_export.allProducts_schema_schema_changelog. Cannot add fields (field: mainImage_main)

I am not sure if this is caused by the change but map type is no longer working. @russellwheatley any thoughts?

[laurenzlong]

If your Firestore document is structured like:

"mainImage": {
  "main": "url1",
  "small": "url2",
  "medium": "url3",
  "thumbnail": "url4"
}

Can you ensure your schema file is:

{
  "fields": [
    {
      "fields": [
 { "name": "main", "type": "string" }, { "name": "medium", "type": "string" }, { "name": "small", "type": "string" }, { "name": "thumbnail", "type": "string" }
      ],
      "name": "mainImage",
      "type": "map"
    }
  ]
}

Note the top level fields key, in addition to the nested fields key.

[newadays]

Thank you @laurenzlong. I tried to run with the format you shared with but it gave a different error.

{"code":400,"errors":[{"message":"Provided Schema does not match Table PROJECT_ID:firestore_export.allProducts_schema_schema_changelog. Cannot add fields (field: mainImage_main)","domain":"global","reason": "invalid"}],"message":"Provided Schema does not match Table PROJECT_ID:firestore_export.allProducts_schema_schema_changelog. Cannot add fields (field: mainImage_main)"} Provided Schema does not match Table PROJECT_ID:firestore_export.allProducts_schema_schema_changelog. Cannot add fields (field: mainImage_main)

It seems the script adds "_" whereas the schema generated from the Firestore export has the "." map notations e.g. mainImage.main is the correct field name where as "mainImage_main" does not exist. Is there a way to have the name generated by the firestore export.

[laurenzlong]

Hi @newadays could you please file a new Github issue for this?