Skip to content

feat: add bigquery usage extraction #271

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mabdh wants to merge 1 commit into from
Closed

feat: add bigquery usage extraction #271

mabdh wants to merge 1 commit into from

Conversation

@mabdh
Copy link
Member

@mabdh mabdh commented Nov 17, 2021 •
edited
Loading

  • Add google logadmin API integration basic function
  • Calculate table usage
  • Calculate common join usage
  • Integrate with the main bigquery flow
  • Table usage will be a feature flag and pre-calculated before extracting BQ tables and columns

TODO

  • Synchronize table proto with proton (PR is here)

@mabdh mabdh linked an issue Nov 18, 2021 that may be closed by this pull request
Add metadata extractor for bigquery_usage #93
Closed
@mabdh mabdh force-pushed the bigquery-usage-extractor branch from fe3c447 to 2b73d1f Compare November 22, 2021 03:13
@mabdh mabdh changed the title feat(bigquery usage)(wip): add bigquery usage extraction feat(bigquery usage): add bigquery usage extraction Nov 22, 2021
@mabdh mabdh mentioned this pull request Nov 22, 2021
Merged
@mabdh mabdh marked this pull request as ready for review November 22, 2021 08:46
@mabdh mabdh force-pushed the bigquery-usage-extractor branch from 25cb210 to 384a1ec Compare November 25, 2021 11:17
@mabdh mabdh force-pushed the bigquery-usage-extractor branch from 384a1ec to bd1cd65 Compare November 25, 2021 11:23
@mabdh mabdh changed the title feat(bigquery usage): add bigquery usage extraction feat: add bigquery usage extraction Nov 26, 2021
@mabdh
Copy link
Member Author

mabdh commented Nov 26, 2021

Closing this PR as it is duplicated with this PR

@mabdh mabdh closed this Nov 26, 2021
@ravisuhag ravisuhag deleted the bigquery-usage-extractor branch December 6, 2021 05:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add metadata extractor for bigquery_usage

2 participants

@mabdh