Removing dataset_id from allocate_ids and get_entities #510

dhermes · 2015-01-08T03:16:27Z

Since we require Key to have a dataset ID, these methods should not be allowed to ignore this value.

For allocate_ids this is cut and dry but for get_entities we may want to allow the user to provide dataset_id since there are potentially many keys to check. (Performance sensitive.)

@pcostell

In the get_entities change, we check that the dataset_id of every Key in a list is identical.

This can cause all kinds of trouble, just in comparing prefixed and un-prefixed alone. Do you think this check can be avoided? Due to googleapis/google-cloud-datastore#59 it's not possible to just set the dataset ID on the Key protobuf and let the backend do the checking for us.

OTOH, creating the protobufs for each Key will be the same O(n) work that checking the dataset IDs will, so maybe the performance cost is negligible?

coveralls · 2015-01-08T03:18:44Z

Coverage remained the same when pulling 9797f96 on dhermes:remove-dependence-on-dataset-id into f577b30 on GoogleCloudPlatform:master.

dhermes · 2015-01-08T03:19:28Z

@tseaver Feel free to ignore this until we chat about #509

pcostell · 2015-01-08T15:05:54Z

ISTM that the Key should not have the dataset id set in this case since the backend will fill in the dataset id if it is empty.

It may be possible for us to update the backend to accept foo and s/e~foo in the key and normalize appropriately, but I'll need to chat with the team to see if I'm missing anything.

dhermes · 2015-01-08T15:53:44Z

@pcostell Thanks. We are currently dropping the dataset ID in a helper called _prepare_key_for_request.

I was just worried that the extra O(n) work could be a performance hit and wanted to see what you thought.

coveralls · 2015-01-08T16:01:04Z

Coverage remained the same when pulling 3f7a067 on dhermes:remove-dependence-on-dataset-id into 6bde37d on GoogleCloudPlatform:master.

coveralls · 2015-01-08T18:24:52Z

Changes Unknown when pulling 294ffe2 on dhermes:remove-dependence-on-dataset-id into * on GoogleCloudPlatform:master*.

dhermes · 2015-01-08T18:47:33Z

@tseaver PTAL. I am torn between leaving in dataset_id=None in get_entities() as a speed-up or leaving the code as-is.

coveralls · 2015-01-08T19:22:16Z

Coverage remained the same when pulling e08c3ef on dhermes:remove-dependence-on-dataset-id into 2b4f52f on GoogleCloudPlatform:master.

coveralls · 2015-01-08T20:08:08Z

Coverage remained the same when pulling daaa26e on dhermes:remove-dependence-on-dataset-id into 77cfe10 on GoogleCloudPlatform:master.

It can be implied from the incomplete_key passed in.

It can be implied from the keys passed in.

dhermes · 2015-01-08T21:53:02Z

@tseaver Rebased on top of #518, PTAL

coveralls · 2015-01-08T21:56:41Z

Coverage remained the same when pulling ac5221b on dhermes:remove-dependence-on-dataset-id into 7c01fcc on GoogleCloudPlatform:master.

tseaver · 2015-01-08T22:34:35Z

I can live with this without the explicit dataset ID param.

dhermes · 2015-01-08T22:35:31Z

Cool. We can always add it back if it is causing performance issues.

Removing dataset_id from allocate_ids and get_entities

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Source-Link: googleapis/synthtool@1b71c10 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:00c9d764fd1cd56265f12a5ef4b99a0c9e87cf261018099141e2ca5158890416 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* feat: Add support for python 3.11 chore: Update gapic-generator-python to v1.8.0 PiperOrigin-RevId: 500768693 Source-Link: googleapis/googleapis@190b612 Source-Link: googleapis/googleapis-gen@7bf29a4 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiN2JmMjlhNDE0YjllY2FjMzE3MGYwYjY1YmRjMmE5NTcwNWMwZWYxYSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* chore: update to gapic-generator-python 1.5.0 feat: add support for `google.cloud.<api>.__version__` PiperOrigin-RevId: 484665853 Source-Link: googleapis/googleapis@8eb249a Source-Link: googleapis/googleapis-gen@c8aa327 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzhhYTMyN2I1ZjQ3ODg2NWZjM2ZkOTFlM2MyNzY4ZTU0ZTI2YWQ0NCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * update version in gapic_version.py * chore: Update to gapic-generator-python 1.6.0 feat(python): Add typing to proto.Message based class attributes feat(python): Snippetgen handling of repeated enum field PiperOrigin-RevId: 487326846 Source-Link: googleapis/googleapis@da380c7 Source-Link: googleapis/googleapis-gen@61ef576 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjFlZjU3NjJlZTY3MzFhMGNiYmZlYTIyZmQwZWVjZWU1MWFiMWM4ZSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: new APIs added to reflect updates to the filestore service - Add ENTERPRISE Tier - Add snapshot APIs: RevertInstance, ListSnapshots, CreateSnapshot, DeleteSnapshot, UpdateSnapshot - Add multi-share APIs: ListShares, GetShare, CreateShare, DeleteShare, UpdateShare - Add ConnectMode to NetworkConfig (for Private Service Access support) - New status codes (SUSPENDED/SUSPENDING, REVERTING/RESUMING) - Add SuspensionReason (for KMS related suspension) - Add new fields to Instance information: max_capacity_gb, capacity_step_size_gb, max_share_count, capacity_gb, multi_share_enabled PiperOrigin-RevId: 487492758 Source-Link: googleapis/googleapis@5be5981 Source-Link: googleapis/googleapis-gen@ab0e217 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYWIwZTIxN2Y1NjBjYzJjMWFmYzExNDQxYzJlYWI2YjY5NTBlZmQyYiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * chore: Update gapic-generator-python to v1.6.1 PiperOrigin-RevId: 488036204 Source-Link: googleapis/googleapis@08f275f Source-Link: googleapis/googleapis-gen@555c094 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNTU1YzA5NDVlNjA2NDllMzg3MzlhZTY0YmM0NTcxOWNkZjcyMTc4ZiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * move version to gapic_version.py; clean up setup.py and owlbot.py * configure release-please to use manifest * set coverage level to 99% * remove commented code * customize testing/constraints-3.7.txt * fix(deps): Require google-api-core >=1.34.0, >=2.11.0 fix: Drop usage of pkg_resources fix: Fix timeout default values docs(samples): Snippetgen should call await on the operation coroutine before calling result PiperOrigin-RevId: 493260409 Source-Link: googleapis/googleapis@fea4387 Source-Link: googleapis/googleapis-gen@387b734 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMzg3YjczNDRjNzUyOWVlNDRiZTg0ZTYxM2IxOWE4MjA1MDhjNjEyYiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * add gapic_version.py * add gapic_version.py Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <[email protected]>

* build: [autoapprove] bump cryptography from 41.0.2 to 41.0.3 Source-Link: googleapis/synthtool@352b9d4 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:3e3800bb100af5d7f9e810d48212b37812c1856d20ffeafb99ebe66461b61fc7 * lint E721 --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <[email protected]>

Source-Link: googleapis/synthtool@7197a00 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:c43f1d918bcf817d337aa29ff833439494a158a0831508fda4ec75dc4c0d0320 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Source-Link: googleapis/synthtool@50db768 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:e09366bdf0fd9c8976592988390b24d53583dd9f002d476934da43725adbb978 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

Source-Link: googleapis/synthtool@06e8279 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:b3500c053313dc34e07b1632ba9e4e589f4f77036a7cf39e1fe8906811ae0fce

….1 (#510)

Fix a common typo: `s/the the /the /`. Fixes #510 🦕

…#1053) * feat!: Support SQLAlchemy 2.0, drop support for 1.x This patch makes this library backward incompatible for versions of SQLAlchemy < 2.0. Fixes #510 * constraints updated * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * fixing README.rst * fixing README.rst * upping sqlalchemy version in constraints-3.8.txt * adding 2.0 version restrictions to owlbot.py * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * fix for * Updated some compliance tests for sqla2 and bq * Addressed snippet errors * revert bad commit * More compliance tests checking * reformatted with black * Changed more compliance tests, updated requirements for testing * Fixed attributeerror in failing sample test * Fixed geolography test failing issue * Minor tweaks to tests and code * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Fixed small error in compliance tests, added pip freeze for owl bot testing * Fixed some failing compliance tests by reformatting * Added UuidTest to compliance tests * Moved back sqlalchemy constraints to 1.4 * Update testing/constraints-3.8.txt * Fixed minimum version of sqlalchemy for 1.4 backwards compatibility * Bumping support for sqlalchemy 1.4.16 for sample tests * Bump setup.py sqlalchemy to 1.4.16 * Updated compliance sqlalchemy to 1.4.16 * Fixed broken code in last merged main, as we need to avoid duplicate entries and potential implicit joins due to identical table names * modified tests for join order variation in 1.4 vs 2.0 * typo * Modified one compliance StringTest thats been flaky * Updated docs * minor fixes to noxfile and README * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * cleaned up code from review, removed unnecessary code and files * Update tests/sqlalchemy_dialect_compliance/test_dialect_compliance.py * create development release 1.11.0.dev0 branch * chore: cleanup compliance tests for sqlalchemy migration (#1013) * chore: remove code for sqlalchemy before 1_4 * reformatted with black: * Removed sqlalchemy compliance tests from versions before 1.4 * removed code in base.py for sqlalchemy < 1.4 * fix coverage issues in base.py * temporarily commented out code lines not passing coverage for testing purposes * replaced functions previously removed for not passing cover * testing removing functions for coverage * add no cover tag to untested code and clean up commented out functions * fix lint issues * black * chore: cleanup compliance file tests after migration * lint * fixed small import error --------- Co-authored-by: Sharoon Thomas <[email protected]> * Revert "chore: cleanup compliance tests for sqlalchemy migration" (#1015) * chore: fix coverage tests sqlalchemy 2.0 migration (#987) * chore: remove code for sqlalchemy before 1_4 * reformatted with black: * Removed sqlalchemy compliance tests from versions before 1.4 * removed code in base.py for sqlalchemy < 1.4 * fix coverage issues in base.py * temporarily commented out code lines not passing coverage for testing purposes * replaced functions previously removed for not passing cover * testing removing functions for coverage * add no cover tag to untested code and clean up commented out functions * fix lint issues * black * Readded deleted tests and renamed them from deprecated names * black --------- Co-authored-by: Sharoon Thomas <[email protected]> * chore: sqlalchemy test compliance suite cleanup (#1018) * chore: remove code for sqlalchemy before 1_4 * reformatted with black: * Removed sqlalchemy compliance tests from versions before 1.4 * removed code in base.py for sqlalchemy < 1.4 * fix coverage issues in base.py * temporarily commented out code lines not passing coverage for testing purposes * replaced functions previously removed for not passing cover * testing removing functions for coverage * add no cover tag to untested code and clean up commented out functions * fix lint issues * black * Readded deleted tests and renamed them from deprecated names * black * chore: sqlalchemy test compliance suite cleanup code * black * black --------- Co-authored-by: Sharoon Thomas <[email protected]> * create development release 1.11.0.dev1 branch * feat: grouping sets, rollup and cube compatibility * create development release 1.11.0.dev2 * test commit to run kokooro tests * removed unnecessary clause function changes, edited tests * test basic implementation of group_by_clause and visit_label * fixed render label as label assignment * added test case * reformat logic * test commit * create development build 1.11.0.dev3 * chore: add more grouping sets/rollup/cube tests (#1029) * chore: add more tests for grouping functions fix * reformatted tests * update changelog * revert changelog * remove note * don't install prerelease in compliance session * sync owlbot * Update tests/sqlalchemy_dialect_compliance/test_dialect_compliance.py * make results order not matter * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Sharoon Thomas <[email protected]> Co-authored-by: Nishant Nayak <[email protected]> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: kiraksi <[email protected]> Co-authored-by: Chalmer Lowe <[email protected]> Co-authored-by: Chalmer Lowe <[email protected]> Co-authored-by: Kira <[email protected]>

…510) Source-Link: googleapis/synthtool@280ddae Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:346ab2efb51649c5dde7756cbbdc60dd394852ba83b9bbffc292a63549f33c17 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* updated CHANGELOG.md [ci skip] * updated setup.cfg [ci skip] * updated setup.py [ci skip] Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

dhermes force-pushed the remove-dependence-on-dataset-id branch from 9797f96 to 3f7a067 Compare January 8, 2015 15:58

dhermes force-pushed the remove-dependence-on-dataset-id branch from 3f7a067 to 294ffe2 Compare January 8, 2015 18:22

dhermes force-pushed the remove-dependence-on-dataset-id branch from 294ffe2 to e08c3ef Compare January 8, 2015 19:13

dhermes force-pushed the remove-dependence-on-dataset-id branch from e08c3ef to daaa26e Compare January 8, 2015 20:05

dhermes mentioned this pull request Jan 8, 2015

Move user friendly methods from datastore.__init__ to dedicated module #519

Merged

dhermes added 2 commits January 8, 2015 13:52

Removing dataset_id from allocate_ids.

9b7dae7

It can be implied from the incomplete_key passed in.

Removing dataset_id from get_entities.

ac5221b

It can be implied from the keys passed in.

dhermes force-pushed the remove-dependence-on-dataset-id branch from daaa26e to ac5221b Compare January 8, 2015 21:52

dhermes added a commit that referenced this pull request Jan 8, 2015

Merge pull request #510 from dhermes/remove-dependence-on-dataset-id

6c5b311

Removing dataset_id from allocate_ids and get_entities

dhermes merged commit 6c5b311 into googleapis:master Jan 8, 2015

dhermes deleted the remove-dependence-on-dataset-id branch January 8, 2015 22:36

dhermes added the api: datastore Issues related to the Datastore API. label Dec 31, 2015

atulep pushed a commit that referenced this pull request Apr 3, 2023

chore(main): release 2.19.0 (#510)

cccdf23

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

atulep pushed a commit that referenced this pull request Apr 6, 2023

chore(main): release 2.19.0 (#510)

a1e62d7

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

atulep pushed a commit that referenced this pull request Apr 6, 2023

chore(main): release 2.19.0 (#510)

86e5c15

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

atulep pushed a commit that referenced this pull request Apr 18, 2023

chore(main): release 2.19.0 (#510)

8ac7e8a

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

parthea pushed a commit that referenced this pull request Aug 15, 2023

chore(deps): update dependency pytest to v7.2.2 (#510)

85ba349

parthea pushed a commit that referenced this pull request Oct 22, 2023

chore(main): release 2.19.0 (#510)

baa83a3

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

parthea pushed a commit that referenced this pull request Nov 22, 2025

chore(deps): update google-github-actions/setup-gcloud action to v0.5…

3df8cd1

….1 (#510)

parthea pushed a commit that referenced this pull request Nov 24, 2025

Cross patch #470 to ads templates (#510)

ddcea4b

parthea pushed a commit that referenced this pull request Nov 24, 2025

docs: fix a typo in google/api_core/page_iterator.py (#511)

c05d769

Fix a common typo: `s/the the /the /`. Fixes #510 🦕

parthea pushed a commit that referenced this pull request Nov 26, 2025

chore: release 1.15.0 (#510)

4878627

* updated CHANGELOG.md [ci skip] * updated setup.cfg [ci skip] * updated setup.py [ci skip] Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Removing dataset_id from allocate_ids and get_entities #510

Removing dataset_id from allocate_ids and get_entities #510

Uh oh!

Conversation

dhermes commented Jan 8, 2015

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

pcostell commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants