Skip to content

fix: to_gbq allows strings for DATE and floats for NUMERIC, require pandas 0.24+ and db-dtypes #423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tswast merged 22 commits into from
Nov 22, 2021
Merged

fix: to_gbq allows strings for DATE and floats for NUMERIC, require pandas 0.24+ and db-dtypes #423

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
9b57878
test: upload DATE column with various dtypes
tswast Nov 10, 2021
907a1d5
add dbdate tests
tswast Nov 10, 2021
cf2a5bf
test with db-dtypes only with newer pandas
tswast Nov 11, 2021
1ea8fda
🦉 Updates from OwlBot
gcf-owl-bot[bot] Nov 11, 2021
b869a9f
sort by row number
tswast Nov 11, 2021
4d304f5
Merge branch 'issue362-to_gbq-date' of github.com:googleapis/python-b…
tswast Nov 11, 2021
b876cc7
Merge branch 'main' into issue362-to_gbq-date
tswast Nov 11, 2021
625df5a
fix: allow strings when writing to DATE and floats when writing to NU…
tswast Nov 11, 2021
bd3604e
🦉 Updates from OwlBot
gcf-owl-bot[bot] Nov 11, 2021
19df618
require db-dtypes
tswast Nov 11, 2021
e0c0a1f
Merge remote-tracking branch 'upstream/issue421-numeric' into issue42…
tswast Nov 11, 2021
ab85d8a
add unit tests for dataframe conversion
tswast Nov 11, 2021
a08f90d
fix unit tests
tswast Nov 11, 2021
4426957
🦉 Updates from OwlBot
gcf-owl-bot[bot] Nov 11, 2021
15d7b51
remove 'default' from system tests. redundant with load_parquet
tswast Nov 11, 2021
5ded1e8
Merge branch 'main' into issue421-numeric
tswast Nov 16, 2021
4ddcf9d
🦉 Updates from OwlBot
gcf-owl-bot[bot] Nov 16, 2021
95f051f
correct repeated_col name
tswast Nov 17, 2021
f8dddb2
used namedtuple in tests
tswast Nov 17, 2021
7563017
protect against explicit None
tswast Nov 17, 2021
92837e6
update tests since pandas 0.24+ is required
tswast Nov 17, 2021
f981705
update conda test pandas version
tswast Nov 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ci/requirements-3.7-0.23.2.conda
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
codecov
coverage
db-dtypes==0.3.0
fastavro
flake8
numpy==1.16.6
Expand Down
1 change: 1 addition & 0 deletions ci/requirements-3.9-NIGHTLY.conda
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
db-dtypes
pydata-google-auth
google-cloud-bigquery
google-cloud-bigquery-storage
Expand Down
2 changes: 1 addition & 1 deletion owlbot.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
templated_files = common.py_library(
unit_test_python_versions=["3.7", "3.8", "3.9", "3.10"],
system_test_python_versions=["3.7", "3.8", "3.9", "3.10"],
cov_level=86,
cov_level=88,
unit_test_extras=extras,
system_test_extras=extras,
intersphinx_dependencies={
Expand Down
3 changes: 2 additions & 1 deletion pandas_gbq/load.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ def cast_dataframe_for_parquet(
column_type = column.get("type", "").upper()
if (
column_type == "DATE"
and dataframe[column_name].dtype != db_dtypes.DateDtype()
# Use extension dtype first so that it uses the correct equality operator.
and db_dtypes.DateDtype() != dataframe[column_name].dtype
):
# Construct converted column manually, because I can't use
# .astype() with DateDtype. With .astype(), I get the error:
Expand Down