Skip to content

Commit 89a21c0

Browse files
authored
chore: removing references to using partition_filters for partition overwrite (#3912)
# Description Remove references in docs that suggest using `partition_filters` for selectively overwriting partitions, which has been removed from the `write_deltalake` API. # Related Issue(s) fixes #3904 Signed-off-by: zyd14 <[email protected]>
1 parent 84d954a commit 89a21c0

File tree

1 file changed

+4
-28
lines changed

1 file changed

+4
-28
lines changed

docs/usage/writing/index.md

Lines changed: 4 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ The `data` parameter will accept a Pandas DataFrame, a PyArrow Table, or
55
an iterator of PyArrow Record Batches.
66

77
``` python
8+
>>> import pandas as pd
89
>>> from deltalake import write_deltalake
910
>>> df = pd.DataFrame({'x': [1, 2, 3]})
1011
>>> write_deltalake('path/to/table', df)
@@ -27,33 +28,6 @@ alter the schema as part of an overwrite pass in `schema_mode="overwrite"` or `s
2728
`schema_mode="overwrite"` will completely overwrite the schema, even if columns are dropped; merge will append the new columns
2829
and fill missing columns with `null`. `schema_mode="merge"` is also supported on append operations.
2930

30-
## Overwriting a partition
31-
32-
You can overwrite a specific partition by using `mode="overwrite"`
33-
together with `partition_filters`. This will remove all files within the
34-
matching partition and insert your data as new files. This can only be
35-
done on one partition at a time. All the input data must belong to
36-
that partition or else the method will raise an error.
37-
38-
``` python
39-
>>> from deltalake import write_deltalake
40-
>>> df = pd.DataFrame({'x': [1, 2, 3], 'y': ['a', 'a', 'b']})
41-
>>> write_deltalake('path/to/table', df, partition_by=['y'])
42-
43-
>>> table = DeltaTable('path/to/table')
44-
>>> df2 = pd.DataFrame({'x': [100], 'y': ['b']})
45-
>>> write_deltalake(table, df2, partition_filters=[('y', '=', 'b')], mode="overwrite")
46-
47-
>>> table.to_pandas()
48-
x y
49-
0 1 a
50-
1 2 a
51-
2 100 b
52-
```
53-
54-
This method could also be used to insert a new partition if one doesn't
55-
already exist, making this operation idempotent.
56-
5731
## Overwriting part of the table data using a predicate
5832

5933
!!! note
@@ -64,7 +38,9 @@ When you don’t specify the `predicate`, the overwrite save mode will replace
6438
the entire table. Instead of replacing the entire table (which is costly!), you
6539
may want to overwrite only the specific parts of the table that should be
6640
changed. In this case, you can use a `predicate` to overwrite only the relevant
67-
records or partitions.
41+
records or partitions. If the predicate and source data being written contain
42+
partitions that do not exist in the target table, they will be added to the
43+
target table.
6844

6945
!!! note
7046

0 commit comments

Comments
 (0)