Mash Branch

[abhi-airspace-intelligence]

• refactor: use EagerSnapshot in datafusion module (refactor: use EagerSnapshot in datafusion module delta-io/delta-rs#3796)
• feat: allow passing a SessionState into a OptimizeBuilder (feat: allow passing a SessionState into a OptimizeBuilder delta-io/delta-rs#3802)
• refactor: remove table_url from Snapshot (refactor: remove table_url from Snapshot delta-io/delta-rs#3803)
• fix: maintaining load config from state (fix: maintaining load config from state delta-io/delta-rs#3805)
• refactor: consolidate extension planners (refactor: consolidate extension planners delta-io/delta-rs#3804)
• ci: split out integration tests (ci: split out integration tests delta-io/delta-rs#3806)
• feat: access tombstones via TombstoneView (feat: access tombstones via TombstoneView delta-io/delta-rs#3809)
• refactor: use EagerSnapshot in vacuum operation consistently use EagerSnapshot in datafusion module. delta-io/delta-rs#3784 (refactor: use EagerSnapshot in vacuum operation delta-io/delta-rs#3812)
• fix: track log files as vec of batches
• feat: process log repaly on stream
• fix: track single representation of file stats Delta rs seems to allocate a lot of memory delta-io/delta-rs#3371
• refactor: better log data processing isolation
• refactor: avoid downcasting to SessionState (refactor: avoid downcasting to SessionState delta-io/delta-rs#3813)
• add deletion_vector_descriptor method
• chore: add some rustdoc for a new/temporary public method
• refactors: move find_files into dedicated mod (refactor: move find_files into dedicated mod delta-io/delta-rs#3815)
• chore: remove unreferenced file (chore: remove unreferenced file delta-io/delta-rs#3819)
• feat: shim kernel Scan and ScanBuilder (feat: shim kernel Scan and ScanBuilder delta-io/delta-rs#3818)
• chore(deps): update datatest-stable requirement from 0.2 to 0.3 (chore(deps): update datatest-stable requirement from 0.2 to 0.3 delta-io/delta-rs#3817)
• feat: expose arrow schema on snapshots (feat: expose arrow schema on snapshots delta-io/delta-rs#3822)
• fix: update deprecation versions to next release (fix: update deprecation versions to next release delta-io/delta-rs#3828)
• chore: unify inconsistent SessionState in datafusion operations (chore: unify inconsistent SessionState in datafusion operations delta-io/delta-rs#3816)
• fix: Protect recently written uncommitted files in vacuum Full mode
• feat: enable ability to do writes through Unity Catalog (feat: enable ability to do writes through Unity Catalog delta-io/delta-rs#3834)
• chore(performance): optimize JSON parsing in get_actions and snapshot reading
• refactor(bench): remove baseline and keep json_parsing benchmark
• perf(path): only clone string for the path (perf(path): only clone string for the path delta-io/delta-rs#3841)
• feat(tracing): add tracing spans to all I/O sections (feat(tracing): add tracing spans to all I/O sections delta-io/delta-rs#3795)
• feat(bench): add new benchmarking script, harness, and profiling guide (feat(bench): add new benchmarking script, harness, and profiling guide delta-io/delta-rs#3840)
• chore: bump version from 1.1.4 to 1.2.0 (chore: bump version from 1.1.4 to 1.2.0 delta-io/delta-rs#3842)
• feat: datafusiuon based kernel engine
• fix: update pyproject.toml (fix: update pyproject.toml delta-io/delta-rs#3854)
• Upgrade datafusion, arrow and parquet
• feat: allow RecordBatchWriter to pass through pass-through-commit-properties
• perf: support pushing physical filters down through DeltaScan (perf: support pushing physical filters down through DeltaScan delta-io/delta-rs#3859)
• chore: remove some deprecated methods (chore: remove some deprecated methods delta-io/delta-rs#3861)
• fix: resolve some warnings (fix: resolve some warnings delta-io/delta-rs#3862)
• chore: deprecate file_actions on state (chore: deprecate file_actions on state delta-io/delta-rs#3863)
• feat: consolidate datafusion session setup
• Fix Result handling after get_actions async removal
• chore: change the core and meta crate versions for release
• chore: use form based issue templates (chore: use form based issue templates delta-io/delta-rs#3865)
• feat: in-flight, streaming PartitionWriter (feat: in-flight, streaming PartitionWriter delta-io/delta-rs#3857)
• chore: add python deprecation warnings (chore: add python deprecation warnings delta-io/delta-rs#3869)
• fix: use single writer for all partition streams (fix: use single writer for all partition streams delta-io/delta-rs#3870)
• Implement smoke 'benchmark'
• Extract merge benchmark to CLI
• Port TPC-DS benchmarks
• Cleanup
• Enable clap env feature and update README
• fix: add regression test for working with dotted-named columns in Python
• fix: add a regression test while I'm tooting around
• chore: clean up missed python formatting in last pull request
• feat: allow for lazy loading files in operations (feat: allow for lazy loading files in operations delta-io/delta-rs#3872)
• fix: remove manylinux 217 builds for aarch64 (fix: remove manylinux 217 builds for aarch64 delta-io/delta-rs#3880)
• fix: display kernel-rs errors better (fix: display kernel-rs errors better delta-io/delta-rs#3883)
• fix: needs ci release python (fix: needs ci release python delta-io/delta-rs#3885)
• feat: add multiple constraints at once (feat: add multiple constraints at once delta-io/delta-rs#3879)
• chore: create the next release of the rust core package
• fix: surface the correct kernel objectstore error (fix: surface the correct kernel objectstore error delta-io/delta-rs#3888)
• feat(memory): optimize collection preallocation where capacity is known (feat(memory): optimize collection preallocation where capacity is known delta-io/delta-rs#3895)
• feat: tracing spans across threadpool (feat: tracing spans across threadpool delta-io/delta-rs#3894)
• chore: reduce wheel size (chore: reduce wheel size delta-io/delta-rs#3878)
• fix: use the default features of aws-config
• perf(snapshot): use std::mem:take to reduce memory allocation and comsumption without cloning
• perf(snapshot): use std::mem:take to reduce memory allocation and comsumption without cloning
• create an dumb solution for generate symlink
• add more docs to make the generate command
• feat: bring the manifest generation into DeltaOps and wire into Python
• feat: implement partition-capable generation of symlink_format_manifest files
• feat(typed-builder): migrate builder to use TypedBuilder in aws crate
• feat(typed-builder): migrate builder to use TypedBuilder in catalog-unity crate
• chore: remove deprecated warnings on pub(crate) and a few others
• chore: grab-bag of compiler warning removals
• Decimal fix
• feat: add GCS auto-registration via ctor hooks (feat: add GCS auto-registration via ctor hooks delta-io/delta-rs#3923)
• chore: bump the patch version to release fixes
• chore(cargo): unify cargo profiles with dev, ci and release for Python and Rust
• chore(ci): adapt the integration workflow for the CI
• fix: handle Urls with dots in them correctly in DeltaTableBuilder::from_uri
• chore: add some opportunistic testing of PartitionsExt
• chore(deps): update ctor requirement from 0.2 to 0.6
• chore: remove proofs/ which are no longer used
• chore(deps): update convert_case requirement from 0.8.0 to 0.9.0
• chore: adding more test coverage to the Gcp crate
• Fix: Handle empty tables in get_add_actions()
• test: fix test_get_add_actions_on_empty_table assertions
• chore: add a regression test into the Rust crate
• chore: removing references to using partition_filters for partition overwrite (chore: removing references to using partition_filters for partition overwrite delta-io/delta-rs#3912)
• chore: remove Python 3.9 from our build infrastructure
• Consolidate target_file_size and allow unbounded writes
• Move to u64 to represent file sizes
• Make sure z-ordering works as expected
• Make write with None snapshot write to default size
• type signature
• Fix python type signatures
• Restore get_target_file_size
• Add support for blind appends

[github-actions]

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.