refactor(python): Prepare deltalake 1.0

[ion-elgreco]

Some minor changes to handle the breaking changes that will land in 1.0.0, we can decide to merge this before or after 1.0.0 lands, I don't have a preference : )

• closes Remove pyarrow import in scan_delta #22927

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.38%. Comparing base (5993d95) to head (0cb3e80).
Report is 37 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #22931      +/-   ##
==========================================
- Coverage   80.66%   80.38%   -0.28%     
==========================================
  Files        1672     1682      +10     
  Lines      221997   223231    +1234     
  Branches     2798     2803       +5     
==========================================
+ Hits       179065   179440     +375     
- Misses      42265    43123     +858     
- Partials      667      668       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ritchie46]

Thanks!