Upgrade Pyiceberg Dependency

[scott-routledge2]

Changes included in this PR

Upgrade pyicberg test dependency to 0.11.0 and fix errors.

Adds namespace_exists method to fix type errors (just raises NotImplementedErrror)

Testing strategy

PR CI and nightly Iceberg tests

User facing changes

Checklist

• Pipelines passed before requesting review. To run CI you must include [run CI] in your commit message.
• I am familiar with the Contributing Guide
• I have installed + ran pre-commit hooks.

[codecov]

Codecov Report

❌ Patch coverage is 71.42857% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.45%. Comparing base (c33fbb5) to head (357e63f).
⚠️ Report is 212 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1031      +/-   ##
==========================================
+ Coverage   66.68%   68.45%   +1.77%     
==========================================
  Files         186      195       +9     
  Lines       66795    68060    +1265     
  Branches     9507     9706     +199     
==========================================
+ Hits        44543    46593    +2050     
+ Misses      19572    18605     -967     
- Partials     2680     2862     +182     

[scott-routledge2]

I wasn't sure if we can implement this method in some meaningful way since namespace in this context is just a directory and we aren't implementing similar methods like drop_namespace or list_namespaces. We have an implementation of create_namespace but is just does a noop.

[IsaacWarren]

Thanks Scott, I think we need to do this for our Snowflake and S3 Tables catalogs too, right?

[scott-routledge2]

Snowflake

Snowflake and S3 subclass MetaStoreCatalog, which already has an implementation of namespace_exists defined.
https://github.com/apache/iceberg-python/blob/main/pyiceberg/catalog/__init__.py#L835

[scott-routledge2]

Looks like the azure-polaris rest catalog tests are hanging:
BodoSQL/bodosql/tests/test_types/test_rest_iceberg_catalog.py::test_rest_catalog_iceberg_write[azure...
Not sure if it's related to anything in the upgrade or a dependency change.

Update:
It looks like the test is hanging on main for me when running locally. I'm not sure if it's maybe related to the order the tests execute?

[scott-routledge2]

So the error in Nightly is an actual bug. Something changed with how Pyiceberg's file io handles paths in Azure filesystems so it can no longer resolve relative paths to the correct container.

Edit:
So the issue turned out to be our monkey patch failing due to an import error, but the import error was being suppressed leading to the strange behavior. I moved the monkey patch outside of the try...except to avoid this issue moving forward.