Skip to content

Wrongly handling non-numeric values of double type during the conversion from Delta #524

@emilie-wang

Description

@emilie-wang

Search before asking

  • I had searched in the issues and found no similar issues.

Please describe the bug 🐞

This error happened during the translation of column stats from Delta → internal table →Iceberg. There is an edge case, when the double type field’s value is non-numeric like "NaN" or "Infinity", the value type become String when processing the snapshot from Delta source.

...INFO org.apache.xtable.iceberg.IcebergColumnStatsConverter:76 - name=weight, fieldType=double, valueType=class java.lang.String, getMinValue=-Infinity 
...INFO org.apache.xtable.iceberg.IcebergColumnStatsConverter:80 - name=weight, fieldType=double, valueType=class java.lang.String, getMaxValue=NaN

Are you willing to submit PR?

  • I am willing to submit a PR!
  • I am willing to submit a PR but need help getting started!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions