fix(dataframe_serialize): truncate content of dataframe columns

[ArslanSaleem]

Important

DataframeSerializer now truncates DataFrame text values exceeding 200 characters during serialization, with new tests verifying this behavior.

• Behavior:
  • DataframeSerializer.serialize() now truncates text values exceeding 200 characters with an ellipsis.
  • Handles JSON-like objects by converting them to strings before truncation.
• Implementation:
  • Added MAX_COLUMN_TEXT_LENGTH constant to DataframeSerializer.
  • Introduced _truncate_dataframe() method to handle truncation logic.
• Testing:
  • Added test_serialize_with_dataframe_long_strings() in test_dataframe_serializer.py to verify truncation behavior.

^{This description was created by for ebd422e. It will automatically update as commits are pushed.}

[ellipsis-dev]

❌ Changes requested. Reviewed everything up to ebd422e in 1 minute and 53 seconds

More details

• Looked at 109 lines of code in 2 files
• Skipped 0 files when reviewing.
• Skipped posting 5 drafted comments based on config settings.

1. pandasai/helpers/dataframe_serializer.py:25

• Draft comment:
  Consider adding error handling or assertions to ensure that df.schema and df.schema.name exist to avoid attribute errors when schema is missing.
• Reason this comment was not posted:
  Comment was on unchanged code.

2. pandasai/helpers/dataframe_serializer.py:48

• Draft comment:
  Rename the inner function variable name 'value' to a more descriptive name (e.g., 'cell_value') to improve clarity.
• Reason this comment was not posted:
  Confidence changes required: 50% <= threshold 50%
  None

3. tests/unit_tests/helpers/test_dataframe_serializer.py:44

• Draft comment:
  Consider adding an assertion error message to the assert statement to provide more context on test failures.
• Reason this comment was not posted:
  Confidence changes required: 50% <= threshold 50%
  None

4. pandasai/helpers/dataframe_serializer.py:25

• Draft comment:
  Consider sanitizing or escaping schema attributes (e.g., name and description) when embedding them in the tag to avoid injection risks.
• Reason this comment was not posted:
  Comment was on unchanged code.
5. tests/unit_tests/helpers/test_dataframe_serializer.py:20
• Draft comment:
  Add a test case for verifying that the 'description' attribute is correctly serialized when provided in df.schema.
• Reason this comment was not posted:
  Comment was not on a location in the diff, so it can't be submitted as a review comment.

Workflow ID: wflow_7nTjGmN6IWaqwIss

---

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[codecov]

Codecov Report

Attention: Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review.

Project coverage is 91.72%. Comparing base (890fbba) to head (f7280af).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
pandasai/helpers/dataframe_serializer.py	93.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1691      +/-   ##
==========================================
+ Coverage   91.69%   91.72%   +0.02%     
==========================================
  Files          72       72              
  Lines        2793     2803      +10     
==========================================
+ Hits         2561     2571      +10     
  Misses        232      232              

Flag	Coverage Δ
unittests	91.72% <93.33%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.