feature(LocalLoader): enabling local loader to use duckdb loader

[scaliseraoul]

• Tests added and passed if fixing a bug or adding a new feature.
• All code checks passed.

---

Important

Refactor LocalDatasetLoader to use DuckDB for query execution, updating LocalQueryBuilder and tests accordingly.

• Behavior:
  • LocalDatasetLoader in local_loader.py now uses DuckDB for executing queries instead of loading files directly.
  • LocalQueryBuilder in local_query_builder.py constructs queries for DuckDB using read_csv and read_parquet.
• Tests:
  • Updated tests in test_loader.py to mock execute_query instead of file reading methods.
  • Added tests in test_query_builder.py for LocalQueryBuilder to verify query construction for CSV and Parquet.
  • Modified test_group_by.py to reflect changes in query building for local sources.

^{This description was created by for 87d12c5. It will automatically update as commits are pushed.}

[ellipsis-dev]

❌ Changes requested. Reviewed everything up to 87d12c5 in 2 minutes and 26 seconds

More details

• Looked at 408 lines of code in 5 files
• Skipped 0 files when reviewing.
• Skipped posting 6 drafted comments based on config settings.

1. pandasai/data_loader/local_loader.py:27

• Draft comment:
  Initializing LocalQueryBuilder with both schema and dataset_path is clear. Ensure that any future changes to the constructor are reflected in tests so that context awareness of the local path is maintained.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50%
  None

2. pandasai/data_loader/local_loader.py:39

• Draft comment:
  Refactored load() method now uses execute_query to build and execute the query. Confirm that any prior filtering/transformations are intentionally removed and that execute_query covers all necessary operations.
• Reason this comment was not posted:
  Confidence changes required: 33% <= threshold 50%
  None

3. tests/unit_tests/data_loader/test_loader.py:15

• Draft comment:
  Tests now correctly patch execute_query instead of the previous _read_csv_or_parquet. Ensure tests continue to cover schema validation and file reading for all supported local source types.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50%
  None

4. pandasai/data_loader/local_loader.py:11

• Draft comment:
  Unused import: 'LOCAL_SOURCE_TYPES' is imported but no longer used. Consider removing this import for cleaner code.
• Reason this comment was not posted:
  Comment was not on a location in the diff, so it can't be submitted as a review comment.

5. pandasai/query_builders/local_query_builder.py:22

• Draft comment:
  To match the test expectations, consider returning uppercase function calls (e.g. 'READ_CSV' and 'READ_PARQUET') instead of 'read_csv'/'read_parquet'.
• Reason this comment was not posted:
  Marked as duplicate.

6. pandasai/data_loader/local_loader.py:48

• Draft comment:
  Good use of SQL sanitization via is_sql_query_safe. Consider rewording the error message to 'The SQL query is considered unsafe and will not be executed.' for clarity.
• Reason this comment was not posted:
  Confidence changes required: 33% <= threshold 50%
  None

Workflow ID: wflow_lCEcJtmwyc6i1W5M

---

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.