fix(export-workbook): export all schema fields even when values missing

[devin-ai-integration]

Summary

Fixes IA-294777 where columns defined in the sheet schema but missing from record values were completely omitted from Excel exports.

Root Cause: The export plugin iterated over Object.keys(record.values), which only included fields present in the record values object. If a field was defined in the schema but the API didn't return it in record values (e.g., null/empty values), that column would be missing from the export entirely.

Solution: Changed to iterate over the union of:

1. All schema fields (in blueprint order) - ensures all defined columns appear
2. Additional fields present in record values - preserves allowAdditionalFields functionality

Missing values are now exported as empty cells rather than omitting the column entirely.

Performance Optimization: Moved schema keys computation outside the record loop to avoid O(n*m) overhead where n = records, m = fields.

Please explain how to summarize this PR for the Changelog:

Fixed issue where columns defined in sheet schema but missing from record values were omitted from Excel exports. Export now includes all schema fields even when values are null/empty, while preserving additional fields from allowAdditionalFields.

Tell code reviewer how and what to test:

Manual Testing:

1. Create a sheet with fields that may have null/empty values (e.g., Employee ID, Status, End Date)
2. Upload records where some of these fields are missing/null in the API response
3. Export the workbook to Excel using the export-workbook plugin
4. Verify all schema columns appear in the export (even if empty)
5. If using allowAdditionalFields, verify additional columns still appear after schema columns

Key Review Points:

• ⚠️ Column ordering change: Schema fields now always appear first, then additional fields. Confirm this ordering is acceptable for users.
• ⚠️ Performance: The new implementation creates a Set and filters keys once per sheet (moved outside record loop). For large datasets (10k+ records), verify this doesn't cause issues.
• ⚠️ Duplicate labels: If columnNameTransformer produces identical labels for different keys, json_to_sheet may collapse columns. Review if deduplication is needed.
• ⚠️ Edge cases: Review handling of:
  • Records with no values at all
  • Sheets with allowAdditionalFields enabled
  • Fields excluded via excludeFields option
  • Empty/null vs undefined cell values

Potential Issues:

• ⚠️ No integration test added for this specific scenario (schema field missing from record values)
• ⚠️ Root cause not fully verified - didn't confirm why certain fields are missing from API response
• ⚠️ CI shows unrelated E2E test failures in record-hook plugin (appears to be flaky "timeout" errors unrelated to export-workbook changes)

---

Link to Devin run: https://app.devin.ai/sessions/69f9079121c84b029c7a3b02d21eaf40
Requested by: [email protected] (@cnharrison)

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[promptless]

📝 Documentation updates detected!

New suggestion: Add changelog entry for export-workbook schema fields fix

[devin-ai-integration]

CI E2E Test Failures Analysis

The E2E test failures appear to be unrelated flaky tests rather than issues caused by this PR. Evidence:

Test Results Across 3 CI Runs:

1. Run 1 (commit 03b1728): @flatfile/plugin-record-hook E2E - 1 test failed with "fetch failed" error
2. Run 2 (commit 3f477e2): @flatfile/plugin-record-hook E2E - 5 tests failed with timeout errors
3. Run 3 (commit 953e496): @flatfile/plugin-automap E2E - 1 test failed with timeout error

Why These Are Unrelated:

• ✅ All 33 unit tests pass consistently, including @flatfile/plugin-export-workbook unit tests
• ✅ Failures occur in different plugins (record-hook, automap) than the modified plugin (export-workbook)
• ✅ Inconsistent failure patterns: different plugins fail on different runs with different error types
• ✅ This PR only modifies plugins/export-workbook/src/plugin.ts - no shared utilities or cross-plugin dependencies affected
• ✅ The changes are purely internal to export-workbook's record processing logic

Recommendation:

These E2E failures appear to be environmental issues (network timeouts, API availability) rather than functional regressions. The export-workbook plugin changes are isolated and all relevant unit tests pass.

Would appreciate maintainer confirmation if these E2E flakes are a known issue in CI.

[devin-ai-integration]

Closing this PR to investigate the root cause further.

After examining the customer's Excel file, I found that the columns (Employee ID, Status, End date) ARE present in the export but have empty cells. This suggests the issue is not about missing columns, but rather about data not being populated.

The customer reports that data IS visible in the review screen but NOT in the exported file, which points to a potential data layer mismatch between what the UI displays and what the export plugin accesses.

Will investigate further and provide findings before proposing any code changes.