export-toolkit

Tool for exporting data to a format that can be used for reporting such as CSV, JSON, etc.

✨ Features

• 🚀 Fluent Builder API - Intuitive, chainable configuration
• 📝 CSV & JSON Support - Export to popular formats
• 🔄 Async Generator Streaming - Handle large datasets efficiently
• 🪝 Lifecycle Hooks - Transform, validate, and track progress
• 💪 Type-Safe - Full TypeScript support with strict typing
• ⚡ High Performance - Automatic batching and memory optimization
• 🎯 Commander.js Integration - Perfect for CLI tools
• 🧪 Well-Tested - 170+ tests with 80%+ coverage

🚀 Quick Start

Installation

npm install @scottluskcis/export-toolkit
# or
pnpm add @scottluskcis/export-toolkit
# or
yarn add @scottluskcis/export-toolkit

Simple Export

import { outport } from '@scottluskcis/export-toolkit';

interface User {
  id: number;
  name: string;
  email: string;
}

const users: User[] = [
  { id: 1, name: 'Alice', email: '[email protected]' },
  { id: 2, name: 'Bob', email: '[email protected]' },
];

// CSV export
await outport<User>().to('./users.csv').write(users);

// JSON export
await outport<User>().to('./users.json').prettyPrint().write(users);

With Configuration

// Tab-separated CSV with custom headers
await outport<User>()
  .to('./users.tsv')
  .withDelimiter('\t')
  .withHeaders(['ID', 'Full Name', 'Email Address'])
  .withUtf8Bom(true)
  .write(users);

With Progress Tracking

await outport<User>()
  .to('./users.csv')
  .onProgress((current, total) => {
    console.log(`Progress: ${current}/${total}`);
  })
  .write(users);

Streaming Large Datasets

async function* fetchUsers(): AsyncGenerator<User> {
  for (let page = 1; page <= 100; page++) {
    const users = await api.getUsers(page);
    for (const user of users) {
      yield user;
    }
  }
}

// Automatically batched for efficiency
const result = await outport<User>()
  .to('./users.csv')
  .withBatchSize(100)
  .onProgress((count) => console.log(`Exported ${count} users...`))
  .fromAsyncGenerator(fetchUsers());

console.log(`Total exported: ${result.value}`);

Commander.js Integration

import { Command } from 'commander';
import { outport } from '@scottluskcis/export-toolkit';

const program = new Command();

program
  .command('export')
  .option('-o, --output <file>', 'Output file')
  .action(async (options) => {
    const users = await fetchUsers();

    await outport<User>()
      .to(options.output)
      .onProgress((current, total) => {
        process.stdout.write(`\rExporting: ${current}/${total}`);
      })
      .onComplete((result, total) => {
        if (result.success) {
          console.log(`\n✓ Exported ${total} users`);
        }
      })
      .write(users);
  });

📚 Documentation

• Builder API Guide - Complete guide to the fluent builder API
• CSV Writer Guide - CSV-specific examples and patterns
• JSON Writer Guide - JSON-specific examples and patterns
• Type Safety Examples - TypeScript usage patterns

🎯 Key Concepts

Builder Pattern

The fluent builder API makes configuration intuitive and self-documenting:

await outport<User>()
  .to('./users.csv') // Where to write
  .withDelimiter(',') // CSV config
  .withHeaders(['ID', 'Name']) // Custom headers
  .onProgress(trackProgress) // Lifecycle hooks
  .write(users); // Execute

Lifecycle Hooks

Tap into the export process at key points:

await outport<User>()
  .to('./users.csv')
  .onBeforeWrite((data) => data.filter((u) => u.active)) // Transform
  .onProgress((current, total) => updateUI(current)) // Track
  .onAfterWrite((data, count) => logExport(count)) // Post-process
  .onError((error) => handleError(error)) // Error handling
  .onComplete((result, total) => notify(total)) // Completion
  .write(users);

Async Generator Streaming

Process millions of records without loading them all into memory:

async function* streamFromDatabase() {
  let offset = 0;
  const batchSize = 1000;

  while (true) {
    const records = await db.query({ offset, limit: batchSize });
    if (records.length === 0) break;

    for (const record of records) {
      yield record;
    }

    offset += batchSize;
  }
}

// Automatically batched and memory-efficient
await outport<Record>()
  .to('./records.csv')
  .withBatchSize(500)
  .fromAsyncGenerator(streamFromDatabase());

🏗️ Architecture

Outport follows SOLID principles and clean architecture:

• Single Responsibility: Each class has one job (formatting, writing, batching)
• Open/Closed: Extend with hooks without modifying core code
• Liskov Substitution: All writers implement the same interface
• Interface Segregation: Separate interfaces for different concerns
• Dependency Inversion: Depend on abstractions, not concretions

Core Components

Builder API (Fluent Interface)
     ↓
WriterFactory (Abstraction)
     ↓
├── CsvWriter ──→ CsvFormatter, CsvHeaderManager
└── JsonWriter ──→ JsonFormatter
     ↓
FileWriter (I/O Abstraction)
     ↓
Node.js File System

🔧 Development

Setup

# Clone the repository
git clone https://github.com/scottluskcis/export-toolkit.git
cd export-toolkit

# Install dependencies
pnpm install

Available Scripts

Script	Description
pnpm run build	Compile TypeScript to JavaScript
pnpm run test	Run tests once
pnpm run test:watch	Run tests in watch mode
pnpm run test:coverage	Generate test coverage report
pnpm run lint	Check for linting errors
pnpm run lint:fix	Fix auto-fixable linting errors
pnpm run format	Format all files with Prettier
pnpm run format:check	Check if files are formatted correctly
pnpm run typecheck	Type check without emitting files
pnpm run ci	Run all CI checks (typecheck, lint, format, build, test)

Project Structure

export-toolkit/
├── .github/           # GitHub Actions workflows and configs
├── docs/              # Documentation
│   └── csv-writer.md  # CSV Writer usage guide
├── src/               # Source TypeScript files
│   ├── index.ts       # Main entry point
│   └── index.test.ts  # Test files
├── dist/              # Compiled JavaScript (generated)
├── coverage/          # Test coverage reports (generated)
├── .nvmrc             # Node.js version specification
├── package.json       # Project metadata and dependencies
├── tsconfig.json      # TypeScript configuration
├── vitest.config.ts   # Vitest test configuration
├── eslint.config.js   # ESLint configuration (flat config)
└── .prettierrc        # Prettier configuration

📚 Documentation

• CSV Writer Guide - Examples and usage patterns for the CSV writer
• JSON Writer Guide - Examples and usage patterns for the JSON writer

🧪 Testing

This project uses Vitest for testing with the following features:

• ✅ Global test APIs (describe, it, expect)
• 📊 Coverage reports with v8
• ⚡ Fast execution with Vite
• 🎯 80%+ coverage threshold

Run tests:

# Run once
pnpm run test

# Watch mode
pnpm run test:watch

# With coverage
pnpm run test:coverage

🎨 Code Quality

• TypeScript: Strict mode with modern ES2022+ features
• ESLint: v9+ with flat config and TypeScript support
• Prettier: Consistent code formatting
• Vitest: Comprehensive test coverage
• Husky: Pre-commit hooks for quality checks

🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Write tests for your changes
4. Make your changes
5. Ensure all checks pass (pnpm run ci)
6. Commit your changes (git commit -m 'Add amazing feature')
7. Push to the branch (git push origin feature/amazing-feature)
8. Open a Pull Request

Please make sure to:

• Write tests for new functionality
• Follow the existing code style
• Update documentation as needed
• Ensure all CI checks pass

📄 License

MIT © scottluskcis

🔗 Links

• GitHub Repository
• Issue Tracker
• Changelog