Skip to content

Add activity rendering and direct PDF raster extraction#20

Merged
nicpottier merged 5 commits intomainfrom
nicpottier/activity-rendering
Feb 12, 2026
Merged

Add activity rendering and direct PDF raster extraction#20
nicpottier merged 5 commits intomainfrom
nicpottier/activity-rendering

Conversation

@nicpottier
Copy link
Contributor

Summary

  • Activity rendering: Add LLM-driven rendering for 7 interactive activity types (multiple choice, fill-in-the-blank, fill-in-a-table, matching, sorting, true/false, open-ended) with Liquid prompt templates and automatic answer generation via a second LLM call
  • PDF raster extraction: Replace SVG-based raster image extraction with direct extraction from PDF XObject dictionaries via mupdf, including recursion into Form XObjects for nested images, CMYK JPEG color conversion, and deduplication by object number
  • Schema fixes: Change activity answers LLM schema from z.record() to array of {id, value} for OpenAI structured output compatibility; add config validation for answer_prompt only on activity render types

Test plan

  • All 281 tests pass (pnpm test)
  • TypeScript strict mode passes (pnpm typecheck)
  • Manual verification: raven.pdf images extract correctly
  • Manual verification: ancient_egypt.pdf CMYK JPEGs display with correct colors
  • Manual verification: ancient_egypt pages 6-7 extract nested Form XObject images
  • Manual verification: cuaderno3.pdf extracts correctly

…jects

Activity rendering:
- Add Liquid prompt templates for 7 activity types with answer variants
- Extend config with activity render strategies and per-section model resolution
- Add HTML validation support for activity-generated IDs
- Update web-rendering types for activity sections

PDF raster extraction:
- Replace SVG-based raster image extraction with direct PDF object extraction
- Recurse into Form XObjects to find nested images
- Handle CMYK JPEG color conversion via mupdf
- Deduplicate images by PDF object number
- Support JPEG format throughout storage and API layers
OpenAI structured outputs don't support z.record() (additionalProperties).
Change activityAnswersLLMSchema from record to array of {id, value} objects,
and convert back to record in render-llm.ts for storage. Update all answer
prompt templates to show the new array format.
…e prompt

- Handle .jpeg extension (not just .jpg) in image serving route
- Add superRefine validation: answer_prompt only allowed for activity render type
- Fix true/false prompt: radio values use "true"/"false", shared data-activity-item per question
Resolve conflicts in config.ts (BookFormat/LayoutType from main +
activity RenderType from branch) and books.ts (new routes from main +
updated image serving from branch). Fix books.test.ts to use new
buffer/format image fields.
@nicpottier nicpottier merged commit 90222be into main Feb 12, 2026
1 check passed
@nicpottier nicpottier deleted the nicpottier/activity-rendering branch February 12, 2026 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant