Skip to content

Commit e3645ed

Browse files
authored
Move to the paged document support in Verify 33.3 (#40)
1 parent 6209e6c commit e3645ed

26 files changed

Lines changed: 391 additions & 160 deletions

‎claude.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -29,26 +29,26 @@ All source lives under `src/`. Solution file is `src/Verify.PDFium.slnx`.
2929

3030
### Library (`src/Verify.PDFium/`)
3131

32-
Entry point is `VerifyPDFium.Initialize(dpi = 96, outputs = PdfiumOutputs.All)` which registers a stream converter for the `pdf` extension. The converter loads the document with `Morph.PDFium.PdfiumDocument` and returns a `ConversionResult` containing:
33-
1. `PdfInfo` (page count, per-page size in points and extracted text, document information dictionary) serialized as the info file
34-
2. The pdf bytes as a `pdf` target (`BypassComparersForSubsequentOnDifference` set, mirroring Verify.OpenXml)
35-
3. One `png` target per page, named `page_0001` style
32+
Entry point is `VerifyPDFium.Initialize(dpi = 96)` which registers a stream converter for the `pdf` extension. The converter loads the document with `Morph.PDFium.PdfiumDocument` and builds its `ConversionResult` with Verify's `PagedConversion`:
33+
1. The document information dictionary as `Info`, and each page added with its size in points (`PageSize`), its extracted text and its png
34+
2. The pdf bytes as the `Source`, which is what ties the pages and the info file to the pdf for comparison and for the diff tool
35+
3. `PagedConversion` names the pages (`page_0001`), places the text, and writes the info file in the shape every paged document has
3636

37-
`PdfiumOutputs` (`[Flags]`: `Png`, `Text`, `All`) is a global option: without `Png` pages are not rendered, without `Text` page text is not extracted (`PageInfo.Text` stays null and is omitted from the info file). The pdf target is not governed by it. Since it is global, it is tested in a separate project, `src/StaticSettingsTests/` (Png only).
37+
What is produced is decided by Verify's settings, read through `PagedConversion`, not by an option of this plugin: `PageText` (in the info, a file per page, or none), `PagesToInclude`, `ExcludeDerivedTargets("png")` and `ExcludeTargets("pdf")`. The converter asks `IncludeImages`, `IncludeText` and `Pages` before rendering or reading, so nothing left out is produced. The global forms of those settings are tested in a separate project, `src/StaticSettingsTests/` (no text).
3838

3939
To keep snapshots stable for PDFs freshly generated at test time, the non-deterministic fields are neutralized two ways:
4040
- **In the `pdf` bytes** (`DeterministicPdf.PdfNormalizer.Normalize`): the trailer `/ID`, the info-dictionary `/CreationDate`/`/ModDate`, and the XMP dates plus `xmpMM:DocumentID`/`InstanceID`/`OriginalDocumentID` and `dc:date` are overwritten by an in-place byte scan (no string round-trip, no regex). That value-zeroing is length-preserving, so cross-reference offsets survive it. A final `CanonicalizeXmp` pass then collapses the XMP packet's whitespace — Apache FOP serializes the packet through the platform's XML writer, so indentation varies by JRE — and that pass is **not** length-preserving: it rebuilds the buffer and repairs both the metadata stream length and the classic cross-reference table. A document it cannot safely rewrite (cross-reference stream, incremental update, unlocatable stream length) comes back unchanged. `Normalize` copies its input, so a caller keeps ownership of the array passed in; the call is still made only after the `PdfiumDocument` (which reads lazily from the same buffer) is disposed. Values compressed away inside an `/ObjStm` or a flate-compressed XMP packet are not reachable by the plaintext scan and are left alone.
4141
- **In the info file** (`PdfProperties.Normalize`): `Properties` is a `Dictionary<string, object>` whose `CreationDate`/`ModDate` values are parsed (`PdfDate`) into `DateTimeOffset`, so Verify's built-in date scrubbing renders them deterministically (`DateTimeOffset_1` etc.). Properties are read from the original bytes, before the byte-level pass zeroes them.
4242

43-
Both can be opted out of per verification, via a context key set by a `SettingsTask` extension and read in `Convert`:
44-
- `ExcludePdfDocument` — drops the `.verified.pdf` entirely, for producers whose bytes can never be made deterministic (Aspose.Cells embeds the machine's system fonts).
45-
- `SkipPdfNormalization` — keeps the `.verified.pdf` but snapshots the producer's own bytes, for producers that are already byte-deterministic. Worth knowing when changing this: `CanonicalizeXmp` is the pass that alters bytes even for such a producer, so toggling the setting on an existing suite shifts every stored `.verified.pdf` once.
43+
Both can be opted out of per verification:
44+
- `ExcludeTargets("pdf")`, Verify's own setting — drops the `.verified.pdf` entirely, for producers whose bytes can never be made deterministic (Aspose.Cells embeds the machine's system fonts). `Convert` checks `context.IsTargetExcluded("pdf")` and skips the normalization with it.
45+
- `SkipPdfNormalization` — keeps the `.verified.pdf` but snapshots the producer's own bytes, for producers that are already byte-deterministic. A context key set by a `SettingsTask` extension and read in `Convert`. Worth knowing when changing this: `CanonicalizeXmp` is the pass that alters bytes even for such a producer, so toggling the setting on an existing suite shifts every stored `.verified.pdf` once.
4646

4747
Key files:
48-
- **VerifyPDFium.cs** — initialization, the converter, and the two opt-out extensions
48+
- **VerifyPDFium.cs** — initialization, the converter, and the `SkipPdfNormalization` extension
4949
- **PdfProperties.cs** — projects the pdfium-reported property map to the scrubbable object map
5050
- **PdfDate.cs** — parses PDF date strings (`D:YYYYMMDD…`) to `DateTimeOffset`
51-
- **PdfInfo.cs** / **PageInfo.cs** — info shape for the snapshot (per-page width/height in points and text via `PdfPage.GetText()`)
51+
- **PageSize.cs** — a page's width and height in points, its `Info` in the info file. The rest of that file's shape is Verify's
5252

5353
The byte-level normalizer is **not** a type in this repo — it is `PdfNormalizer` from the [DeterministicPdf](https://github.com/SimonCropp/DeterministicPdf) package, which owns and tests the algorithm.
5454

‎readme.md‎

Lines changed: 98 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,12 @@ Extends [Verify](https://github.com/VerifyTests/Verify) to allow verification of
88

99
Verifying a `pdf` produces:
1010

11-
* A `.verified.txt` with the page count, per-page size (in PDF points) and extracted text, and document information dictionary entries (Title, Author, Producer, dates, etc).
12-
* The pdf itself as `.verified.pdf`. This can be omitted with [`ExcludePdfDocument`](#exclude-the-pdf-document).
11+
* A `.verified.txt` with the document information dictionary entries (Title, Author, Producer, dates, etc), the page count, and the size (in PDF points) and extracted text of each page.
12+
* The pdf itself as `.verified.pdf`. This can be omitted with [`ExcludeTargets`](#exclude-the-pdf-document).
1313
* A PNG render of every page as `#page_0001.verified.png`, `#page_0002.verified.png`, etc.
1414

15+
The page files are named, and the text placed, by Verify's [paged documents](https://github.com/VerifyTests/Verify/blob/main/docs/paged-documents.md) support, which every Verify plugin that splits a document into pages shares. So do the settings that [choose what is verified](#choosing-what-is-verified).
16+
1517
The non-deterministic fields of the pdf (the trailer `/ID`, the `/CreationDate` and `/ModDate`, and the equivalent XMP metadata) are neutralized so the same source document produces a byte-identical `.verified.pdf` across runs. A producer that already emits deterministic bytes can skip that work with [`SkipPdfNormalization`](#skip-pdf-normalization).
1618

1719
Rendering is provided by [Morph.PDFium](https://github.com/Papyrine/Morph.PDFium), which wraps the prebuilt PDFium binaries from [pdfium-binaries](https://github.com/bblanchon/pdfium-binaries) (Windows, Linux, and macOS). Rendering is deterministic for a given Morph.PDFium version: the same input produces byte-identical PNGs on every machine and OS, and no image library dependency is added.
@@ -55,25 +57,65 @@ public static void Initialize() =>
5557
`Initialize` optionally takes the render resolution: `VerifyPDFium.Initialize(dpi: 150)`. The default 96 dpi renders an A4 page at 794 x 1123.
5658

5759

58-
### Outputs
60+
### Choosing what is verified
61+
62+
What a pdf is split into is controlled by Verify's settings for [paged documents](https://github.com/VerifyTests/Verify/blob/main/docs/paged-documents.md). Anything left out is not produced at all (pages are not rendered, text is not extracted), so these also save work.
63+
64+
The text of each page is in the info file by default. `PageText` moves it to a `#page_0001.verified.txt` per page, or leaves it out with `PageTextPlacement.None`:
65+
66+
<!-- snippet: PageTextPerPage -->
67+
<a id='snippet-PageTextPerPage'></a>
68+
```cs
69+
[Test]
70+
public Task PageTextPerPage() =>
71+
VerifyFile("sample.pdf")
72+
.PageText(PageTextPlacement.PerPage);
73+
```
74+
<sup><a href='/src/Tests/Samples.cs#L35-L42' title='Snippet source file'>snippet source</a> | <a href='#snippet-PageTextPerPage' title='Start of snippet'>anchor</a></sup>
75+
<!-- endSnippet -->
5976

60-
`Initialize` optionally takes a `PdfiumOutputs` flags enum controlling which outputs each pdf is split into:
77+
`PagesToInclude` limits the pages that are rendered and read, to the first pages of a document or to those a delegate accepts. The pdf itself is still verified whole:
6178

62-
* `Png`: a rendered png per page.
63-
* `Text`: the extracted text of each page in the info file.
64-
* `None`: none of the above. Only the info and the source document are emitted.
65-
* `All`: both. The default.
79+
<!-- snippet: PagesToInclude -->
80+
<a id='snippet-PagesToInclude'></a>
81+
```cs
82+
[Test]
83+
public Task PagesToInclude() =>
84+
VerifyFile(ProjectFiles.multi_page_pdf.Path)
85+
.PagesToInclude(2)
86+
.ExcludeDerivedTargets("png");
87+
```
88+
<sup><a href='/src/Tests/Samples.cs#L44-L52' title='Snippet source file'>snippet source</a> | <a href='#snippet-PagesToInclude' title='Start of snippet'>anchor</a></sup>
89+
<!-- endSnippet -->
90+
91+
`ExcludeDerivedTargets("png")` leaves out the rendered pages, keeping the pdf and its text:
92+
93+
<!-- snippet: TextOnly -->
94+
<a id='snippet-TextOnly'></a>
95+
```cs
96+
[Test]
97+
public Task TextOnly() =>
98+
VerifyFile("sample.pdf")
99+
.ExcludeDerivedTargets("png");
100+
```
101+
<sup><a href='/src/Tests/Samples.cs#L54-L61' title='Snippet source file'>snippet source</a> | <a href='#snippet-TextOnly' title='Start of snippet'>anchor</a></sup>
102+
<!-- endSnippet -->
66103

67-
Omitted outputs are not produced at all (pages are not rendered, text is not extracted), so this also saves work. The pdf document itself is not controlled by this option; use `VerifierSettings.ExcludeTargets("pdf")` for that.
104+
Each can also be set for every test, on `VerifierSettings`:
68105

69106
<!-- snippet: InitializeOutputs -->
70107
<a id='snippet-InitializeOutputs'></a>
71108
```cs
72109
[ModuleInitializer]
73-
public static void Initialize() =>
74-
VerifyPDFium.Initialize(outputs: PdfiumOutputs.Png);
110+
public static void Initialize()
111+
{
112+
VerifyPDFium.Initialize();
113+
114+
// For every test: no text, so only the pages are verified
115+
VerifierSettings.PageText(PageTextPlacement.None);
116+
}
75117
```
76-
<sup><a href='/src/StaticSettingsTests/ModuleInitializer.cs#L3-L9' title='Snippet source file'>snippet source</a> | <a href='#snippet-InitializeOutputs' title='Start of snippet'>anchor</a></sup>
118+
<sup><a href='/src/StaticSettingsTests/ModuleInitializer.cs#L3-L14' title='Snippet source file'>snippet source</a> | <a href='#snippet-InitializeOutputs' title='Start of snippet'>anchor</a></sup>
77119
<!-- endSnippet -->
78120

79121

@@ -108,15 +150,15 @@ public Task VerifyPdfStream()
108150

109151
### Exclude the pdf document
110152

111-
Some pdf producers embed non-deterministic bytes that cannot be neutralized. For example [Aspose.Cells](https://products.aspose.com/cells/) always embeds the machine's system fonts (it has no way to restrict font resolution to a bundled set), so the pdf bytes differ from one machine to the next even for the same input. `ExcludePdfDocument` drops the `.verified.pdf` from the snapshot for that verification, while still verifying the deterministic rendered pages and info file:
153+
Some pdf producers embed non-deterministic bytes that cannot be neutralized. For example [Aspose.Cells](https://products.aspose.com/cells/) always embeds the machine's system fonts (it has no way to restrict font resolution to a bundled set), so the pdf bytes differ from one machine to the next even for the same input. Verify's `ExcludeTargets("pdf")` drops the `.verified.pdf` from the snapshot for that verification, while still verifying the deterministic rendered pages and info file. `VerifierSettings.ExcludeTargets("pdf")` does the same for every test:
112154

113155
<!-- snippet: ExcludePdfDocument -->
114156
<a id='snippet-ExcludePdfDocument'></a>
115157
```cs
116158
[Test]
117159
public Task ExcludePdfDocument() =>
118160
VerifyFile("sample.pdf")
119-
.ExcludePdfDocument();
161+
.ExcludeTargets("pdf");
120162
```
121163
<sup><a href='/src/Tests/Samples.cs#L26-L33' title='Snippet source file'>snippet source</a> | <a href='#snippet-ExcludePdfDocument' title='Start of snippet'>anchor</a></sup>
122164
<!-- endSnippet -->
@@ -134,14 +176,55 @@ public Task SkipPdfNormalization() =>
134176
VerifyFile("sample.pdf")
135177
.SkipPdfNormalization();
136178
```
137-
<sup><a href='/src/Tests/Samples.cs#L35-L42' title='Snippet source file'>snippet source</a> | <a href='#snippet-SkipPdfNormalization' title='Start of snippet'>anchor</a></sup>
179+
<sup><a href='/src/Tests/Samples.cs#L63-L70' title='Snippet source file'>snippet source</a> | <a href='#snippet-SkipPdfNormalization' title='Start of snippet'>anchor</a></sup>
138180
<!-- endSnippet -->
139181

140182
Only skip it when the producer is genuinely deterministic. Without normalization a freshly generated pdf carries a wall-clock `/CreationDate` and a fresh `/ID`, so the snapshot differs on every run.
141183

142184
The XMP canonicalization is worth calling out, because it is the pass that changes bytes even for an already-deterministic producer: it collapses the packet's whitespace. Turning this setting on for an existing suite therefore shifts every stored `.verified.pdf` once, without anything about the documents having changed.
143185

144186

187+
## Reviewing changes
188+
189+
A change to a pdf is a change to several files: the pdf, its info file, and every page. Verify tells the diff tool that the pages and the info file were derived from the pdf, and [DiffEngineViewer](https://github.com/VerifyTests/DiffEngine/blob/main/docs/viewer.md#files-derived-from-a-document), which draws a pdf's pages itself, shows them as one row and accepts them together. Other diff tools are given each file, as before.
190+
191+
192+
## Migrating from 1.x
193+
194+
Version 2 moves to the paged document support in Verify 33.3. The file names are unchanged. What differs:
195+
196+
| 1.x | 2.x |
197+
| --- | --- |
198+
| `Initialize(outputs: PdfiumOutputs.Png)` | `VerifierSettings.PageText(PageTextPlacement.None)` |
199+
| `Initialize(outputs: PdfiumOutputs.Text)` | `VerifierSettings.ExcludeDerivedTargets("png")` |
200+
| `Initialize(outputs: PdfiumOutputs.None)` | Both of the above |
201+
| `.ExcludePdfDocument()` | `.ExcludeTargets("pdf")` |
202+
203+
The info file has the shape every paged document has: the document properties under `Document`, and each page as its `Number`, its size under `Info`, and its `Text`.
204+
205+
```
206+
{
207+
PageCount: 1, {
208+
Properties: { Document: {
209+
Producer: PDFsharp 6.2.4 Producer: PDFsharp 6.2.4
210+
}, },
211+
Pages: [ PageCount: 1,
212+
{ Pages: [
213+
Width: 612.0, {
214+
Height: 792.0, Number: 1,
215+
Text: Hello, World! Info: {
216+
} Width: 612.0,
217+
] Height: 792.0
218+
} },
219+
Text: Hello, World!
220+
}
221+
]
222+
}
223+
```
224+
225+
So every `.verified.txt` changes once, and nothing else does.
226+
227+
145228
## Icon
146229

147230
[PDF](https://thenounproject.com/icon/pdf-7564953//) designed by [Meilia](https://thenounproject.com/creator/meilia1/) from [The Noun Project](https://thenounproject.com).

‎src/Directory.Build.props‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
<Project>
33
<PropertyGroup>
44
<NoWarn>CS1591;CS0649;CA1416;NU1608;NU1109</NoWarn>
5-
<Version>1.5.0</Version>
5+
<Version>2.0.0</Version>
66
<LangVersion>14.0</LangVersion>
77
<AssemblyVersion>1.0.0</AssemblyVersion>
88
<PackageTags>PDF, PDFium, Verify</PackageTags>

‎src/Directory.Packages.props‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,11 @@
1010
<PackageVersion Include="ProjectDefaults" Version="1.0.182" />
1111
<PackageVersion Include="ProjectFiles" Version="1.6.0" />
1212
<PackageVersion Include="Sbom" Version="0.3.1" />
13-
<PackageVersion Include="Verify" Version="33.2.0" />
14-
<PackageVersion Include="Verify.TUnit" Version="33.2.0" />
13+
<PackageVersion Include="Verify" Version="33.3.0" />
14+
<PackageVersion Include="Verify.TUnit" Version="33.3.0" />
1515
<PackageVersion Include="TUnit" Version="1.72.16" />
1616
<PackageVersion Include="Argon" Version="0.37.0" />
17-
<PackageVersion Include="DiffEngine" Version="20.6.0" />
17+
<PackageVersion Include="DiffEngine" Version="20.7.0" />
1818
<PackageVersion Include="EmptyFiles" Version="8.20.0" />
1919
<PackageVersion Include="SimpleInfoName" Version="3.2.0" />
2020
</ItemGroup>

‎src/StaticSettingsTests/ModuleInitializer.cs‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,13 @@ public static class ModuleInitializer
33
#region InitializeOutputs
44

55
[ModuleInitializer]
6-
public static void Initialize() =>
7-
VerifyPDFium.Initialize(outputs: PdfiumOutputs.Png);
6+
public static void Initialize()
7+
{
8+
VerifyPDFium.Initialize();
9+
10+
// For every test: no text, so only the pages are verified
11+
VerifierSettings.PageText(PageTextPlacement.None);
12+
}
813

914
#endregion
1015

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,18 @@
11
{
2-
PageCount: 1,
3-
Properties: {
2+
Document: {
43
CreationDate: DateTimeOffset_1,
54
Creator: Morph,
65
ModDate: DateTimeOffset_1,
76
Producer: PDFsharp 6.2.4
87
},
8+
PageCount: 1,
99
Pages: [
1010
{
11-
Width: 612.0,
12-
Height: 792.0
11+
Number: 1,
12+
Info: {
13+
Width: 612.0,
14+
Height: 792.0
15+
}
1316
}
1417
]
1518
}

‎src/Tests/Samples.ExcludePdfDocument.verified.txt‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,18 @@
11
{
2-
PageCount: 1,
3-
Properties: {
2+
Document: {
43
CreationDate: DateTimeOffset_1,
54
Creator: Morph,
65
ModDate: DateTimeOffset_1,
76
Producer: PDFsharp 6.2.4
87
},
8+
PageCount: 1,
99
Pages: [
1010
{
11-
Width: 612.0,
12-
Height: 792.0,
11+
Number: 1,
12+
Info: {
13+
Width: 612.0,
14+
Height: 792.0
15+
},
1316
Text: Hello, World! This is a simple paragraph.
1417
}
1518
]

0 commit comments

Comments
 (0)