authorgravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-12 13:22:34-07:00
committergravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-12 17:00:32-07:00
logcb4842d4e6047a60f7c99dcbf033c4fe366e8b8f
tree14fc9a899da3b9e995cd1fc1cbcbda7b49af25ce
parent59c8648048632812f85fd595c05aa9720ad2a051
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

feat: write pictures and attachments through the page model

Inserted pictures and files are paragraph content in the page model (ParagraphContent::Image and ParagraphContent::Attachment, read with their payloads, sizes and the attachment preview icon) and the writer lowers a new one the way OneNote stores it: the payload goes into the section file-data store list as a file-data store object, a file-data declaration names it by identity and extension (a picture container, or an embedded-file object for attachments), and a picture or attachment object with the displayed size, name and the flags every native one carries becomes the paragraph content. The DPAPI-protected per-user blob OneNote adds to pictures is omitted, and an attachment preview icon is optional; OneNote accepts both. Stored pictures and attachments stay fixed and can be removed with their paragraph. The transaction writer gains payload embedding: write_revisions_with_payloads validates file declarations, builds the file-data store objects and appends their nodes to the existing file-data store list or creates it. The squash pass carries file objects through with their jcid and re-embeds payloads the source lacks. The canvas treats picture and attachment paragraphs as read-only content. corpus/picture-edit and corpus/attachment-edit (without and with a preview icon) are the native gates, cold-read by OneNote 2010 with the exact payload bytes; tools/test_picture_edit.py and tools/test_attachment_edit.py check them without a VM. Assisted-by: claude-fable-5.1

66 files changed, 1768 insertions(+), 48 deletions(-)

corpus/attachment-edit/README.md created+23
...@@ -0,0 +1,23 @@
1# Rust attachment authoring
2
3`plain/candidate` and `icon/candidate` are the page writer's output for
4`an_attachment_inserted_on_a_fresh_page_reads_back_and_can_be_removed` in
5`crates/onestore/tests/page_attachments.rs`: on a section created in Rust, a
6paragraph between two text paragraphs becomes an inserted file
7(`notes 🦀.txt`, 28 bytes), stored the way OneNote stores one
8(`corpus/native/cold-05-06-attachment`): the payload embedded in the
9section's file-data store list, an embedded-file object declaring it by
10identity and extension, and an attachment object with the file name, the
1124 pt icon size and the flags every native attachment carries. `icon/` also
12embeds the native fixture's 724-byte icon PNG as the preview picture;
13`plain/` omits it, which OneNote accepts (it renders its own icon).
14OneNote's integrity check refuses an attachment object without
15`0x1c001d61` (twenty bytes: 16, 1, zeros), so the writer always stores it.
16
17`*/cold` are fresh OneNote 2010 reads: the file reads back as an
18`InsertedFile` with the preferred name and the exact payload bytes, between
19the two text paragraphs. `tools/test_attachment_edit.py` checks both captures
20without a VM. Regenerate with `ONESTORE_ATTACHMENT_EXPORT` set to a directory
21while running the test (it writes `plain/` and `icon/`), then cold-open each
22with `tools/native_runner.py OUTPUT/plain COLD --expected-pages 1
23--collect-notebook`.
corpus/attachment-edit/icon/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/attachment-edit/icon/candidate/Open Notebook.onetoc2 differ
corpus/attachment-edit/icon/candidate/files.one created
Binary files /dev/null and b/corpus/attachment-edit/icon/candidate/files.one differ
corpus/attachment-edit/icon/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-5534DCBB -ExpectedPages 1", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/attachment-edit/icon/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-5534dcbb", "hostname": "ONE-M6-5534DCBB"}
corpus/attachment-edit/icon/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/attachment-edit/icon/cold/notebook/Open Notebook.onetoc2 differ
corpus/attachment-edit/icon/cold/notebook/files.one created
Binary files /dev/null and b/corpus/attachment-edit/icon/cold/notebook/files.one differ
corpus/attachment-edit/icon/cold/read/b20a14e044e33eac39b4665e7d79813b8a3672dbbd27ed9d2e4ac866c13ec4b6.attachment created+1
...@@ -0,0 +1 @@
1Rust wrote this attachment.
corpus/attachment-edit/icon/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-5534DCBB",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/attachment-edit/icon/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{6DDE4690-D1C7-4466-8BCC-7708FB0B3EB1}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-12T20:56:45.000Z" color="#B49EDE"><one:Section name="files" ID="{F310D0B9-ED06-021A-30B2-A3BC2CFAD523}{1}{B0}" path="C:\one-tests\runs\capture\notebook\files.one" lastModifiedTime="2026-09-12T20:56:45.000Z" color="#8AA8E4"><one:Page ID="{650CBE9C-812A-0909-10A6-1E17B31C633C}{1}{B0}" name="Before the file" dateTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:56:36.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/attachment-edit/icon/cold/read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{650CBE9C-812A-0909-10A6-1E17B31C633C}{1}{B0}" name="Before the file" dateTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:56:36.000Z" pageLevel="1" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-12T20:54:55.000Z" objectID="{F310D0B9-ED06-021A-30B2-A3BC2CFAD523}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="130.4619598388672" height="89.85542297363281"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:54:55.000Z" objectID="{F310D0B9-ED06-021A-30B2-A3BC2CFAD523}{24}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Before the file]]></one:T></one:OE><one:OE author="Attachment author" lastModifiedBy="Attachment author" creationTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:54:55.000Z" objectID="{7FE89054-6937-03E5-03EB-E0B7A346CCCA}{0}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\773d420c-04e1-4aa8-a774-c588ccac5d00.txt" preferredName="notes 🦀.txt"/></one:OE><one:OE author="Attachment author" lastModifiedBy="Attachment author" creationTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:54:55.000Z" objectID="{F269A078-0B55-0F23-3116-14239F4BB309}{0}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[After the file]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/attachment-edit/icon/cold/read/payloads.json created+10
...@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "b20a14e044e33eac39b4665e7d79813b8a3672dbbd27ed9d2e4ac866c13ec4b6",
4 "name": "notes 🦀.txt",
5 "object": "{7FE89054-6937-03E5-03EB-E0B7A346CCCA}{0}{B0}",
6 "kind": "InsertedFile",
7 "page": "{650CBE9C-812A-0909-10A6-1E17B31C633C}{1}{B0}",
8 "bytes": 28
9 }
10]
\ No newline at end of file
corpus/attachment-edit/icon/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/m4/files/icon",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/attachment-edit/icon/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/attachment-edit/icon/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/attachment-edit/icon/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 1604,
5 "sha256": "98f31fe15497a403a6e2896c8ae2e6566824c0971e541187046a1581c4612721",
6 "mtime_ns": 1789246495204529692
7 },
8 {
9 "path": "files.one",
10 "bytes": 8288,
11 "sha256": "473e6527a585dc98d178343cf53ba9e08354b3d4d53a4f5ce984c0036ecfd76a",
12 "mtime_ns": 1789246495204285026
13 }
14]
corpus/attachment-edit/icon/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/attachment-edit/plain/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/attachment-edit/plain/candidate/Open Notebook.onetoc2 differ
corpus/attachment-edit/plain/candidate/files.one created
Binary files /dev/null and b/corpus/attachment-edit/plain/candidate/files.one differ
corpus/attachment-edit/plain/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-B7D04704 -ExpectedPages 1", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/attachment-edit/plain/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-b7d04704", "hostname": "ONE-M6-B7D04704"}
corpus/attachment-edit/plain/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/attachment-edit/plain/cold/notebook/Open Notebook.onetoc2 differ
corpus/attachment-edit/plain/cold/notebook/files.one created
Binary files /dev/null and b/corpus/attachment-edit/plain/cold/notebook/files.one differ
corpus/attachment-edit/plain/cold/read/b20a14e044e33eac39b4665e7d79813b8a3672dbbd27ed9d2e4ac866c13ec4b6.attachment created+1
...@@ -0,0 +1 @@
1Rust wrote this attachment.
corpus/attachment-edit/plain/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-B7D04704",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/attachment-edit/plain/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{C7752460-7316-40F2-B55D-70CCAF39494B}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-12T20:57:03.000Z" color="#B49EDE"><one:Section name="files" ID="{9EAD6438-F285-0643-3547-49A641DA7552}{1}{B0}" path="C:\one-tests\runs\capture\notebook\files.one" lastModifiedTime="2026-09-12T20:57:03.000Z" color="#8AA8E4"><one:Page ID="{62772D92-5840-014B-3B34-46BA7FAC9472}{1}{B0}" name="Before the file" dateTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:57:03.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/attachment-edit/plain/cold/read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{62772D92-5840-014B-3B34-46BA7FAC9472}{1}{B0}" name="Before the file" dateTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:57:03.000Z" pageLevel="1" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-12T20:54:55.000Z" objectID="{9EAD6438-F285-0643-3547-49A641DA7552}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="130.4619598388672" height="89.85542297363281"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:54:55.000Z" objectID="{9EAD6438-F285-0643-3547-49A641DA7552}{24}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Before the file]]></one:T></one:OE><one:OE author="Attachment author" lastModifiedBy="Attachment author" creationTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:54:55.000Z" objectID="{6DE415DD-4090-0E54-15BB-41A8635DE4EB}{0}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\6b5d60fe-566b-4a4b-aef1-fd37d628779c.txt" preferredName="notes 🦀.txt"/></one:OE><one:OE author="Attachment author" lastModifiedBy="Attachment author" creationTime="2026-09-12T20:54:55.000Z" lastModifiedTime="2026-09-12T20:54:55.000Z" objectID="{DEA34064-D4FD-015B-0A30-243BCAF6F90D}{0}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[After the file]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/attachment-edit/plain/cold/read/payloads.json created+10
...@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "b20a14e044e33eac39b4665e7d79813b8a3672dbbd27ed9d2e4ac866c13ec4b6",
4 "name": "notes 🦀.txt",
5 "object": "{6DE415DD-4090-0E54-15BB-41A8635DE4EB}{0}{B0}",
6 "kind": "InsertedFile",
7 "page": "{62772D92-5840-014B-3B34-46BA7FAC9472}{1}{B0}",
8 "bytes": 28
9 }
10]
\ No newline at end of file
corpus/attachment-edit/plain/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/m4/files/plain",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/attachment-edit/plain/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/attachment-edit/plain/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/attachment-edit/plain/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 1604,
5 "sha256": "a0aa6bb1b80a7e9dd300ffef5d547ea08b685b53cbc91abde9829ea8b6054489",
6 "mtime_ns": 1789246495204137318
7 },
8 {
9 "path": "files.one",
10 "bytes": 7160,
11 "sha256": "cdb9b28f8bd716d612675321ad8bb752349c118735331e9d323c9b8e876aea73",
12 "mtime_ns": 1789246495203833985
13 }
14]
corpus/attachment-edit/plain/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/picture-edit/README.md created+20
...@@ -0,0 +1,20 @@
1# Rust picture authoring
2
3`candidate/` is the page writer's output for
4`a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed` in
5`crates/onestore/tests/page_images.rs`: on a section created in Rust, a
6paragraph between two text paragraphs becomes a picture, stored the way
7OneNote stores an inserted picture (`corpus/native/cold-05-05-image`): the
8PNG embedded in the section's file-data store list as a file-data store
9object, a picture-container file object declaring it by identity and
10extension, and a picture object with the displayed size that the paragraph
11holds as content. OneNote's own picture objects also carry a DPAPI-protected
12blob (`0x1c001dfb`) that only the authoring Windows user can decrypt; the
13writer omits it.
14
15`cold/` is a fresh OneNote 2010 read: the image reads back as `format="png"`
16with the exact payload bytes, between the two text paragraphs.
17`tools/test_picture_edit.py` checks this without a VM. Regenerate with
18`ONESTORE_IMAGE_EXPORT` set to a new absolute directory while running the
19test, then cold-open it with `tools/native_runner.py OUTPUT COLD
20--expected-pages 1 --collect-notebook`.
corpus/picture-edit/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/picture-edit/candidate/Open Notebook.onetoc2 differ
corpus/picture-edit/candidate/pictures.one created
Binary files /dev/null and b/corpus/picture-edit/candidate/pictures.one differ
corpus/picture-edit/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-A5A8705B -ExpectedPages 1", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/picture-edit/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-a5a8705b", "hostname": "ONE-M6-A5A8705B"}
corpus/picture-edit/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/picture-edit/cold/notebook/Open Notebook.onetoc2 differ
corpus/picture-edit/cold/notebook/pictures.one created
Binary files /dev/null and b/corpus/picture-edit/cold/notebook/pictures.one differ
corpus/picture-edit/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-A5A8705B",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/picture-edit/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{BB903F04-A992-47A1-B747-4D7C2D151005}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-12T20:08:56.000Z" color="#B49EDE"><one:Section name="pictures" ID="{54DE5097-F4BE-0BCC-0392-19FB4210AAAD}{1}{B0}" path="C:\one-tests\runs\capture\notebook\pictures.one" lastModifiedTime="2026-09-12T20:08:56.000Z" color="#8AA8E4"><one:Page ID="{D3A0AD9C-D12D-02B2-1683-4D32345076A5}{1}{B0}" name="Before the picture" dateTime="2026-09-12T20:07:15.000Z" lastModifiedTime="2026-09-12T20:08:56.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/picture-edit/cold/read/page-000.xml created+4
...@@ -0,0 +1,4 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{D3A0AD9C-D12D-02B2-1683-4D32345076A5}{1}{B0}" name="Before the picture" dateTime="2026-09-12T20:07:15.000Z" lastModifiedTime="2026-09-12T20:08:56.000Z" pageLevel="1" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-12T20:07:15.000Z" objectID="{54DE5097-F4BE-0BCC-0392-19FB4210AAAD}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="148.3100738525391" height="27.60543251037598"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T20:07:15.000Z" lastModifiedTime="2026-09-12T20:07:15.000Z" objectID="{54DE5097-F4BE-0BCC-0392-19FB4210AAAD}{24}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Before the picture]]></one:T></one:OE><one:OE author="Picture author" lastModifiedBy="Picture author" creationTime="2026-09-12T20:07:15.000Z" lastModifiedTime="2026-09-12T20:07:15.000Z" objectID="{496D54CF-B74C-091A-2D00-F268C53481D1}{0}{B0}" alignment="left"><one:Image format="png"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVQI12P4z8AAAAMBAQAY3Y2w
3AAAAAElFTkSuQmCC
4</one:Data></one:Image></one:OE><one:OE author="Picture author" lastModifiedBy="Picture author" creationTime="2026-09-12T20:07:15.000Z" lastModifiedTime="2026-09-12T20:07:15.000Z" objectID="{39F25FF2-3F08-07F1-06E1-F7F0075DE8F1}{0}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[After the picture]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/picture-edit/cold/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/picture-edit/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/m4/pictures",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/picture-edit/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/picture-edit/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/picture-edit/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 1604,
5 "sha256": "0eddd691b52d26bac222cf7e52add3c9909446647b452fb193f03d49e273426a",
6 "mtime_ns": 1789243635388246438
7 },
8 {
9 "path": "pictures.one",
10 "bytes": 7064,
11 "sha256": "3076880e02cf74eb85400852c133060ab8ed50383c6b9df8a07bde8389891a70",
12 "mtime_ns": 1789243635387879438
13 }
14]
corpus/picture-edit/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
crates/canvas/src/document.rs+5-1
...@@ -153,7 +153,11 @@ pub(crate) fn validate_nodes(...@@ -153,7 +153,11 @@ pub(crate) fn validate_nodes(
153 return Err(EditError::InvalidStructure);153 return Err(EditError::InvalidStructure);
154 }154 }
155 }155 }
156 ParagraphContent::Unsupported(_) => return Err(EditError::UnsupportedContent),156 ParagraphContent::Image(_)
157 | ParagraphContent::Attachment(_)
158 | ParagraphContent::Unsupported(_) => {
159 return Err(EditError::UnsupportedContent);
160 }
157 ParagraphContent::Table(table) => {161 ParagraphContent::Table(table) => {
158 if !ids.insert(table.id)162 if !ids.insert(table.id)
159 || table.rows.is_empty()163 || table.rows.is_empty()
crates/canvas/src/editor.rs+10-3
...@@ -455,7 +455,9 @@ impl TextOutline {...@@ -455,7 +455,9 @@ impl TextOutline {
455 pending.push(visible_paragraphs(cell.paragraphs.iter()));455 pending.push(visible_paragraphs(cell.paragraphs.iter()));
456 }456 }
457 }457 }
458 onestore::page::ParagraphContent::Unsupported(_) => {458 onestore::page::ParagraphContent::Image(_)
459 | onestore::page::ParagraphContent::Attachment(_)
460 | onestore::page::ParagraphContent::Unsupported(_) => {
459 return Err(EditError::UnsupportedContent);461 return Err(EditError::UnsupportedContent);
460 }462 }
461 }463 }
...@@ -504,7 +506,9 @@ impl TextOutline {...@@ -504,7 +506,9 @@ impl TextOutline {
504 .rect[1]506 .rect[1]
505 - 1.86507 - 1.86
506 }508 }
507 onestore::page::ParagraphContent::Unsupported(_) => {509 onestore::page::ParagraphContent::Image(_)
510 | onestore::page::ParagraphContent::Attachment(_)
511 | onestore::page::ParagraphContent::Unsupported(_) => {
508 return Err(EditError::UnsupportedContent);512 return Err(EditError::UnsupportedContent);
509 }513 }
510 };514 };
...@@ -1690,7 +1694,9 @@ impl CanvasEditor {...@@ -1690,7 +1694,9 @@ impl CanvasEditor {
1690 .ok_or(EditError::InvalidStructure)?;1694 .ok_or(EditError::InvalidStructure)?;
1691 f64::from(cell.rect[3]) + 1.681695 f64::from(cell.rect[3]) + 1.68
1692 }1696 }
1693 onestore::page::ParagraphContent::Unsupported(_) => {1697 onestore::page::ParagraphContent::Image(_)
1698 | onestore::page::ParagraphContent::Attachment(_)
1699 | onestore::page::ParagraphContent::Unsupported(_) => {
1694 return Err(EditError::UnsupportedContent.into());1700 return Err(EditError::UnsupportedContent.into());
1695 }1701 }
1696 } + f64::from(outline.origin()[1]);1702 } + f64::from(outline.origin()[1]);
...@@ -4348,6 +4354,7 @@ mod tests {...@@ -4348,6 +4354,7 @@ mod tests {
4348 (120.0, 80.0, true),4354 (120.0, 80.0, true),
4349 ]4355 ]
4350 .map(|(x, y, background)| Image {4356 .map(|(x, y, background)| Image {
4357 size: None,
4351 id: onestore::page::text::new_id().unwrap(),4358 id: onestore::page::text::new_id().unwrap(),
4352 layout: onestore::document::Layout {4359 layout: onestore::document::Layout {
4353 x: Some(x),4360 x: Some(x),
crates/canvas/src/gpu/page.rs+4
...@@ -907,6 +907,7 @@ mod tests {...@@ -907,6 +907,7 @@ mod tests {
907 (120.0, 60.0, false),907 (120.0, 60.0, false),
908 ] {908 ] {
909 page.objects.push(PageObject::Image(Image {909 page.objects.push(PageObject::Image(Image {
910 size: None,
910 id: onestore::page::text::new_id().unwrap(),911 id: onestore::page::text::new_id().unwrap(),
911 bytes: Some(image_bytes.clone()),912 bytes: Some(image_bytes.clone()),
912 layout: Layout {913 layout: Layout {
...@@ -1230,6 +1231,7 @@ mod tests {...@@ -1230,6 +1231,7 @@ mod tests {
1230 unsupported: vec![unknown.clone()],1231 unsupported: vec![unknown.clone()],
1231 }),1232 }),
1232 PageObject::Image(Image {1233 PageObject::Image(Image {
1234 size: None,
1233 id: ExGuid::default(),1235 id: ExGuid::default(),
1234 layout: Layout {1236 layout: Layout {
1235 y: Some(150.0),1237 y: Some(150.0),
...@@ -1388,6 +1390,7 @@ mod tests {...@@ -1388,6 +1390,7 @@ mod tests {
1388 objects: vec![1390 objects: vec![
1389 outline(2.0, "first"),1391 outline(2.0, "first"),
1390 PageObject::Image(Image {1392 PageObject::Image(Image {
1393 size: None,
1391 id: ExGuid::default(),1394 id: ExGuid::default(),
1392 layout: Layout {1395 layout: Layout {
1393 x: Some(5.0),1396 x: Some(5.0),
...@@ -1506,6 +1509,7 @@ mod tests {...@@ -1506,6 +1509,7 @@ mod tests {
1506 margin_origin: [0.0; 2],1509 margin_origin: [0.0; 2],
1507 definitions: BTreeMap::new(),1510 definitions: BTreeMap::new(),
1508 objects: vec![PageObject::Image(Image {1511 objects: vec![PageObject::Image(Image {
1512 size: None,
1509 id: ExGuid::default(),1513 id: ExGuid::default(),
1510 layout: Layout {1514 layout: Layout {
1511 x: Some(2.0),1515 x: Some(2.0),
crates/canvas/src/outline.rs+5-1
...@@ -374,7 +374,11 @@ impl OutlineLayout {...@@ -374,7 +374,11 @@ impl OutlineLayout {
374 bottom += f64::from(child.size[1]);374 bottom += f64::from(child.size[1]);
375 result.append(child, [x, y]);375 result.append(child, [x, y]);
376 }376 }
377 ParagraphContent::Unsupported(_) => return Err(LayoutError::UnsupportedContent),377 ParagraphContent::Image(_)
378 | ParagraphContent::Attachment(_)
379 | ParagraphContent::Unsupported(_) => {
380 return Err(LayoutError::UnsupportedContent);
381 }
378 }382 }
379 if !(bottom as f32).is_finite() || !result.size[0].is_finite() {383 if !(bottom as f32).is_finite() || !result.size[0].is_finite() {
380 return Err(LayoutError::InvalidSpacing);384 return Err(LayoutError::InvalidSpacing);
crates/notebook/README.md+1-1
...@@ -14,7 +14,7 @@ diffs the supplied model against the page stored in the supplied local snapshot,...@@ -14,7 +14,7 @@ diffs the supplied model against the page stored in the supplied local snapshot,
14writes the difference into the working image and queues one14writes the difference into the working image and queues one
15`Operation::Page(PageIntent { before, after, author })`; `before` is the page the15`Operation::Page(PageIntent { before, after, author })`; `before` is the page the
16edit started from and is the precondition reconciliation checks. Text, styles,16edit started from and is the precondition reconciliation checks. Text, styles,
17paragraph formatting, hyperlinks, bullets, numbering, note tags, table rows and columns, paragraph structure,17paragraph formatting, hyperlinks, bullets, numbering, note tags, table rows and columns, pictures and attachments, paragraph structure,
18outline layout, insertions and deletions are all differences between `before` and18outline layout, insertions and deletions are all differences between `before` and
19`after`; the library never sees editor operations.19`after`; the library never sees editor operations.
2020
crates/onestore/README.md+1-1
...@@ -57,7 +57,7 @@ harness also accepts `--client-profile release`....@@ -57,7 +57,7 @@ harness also accepts `--client-profile release`.
57| `ParagraphJoin`, `PreparedEdit::join` | Join adjacent ordinary text while preserving inherited character styles and native text-identity rules |57| `ParagraphJoin`, `PreparedEdit::join` | Join adjacent ordinary text while preserving inherited character styles and native text-identity rules |
58| `TextAttribute`, `PreparedEdit::format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs |58| `TextAttribute`, `PreparedEdit::format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs |
59| `OutlineEdit`, `PreparedEdit::outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content |59| `OutlineEdit`, `PreparedEdit::outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content |
60| `PreparedEdit::page` | Publish an edited `page::Page` as one revision per changed space: text, character and paragraph formatting, hyperlinks, bullets and numbering, note tags, table rows and columns, paragraph insertion/split/join/move/deletion, outline insertion/deletion/position/width and saved collapse state, lowered onto the typed writers with the model's identities |60| `PreparedEdit::page` | Publish an edited `page::Page` as one revision per changed space: text, character and paragraph formatting, hyperlinks, bullets and numbering, note tags, table rows and columns, inserted pictures and attachments, paragraph insertion/split/join/move/deletion, outline insertion/deletion/position/width and saved collapse state, lowered onto the typed writers with the model's identities |
61| `TreeEdit`, `PreparedEdit::tree` | Move or delete a subtree on one page, normalize surviving containers, and replace an emptied table cell's paragraph atomically |61| `TreeEdit`, `PreparedEdit::tree` | Move or delete a subtree on one page, normalize surviving containers, and replace an emptied table cell's paragraph atomically |
62| `PreparedEdit::commit`, `PreparedEdit::commit_file` | Publish the exact prepared image under caller-held exclusion or the conservative filesystem adapter |62| `PreparedEdit::commit`, `PreparedEdit::commit_file` | Publish the exact prepared image under caller-held exclusion or the conservative filesystem adapter |
63| `read_file` | Read a snapshot under whole-file exclusion |63| `read_file` | Read a snapshot under whole-file exclusion |
crates/onestore/src/document.rs+5
...@@ -374,6 +374,9 @@ pub enum Kind<'a> {...@@ -374,6 +374,9 @@ pub enum Kind<'a> {
374 source_path: Option<String>,374 source_path: Option<String>,
375 recording_id: Option<[u8; 16]>,375 recording_id: Option<[u8; 16]>,
376 recording_type: Option<u32>,376 recording_type: Option<u32>,
377 /// Displayed icon width and height in points.
378 icon_width: Option<f32>,
379 icon_height: Option<f32>,
377 },380 },
378 File {381 File {
379 reference: FileDataReference,382 reference: FileDataReference,
...@@ -1161,6 +1164,8 @@ impl<'a> Element<'a> {...@@ -1161,6 +1164,8 @@ impl<'a> Element<'a> {
1161 source_path: f.text(0x1c001d9d)?,1164 source_path: f.text(0x1c001d9d)?,
1162 recording_id: f.fixed(0x1c001c97)?,1165 recording_id: f.fixed(0x1c001c97)?,
1163 recording_type: f.u32(0x14001d24)?,1166 recording_type: f.u32(0x14001d24)?,
1167 icon_width: f.float(0x140034cd, 36.0)?,
1168 icon_height: f.float(0x140034ce, 36.0)?,
1164 },1169 },
1165 0x120001 => Kind::Author {1170 0x120001 => Kind::Author {
1166 name: f.text(0x1c001d75)?,1171 name: f.text(0x1c001d75)?,
crates/onestore/src/page/mod.rs+126-26
...@@ -109,9 +109,81 @@ pub struct PageParagraph {...@@ -109,9 +109,81 @@ pub struct PageParagraph {
109pub enum ParagraphContent {109pub enum ParagraphContent {
110 Text(TextObject),110 Text(TextObject),
111 Table(Table),111 Table(Table),
112 /// A picture inside the paragraph, as OneNote inserts pictures into outlines.
113 Image(Image),
114 /// An embedded file, as OneNote inserts attachments into outlines.
115 Attachment(Attachment),
112 Unsupported(Unsupported),116 Unsupported(Unsupported),
113}117}
114118
119/// Payload bytes are identified by `id`, so equality and serialization leave them out.
120#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)]
121pub struct Attachment {
122 pub id: ExGuid,
123 /// The name OneNote shows and saves the file under.
124 pub filename: String,
125 /// Where the file was inserted from, when recorded.
126 pub source_path: Option<String>,
127 /// The icon OneNote rendered for the file, when present.
128 pub size: Option<[f32; 2]>,
129 #[serde(skip)]
130 pub bytes: Option<Arc<[u8]>>,
131 #[serde(skip)]
132 pub preview: Option<Arc<[u8]>>,
133}
134
135impl PartialEq for Attachment {
136 fn eq(&self, other: &Self) -> bool {
137 self.id == other.id
138 && self.filename == other.filename
139 && self.source_path == other.source_path
140 && self.size == other.size
141 }
142}
143
144impl Attachment {
145 fn read(
146 revision: &Revision<'_>,
147 id: ExGuid,
148 node: &crate::document::Element<'_>,
149 ) -> Result<Self, Error> {
150 let invalid = |message| Error { offset: 0, message };
151 let Kind::Attachment {
152 container,
153 preview,
154 filename,
155 source_path,
156 icon_width,
157 icon_height,
158 ..
159 } = &node.kind
160 else {
161 unreachable!()
162 };
163 let payload = |target: &Option<ExGuid>| -> Result<Option<Arc<[u8]>>, Error> {
164 let Some(target) = target else {
165 return Ok(None);
166 };
167 let data = revision
168 .nodes
169 .get(target)
170 .ok_or_else(|| invalid("Missing canvas attachment data"))?;
171 match &data.kind {
172 Kind::File { payload, .. } => Ok(payload.map(Arc::from)),
173 _ => Err(invalid("Canvas attachment data has the wrong type")),
174 }
175 };
176 Ok(Self {
177 id,
178 filename: filename.clone().unwrap_or_default(),
179 source_path: source_path.clone(),
180 size: icon_width.zip(*icon_height).map(|(w, h)| [w, h]),
181 bytes: payload(container)?,
182 preview: payload(preview)?,
183 })
184 }
185}
186
115#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]187#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
116pub struct Table {188pub struct Table {
117 pub id: ExGuid,189 pub id: ExGuid,
...@@ -157,6 +229,8 @@ pub struct TextObject {...@@ -157,6 +229,8 @@ pub struct TextObject {
157pub struct Image {229pub struct Image {
158 pub id: ExGuid,230 pub id: ExGuid,
159 pub layout: Layout,231 pub layout: Layout,
232 /// Displayed picture width and height in points.
233 pub size: Option<[f32; 2]>,
160 #[serde(skip)]234 #[serde(skip)]
161 pub bytes: Option<Arc<[u8]>>,235 pub bytes: Option<Arc<[u8]>>,
162 pub alt: Option<String>,236 pub alt: Option<String>,
...@@ -167,11 +241,53 @@ impl PartialEq for Image {...@@ -167,11 +241,53 @@ impl PartialEq for Image {
167 fn eq(&self, other: &Self) -> bool {241 fn eq(&self, other: &Self) -> bool {
168 self.id == other.id242 self.id == other.id
169 && self.layout == other.layout243 && self.layout == other.layout
244 && self.size == other.size
170 && self.alt == other.alt245 && self.alt == other.alt
171 && self.background == other.background246 && self.background == other.background
172 }247 }
173}248}
174249
250impl Image {
251 fn read(
252 revision: &Revision<'_>,
253 id: ExGuid,
254 node: &crate::document::Element<'_>,
255 ) -> Result<Self, Error> {
256 let invalid = |message| Error { offset: 0, message };
257 let Kind::Image {
258 container,
259 alt,
260 background,
261 picture_width,
262 picture_height,
263 ..
264 } = &node.kind
265 else {
266 unreachable!()
267 };
268 let bytes = if let Some(container) = container {
269 let data = revision
270 .nodes
271 .get(container)
272 .ok_or_else(|| invalid("Missing canvas image data"))?;
273 match &data.kind {
274 Kind::File { payload, .. } => payload.map(Arc::from),
275 _ => return Err(invalid("Canvas image data has the wrong type")),
276 }
277 } else {
278 None
279 };
280 Ok(Self {
281 id,
282 layout: node.layout.clone(),
283 size: picture_width.zip(*picture_height).map(|(w, h)| [w, h]),
284 bytes,
285 alt: alt.clone(),
286 background: background.unwrap_or(false),
287 })
288 }
289}
290
175#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]291#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
176pub struct Unsupported {292pub struct Unsupported {
177 pub id: ExGuid,293 pub id: ExGuid,
...@@ -341,31 +457,9 @@ impl Page {...@@ -341,31 +457,9 @@ impl Page {
341 page.objects.push(PageObject::Outline(outline));457 page.objects.push(PageObject::Outline(outline));
342 }458 }
343 }459 }
344 Kind::Image {460 Kind::Image { .. } => {
345 container,461 page.objects
346 alt,462 .push(PageObject::Image(Image::read(revision, id, node)?));
347 background,
348 ..
349 } => {
350 let bytes = if let Some(container) = container {
351 let data = revision
352 .nodes
353 .get(container)
354 .ok_or_else(|| invalid("Missing canvas image data"))?;
355 match &data.kind {
356 Kind::File { payload, .. } => payload.map(Arc::from),
357 _ => return Err(invalid("Canvas image data has the wrong type")),
358 }
359 } else {
360 None
361 };
362 page.objects.push(PageObject::Image(Image {
363 id,
364 layout: node.layout.clone(),
365 bytes,
366 alt: alt.clone(),
367 background: background.unwrap_or(false),
368 }));
369 }463 }
370 _ => page.objects.push(PageObject::Unsupported(Unsupported {464 _ => page.objects.push(PageObject::Unsupported(Unsupported {
371 id,465 id,
...@@ -493,6 +587,10 @@ fn read_paragraphs(...@@ -493,6 +587,10 @@ fn read_paragraphs(
493 text: text_content,587 text: text_content,
494 tags: content.tags.clone(),588 tags: content.tags.clone(),
495 })589 })
590 } else if matches!(content.kind, Kind::Image { .. }) {
591 ParagraphContent::Image(Image::read(revision, *content_id, content)?)
592 } else if matches!(content.kind, Kind::Attachment { .. }) {
593 ParagraphContent::Attachment(Attachment::read(revision, *content_id, content)?)
496 } else if matches!(content.kind, Kind::Table { .. }) {594 } else if matches!(content.kind, Kind::Table { .. }) {
497 ParagraphContent::Table(read_table(595 ParagraphContent::Table(read_table(
498 revision,596 revision,
...@@ -515,7 +613,9 @@ fn read_paragraphs(...@@ -515,7 +613,9 @@ fn read_paragraphs(
515 .chain(match &content {613 .chain(match &content {
516 ParagraphContent::Text(text) => text.tags.as_slice(),614 ParagraphContent::Text(text) => text.tags.as_slice(),
517 ParagraphContent::Table(table) => table.tags.as_slice(),615 ParagraphContent::Table(table) => table.tags.as_slice(),
518 ParagraphContent::Unsupported(_) => &[],616 ParagraphContent::Image(_)
617 | ParagraphContent::Attachment(_)
618 | ParagraphContent::Unsupported(_) => &[],
519 })619 })
520 .filter_map(|tag| tag.definition.as_ref())620 .filter_map(|tag| tag.definition.as_ref())
521 .map(|id| (id, false)),621 .map(|id| (id, false)),
crates/onestore/src/page/write.rs+262-8
...@@ -6,7 +6,7 @@ use crate::{...@@ -6,7 +6,7 @@ use crate::{
6 Error, ExGuid, Insertion, ObjectData, OutlineEdit, ParagraphJoin, ParagraphSplit, PropertySets,6 Error, ExGuid, Insertion, ObjectData, OutlineEdit, ParagraphJoin, ParagraphSplit, PropertySets,
7 RevisionIndex, Store, TextAttribute, TreeEdit, Value,7 RevisionIndex, Store, TextAttribute, TreeEdit, Value,
8 document::{Document, Format, Kind, Tag},8 document::{Document, Format, Kind, Tag},
9 write::{PropertyObject, RevisionEdit, write_revisions},9 write::{PropertyObject, RevisionEdit},
10};10};
11use std::{11use std::{
12 collections::{BTreeMap, BTreeSet},12 collections::{BTreeMap, BTreeSet},
...@@ -22,6 +22,12 @@ fn invalid(message: &'static str) -> Error {...@@ -22,6 +22,12 @@ fn invalid(message: &'static str) -> Error {
2222
23/// A native measurement array: a count byte, `header - 1` reserved bytes, then each value23/// A native measurement array: a count byte, `header - 1` reserved bytes, then each value
24/// in inches.24/// in inches.
25/// The `<ifndf>{GUID}` form a file-data declaration uses to name a payload in the file-data store.
26fn payload_reference(guid: [u8; 16]) -> String {
27 let id = ExGuid { guid, n: 0 }.to_string();
28 format!("<ifndf>{}", id.split(',').next().unwrap())
29}
30
25fn measurement_bytes(values: &[f32], header: usize) -> Result<Vec<u8>, Error> {31fn measurement_bytes(values: &[f32], header: usize) -> Result<Vec<u8>, Error> {
26 let count = u8::try_from(values.len())32 let count = u8::try_from(values.len())
27 .map_err(|_| invalid("A measurement array exceeds the document range"))?;33 .map_err(|_| invalid("A measurement array exceeds the document range"))?;
...@@ -379,6 +385,16 @@ impl Lowering<'_> {...@@ -379,6 +385,16 @@ impl Lowering<'_> {
379 }385 }
380 }386 }
381 (ParagraphContent::Unsupported(a), ParagraphContent::Unsupported(b)) if a == b => {}387 (ParagraphContent::Unsupported(a), ParagraphContent::Unsupported(b)) if a == b => {}
388 (ParagraphContent::Image(a), ParagraphContent::Image(b)) => {
389 if a != b {
390 return Err(invalid("A stored picture cannot be edited"));
391 }
392 }
393 (ParagraphContent::Attachment(a), ParagraphContent::Attachment(b)) => {
394 if a != b {
395 return Err(invalid("A stored attachment cannot be edited"));
396 }
397 }
382 _ => return Err(invalid("Paragraph content type cannot change")),398 _ => return Err(invalid("Paragraph content type cannot change")),
383 }399 }
384 }400 }
...@@ -609,10 +625,205 @@ impl Lowering<'_> {...@@ -609,10 +625,205 @@ impl Lowering<'_> {
609 });625 });
610 self.place_containers(old, new, placed, &existing)?;626 self.place_containers(old, new, placed, &existing)?;
611 self.edit_table_structure(old, new)?;627 self.edit_table_structure(old, new)?;
628 self.edit_images(old, new)?;
629 self.edit_attachments(old, new)?;
612 self.place_containers(old, new, placed, &deferred)?;630 self.place_containers(old, new, placed, &deferred)?;
613 Ok(())631 Ok(())
614 }632 }
615633
634 /// Gives each new attachment paragraph what OneNote stores for an inserted file: the
635 /// payload embedded in the file-data store, an embedded-file container declaring it,
636 /// and an attachment object naming the file that the paragraph holds as content.
637 fn edit_attachments(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
638 for (paragraph_id, paragraph) in &new.paragraphs {
639 let ParagraphContent::Attachment(attachment) = &paragraph.content else {
640 continue;
641 };
642 if let Some(previous) = old.paragraphs.get(paragraph_id) {
643 if previous.content != paragraph.content {
644 return Err(invalid("A stored attachment cannot be edited"));
645 }
646 continue;
647 }
648 let Some(bytes) = &attachment.bytes else {
649 return Err(invalid("A new attachment needs its payload"));
650 };
651 let name = attachment.filename.as_str();
652 if name.is_empty() || name.contains(['\0', '/', '\\']) {
653 return Err(invalid(
654 "An attachment needs a file name without path separators",
655 ));
656 }
657 let extension = name
658 .rfind('.')
659 .filter(|dot| *dot > 0)
660 .map(|dot| &name[dot..])
661 .unwrap_or("");
662 let attachment_id = self.allocate(attachment.id)?;
663 let file_id = ExGuid {
664 guid: crate::write::fresh_guid()?,
665 n: 1,
666 };
667 let payload_guid = crate::write::fresh_guid()?;
668 let reference = payload_reference(payload_guid);
669 let modified = crate::create::current_timestamps()?.0.to_le_bytes();
670 let mut values: Values = vec![(0x14001d7a, modified.to_vec())];
671 if let Some([width, height]) = attachment.size {
672 if !(width.is_finite() && height.is_finite() && width > 0.0 && height > 0.0) {
673 return Err(invalid("Attachment icon size must be positive"));
674 }
675 values.push((0x140034cd, (width / 36.0).to_le_bytes().to_vec()));
676 values.push((0x140034ce, (height / 36.0).to_le_bytes().to_vec()));
677 }
678 values.push((0x14001c3b, 0x409_u32.to_le_bytes().to_vec()));
679 values.push((0x10001cfe, 0x409_u16.to_le_bytes().to_vec()));
680 values.push((0x1c001dcf, vec![0; 32]));
681 values.push((
682 0x1c001d61,
683 [16u32, 1, 0, 0, 0]
684 .iter()
685 .flat_map(|v| v.to_le_bytes())
686 .collect(),
687 ));
688 values.push((0x14001c3e, 1u32.to_le_bytes().to_vec()));
689 values.push((0x14001c84, 1u32.to_le_bytes().to_vec()));
690 values.push((0x1c001c22, crate::create::string(name)));
691 values.push((0x1c001d9c, crate::create::string(name)));
692 if let Some(path) = &attachment.source_path {
693 values.push((0x1c001d9d, crate::create::string(path)));
694 }
695 let holder = self.id(*paragraph_id);
696 let space = self.space;
697 let mut payloads: Vec<([u8; 16], &[u8])> = vec![(payload_guid, bytes)];
698 let mut preview = None;
699 if let Some(icon) = &attachment.preview {
700 if !icon.starts_with(&[0x89, b'P', b'N', b'G']) {
701 return Err(invalid("An attachment preview is a PNG icon"));
702 }
703 let guid = crate::write::fresh_guid()?;
704 payloads.push((guid, icon));
705 preview = Some((
706 ExGuid {
707 guid: crate::write::fresh_guid()?,
708 n: 1,
709 },
710 payload_reference(guid),
711 ));
712 }
713 self.apply(|current| {
714 crate::write::write_revision_with_payloads(current, space, &payloads, |raw| {
715 let mut changed = BTreeMap::new();
716 let mut file = PropertyObject::file(file_id, &reference, extension)?;
717 file.jcid = 0x80036;
718 changed.insert(file_id, file);
719 let mut node = PropertyObject {
720 jcid: 0x60035,
721 bytes: crate::create::properties(&values)?,
722 global_ids: std::sync::Arc::new(BTreeMap::from([(0, attachment_id.guid)])),
723 };
724 node.reference(attachment_id)?;
725 let container = node.reference(file_id)?;
726 node.set(&[(0x20001d9b, &container)])?;
727 if let Some((icon_id, icon_reference)) = &preview {
728 changed.insert(
729 *icon_id,
730 PropertyObject::file(*icon_id, icon_reference, ".png")?,
731 );
732 let icon = node.reference(*icon_id)?;
733 node.set(&[(0x20001c3f, &icon)])?;
734 }
735 changed.insert(attachment_id, node);
736 let mut object = PropertyObject::from_object(&raw.objects[&holder])?;
737 let content = object.reference(attachment_id)?;
738 object.set(&[(0x24001c1f, &content), (0x14001d7a, &modified)])?;
739 changed.insert(holder, object);
740 Ok(changed)
741 })
742 })?;
743 }
744 Ok(())
745 }
746
747 /// Gives each new picture paragraph what OneNote stores for an inserted picture: the
748 /// payload embedded in the section's file-data store, a file-data object declaring it
749 /// by identity and extension, and a picture object the paragraph holds as content.
750 fn edit_images(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
751 for (paragraph_id, paragraph) in &new.paragraphs {
752 let ParagraphContent::Image(image) = &paragraph.content else {
753 continue;
754 };
755 if let Some(previous) = old.paragraphs.get(paragraph_id) {
756 if previous.content != paragraph.content {
757 return Err(invalid("A stored picture cannot be edited"));
758 }
759 continue;
760 }
761 let Some(bytes) = &image.bytes else {
762 return Err(invalid("A new picture needs its payload"));
763 };
764 let extension = match bytes.as_ref() {
765 [0x89, b'P', b'N', b'G', ..] => ".png",
766 [0xff, 0xd8, 0xff, ..] => ".jpg",
767 [b'G', b'I', b'F', b'8', ..] => ".gif",
768 [b'B', b'M', ..] => ".bmp",
769 _ => return Err(invalid("Choose a PNG, JPEG, GIF or BMP picture")),
770 };
771 let image_id = self.allocate(image.id)?;
772 let file_id = ExGuid {
773 guid: crate::write::fresh_guid()?,
774 n: 1,
775 };
776 let payload_guid = crate::write::fresh_guid()?;
777 let reference = payload_reference(payload_guid);
778 let modified = crate::create::current_timestamps()?.0.to_le_bytes();
779 let mut values: Values = vec![(0x14001d7a, modified.to_vec())];
780 if let Some([width, height]) = image.size {
781 if !(width.is_finite() && height.is_finite() && width > 0.0 && height > 0.0) {
782 return Err(invalid("Picture size must be positive"));
783 }
784 values.push((0x140034cd, (width / 36.0).to_le_bytes().to_vec()));
785 values.push((0x140034ce, (height / 36.0).to_le_bytes().to_vec()));
786 }
787 if let Some(alt) = &image.alt {
788 values.push((0x1c001e58, crate::create::string(alt)));
789 }
790 if image.background {
791 values.push((0x08001d13 | (1 << 31), Vec::new()));
792 }
793 values.push((0x08001d85, Vec::new()));
794 let holder = self.id(*paragraph_id);
795 let space = self.space;
796 let payload: &[u8] = bytes;
797 self.apply(|current| {
798 crate::write::write_revision_with_payloads(
799 current,
800 space,
801 &[(payload_guid, payload)],
802 |raw| {
803 let mut changed = BTreeMap::new();
804 let file = PropertyObject::file(file_id, &reference, extension)?;
805 changed.insert(file_id, file);
806 let mut picture = PropertyObject {
807 jcid: 0x60011,
808 bytes: crate::create::properties(&values)?,
809 global_ids: std::sync::Arc::new(BTreeMap::from([(0, image_id.guid)])),
810 };
811 picture.reference(image_id)?;
812 let container = picture.reference(file_id)?;
813 picture.set(&[(0x20001c3f, &container)])?;
814 changed.insert(image_id, picture);
815 let mut object = PropertyObject::from_object(&raw.objects[&holder])?;
816 let content = object.reference(image_id)?;
817 object.set(&[(0x24001c1f, &content), (0x14001d7a, &modified)])?;
818 changed.insert(holder, object);
819 Ok(changed)
820 },
821 )
822 })?;
823 }
824 Ok(())
825 }
826
616 fn place_containers(827 fn place_containers(
617 &mut self,828 &mut self,
618 old: &View<'_>,829 old: &View<'_>,
...@@ -648,7 +859,9 @@ impl Lowering<'_> {...@@ -648,7 +859,9 @@ impl Lowering<'_> {
648 ParagraphContent::Text(text) if text.date_field.is_none() => {859 ParagraphContent::Text(text) if text.date_field.is_none() => {
649 (text.text.text(), Some(text.id))860 (text.text.text(), Some(text.id))
650 }861 }
651 ParagraphContent::Table(_) => ("", None),862 ParagraphContent::Table(_)
863 | ParagraphContent::Image(_)
864 | ParagraphContent::Attachment(_) => ("", None),
652 _ => {865 _ => {
653 return Err(invalid(866 return Err(invalid(
654 "New paragraphs contain plain text without fields or styles",867 "New paragraphs contain plain text without fields or styles",
...@@ -1737,7 +1950,25 @@ fn squash(...@@ -1737,7 +1950,25 @@ fn squash(
1737 .collect();1950 .collect();
1738 let applied_store = Store::parse(applied)?;1951 let applied_store = Store::parse(applied)?;
1739 let applied_index = RevisionIndex::parse(&applied_store)?;1952 let applied_index = RevisionIndex::parse(&applied_store)?;
1740 write_revisions(source, |index| {1953 // Payloads the typed edits embedded travel into the squashed transaction as well.
1954 let source_store = Store::parse(source)?;
1955 let declared = |store: &Store<'_>| -> Vec<[u8; 16]> {
1956 store
1957 .lists
1958 .values()
1959 .flat_map(|list| &list.nodes)
1960 .filter(|node| node.id == 0x94)
1961 .filter_map(|node| node.payload.get(..16).and_then(|g| g.try_into().ok()))
1962 .collect()
1963 };
1964 let existing = declared(&source_store);
1965 let mut payloads = Vec::new();
1966 for guid in declared(&applied_store) {
1967 if !existing.contains(&guid) {
1968 payloads.push((guid, applied_store.file_data(guid)?));
1969 }
1970 }
1971 crate::write::write_revisions_with_payloads(source, &payloads, |index| {
1741 let mut changes = BTreeMap::new();1972 let mut changes = BTreeMap::new();
1742 for sid in applied_index.spaces.keys() {1973 for sid in applied_index.spaces.keys() {
1743 let Some(space) = index.spaces.get(sid) else {1974 let Some(space) = index.spaces.get(sid) else {
...@@ -1766,12 +1997,35 @@ fn squash(...@@ -1766,12 +1997,35 @@ fn squash(
1766 }) {1997 }) {
1767 continue;1998 continue;
1768 }1999 }
1769 let ObjectData::Properties(_) = object.data else {
1770 return Err(invalid("Page edits only produce property objects"));
1771 };
1772 let mut replacement = PropertyObject::from_object(object)?;
1773 remap(&mut replacement, &rename)?;
1774 let id = rename.get(id).copied().unwrap_or(*id);2000 let id = rename.get(id).copied().unwrap_or(*id);
2001 let mut replacement = match object.data {
2002 ObjectData::Properties(_) => {
2003 let mut replacement = PropertyObject::from_object(object)?;
2004 remap(&mut replacement, &rename)?;
2005 replacement
2006 }
2007 ObjectData::File {
2008 reference,
2009 extension,
2010 } => {
2011 let text = |bytes: &[u8]| {
2012 String::from_utf16(
2013 &bytes
2014 .chunks_exact(2)
2015 .map(|pair| u16::from_le_bytes([pair[0], pair[1]]))
2016 .collect::<Vec<_>>(),
2017 )
2018 .map_err(|_| invalid("Invalid UTF-16 file-data declaration"))
2019 };
2020 let mut replacement =
2021 PropertyObject::file(id, &text(reference)?, &text(extension)?)?;
2022 replacement.jcid = object.jcid;
2023 replacement
2024 }
2025 ObjectData::Encrypted(_) => {
2026 return Err(invalid("Page edits only produce property objects"));
2027 }
2028 };
1775 replacement.reference(id)?;2029 replacement.reference(id)?;
1776 if before.objects.contains_key(&id) && rename.values().any(|model| *model == id) {2030 if before.objects.contains_key(&id) && rename.values().any(|model| *model == id) {
1777 return Err(invalid(2031 return Err(invalid(
crates/onestore/src/write.rs+154-6
...@@ -353,6 +353,15 @@ pub(crate) fn replace_objects(...@@ -353,6 +353,15 @@ pub(crate) fn replace_objects(
353 })353 })
354}354}
355355
356/// The object type OneNote gives embedded picture payload declarations; embedded files
357/// use `EMBEDDED_FILE_JCID` with the same declaration shape.
358pub(crate) const FILE_DATA_JCID: u32 = 0x80039;
359pub(crate) const EMBEDDED_FILE_JCID: u32 = 0x80036;
360
361fn is_file_declaration(jcid: u32) -> bool {
362 jcid == FILE_DATA_JCID || jcid == EMBEDDED_FILE_JCID
363}
364
356pub(crate) struct PropertyObject {365pub(crate) struct PropertyObject {
357 pub jcid: u32,366 pub jcid: u32,
358 pub bytes: Vec<u8>,367 pub bytes: Vec<u8>,
...@@ -368,6 +377,51 @@ pub(crate) enum RevisionEdit {...@@ -368,6 +377,51 @@ pub(crate) enum RevisionEdit {
368}377}
369378
370impl PropertyObject {379impl PropertyObject {
380 /// A file-data object declaring an embedded payload by its store identity, or an
381 /// external payload by name; `extension` includes its leading dot.
382 pub fn file(id: ExGuid, reference: &str, extension: &str) -> Result<Self> {
383 if reference.is_empty() || reference.contains('\0') || extension.contains('\0') {
384 return Err(Error {
385 offset: 0,
386 message: "File-data references and extensions are nonempty and contain no NUL",
387 });
388 }
389 let mut bytes = Vec::new();
390 for text in [reference, extension] {
391 let encoded: Vec<u8> = text.encode_utf16().flat_map(u16::to_le_bytes).collect();
392 bytes.extend_from_slice(&u32::try_from(encoded.len()).unwrap().to_le_bytes());
393 bytes.extend_from_slice(&encoded);
394 }
395 Ok(Self {
396 jcid: FILE_DATA_JCID,
397 bytes,
398 global_ids: Arc::new(BTreeMap::from([(0, id.guid)])),
399 })
400 }
401
402 /// The reference and extension of a file-data declaration built by `file`.
403 fn file_declaration(&self) -> Result<(&[u8], &[u8])> {
404 let malformed = || Error {
405 offset: 0,
406 message: "Malformed file-data declaration",
407 };
408 let mut rest = self.bytes.as_slice();
409 let mut parts = Vec::new();
410 for _ in 0..2 {
411 let length = usize::try_from(u32::from_le_bytes(
412 rest.get(..4).ok_or_else(malformed)?.try_into().unwrap(),
413 ))
414 .map_err(|_| malformed())?;
415 let bytes = rest.get(4..4 + length).ok_or_else(malformed)?;
416 parts.push(bytes);
417 rest = &rest[4 + length..];
418 }
419 if !rest.is_empty() {
420 return Err(malformed());
421 }
422 Ok((parts[0], parts[1]))
423 }
424
371 pub fn from_object(object: &crate::Object<'_>) -> Result<Self> {425 pub fn from_object(object: &crate::Object<'_>) -> Result<Self> {
372 let ObjectData::Properties(bytes) = object.data else {426 let ObjectData::Properties(bytes) = object.data else {
373 return Err(Error {427 return Err(Error {
...@@ -649,7 +703,16 @@ pub(crate) fn write_revision(...@@ -649,7 +703,16 @@ pub(crate) fn write_revision(
649 space: ExGuid,703 space: ExGuid,
650 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,704 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
651) -> Result<Vec<u8>> {705) -> Result<Vec<u8>> {
652 write_revisions(source, |index| {706 write_revision_with_payloads(source, space, &[], edit)
707}
708
709pub(crate) fn write_revision_with_payloads(
710 source: &[u8],
711 space: ExGuid,
712 payloads: &[([u8; 16], &[u8])],
713 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
714) -> Result<Vec<u8>> {
715 write_revisions_with_payloads(source, payloads, |index| {
653 let rid = index.active(space)?;716 let rid = index.active(space)?;
654 let revision = index.resolve(space, rid)?;717 let revision = index.resolve(space, rid)?;
655 Ok(BTreeMap::from([(718 Ok(BTreeMap::from([(
...@@ -695,6 +758,17 @@ fn append_fragment(...@@ -695,6 +758,17 @@ fn append_fragment(
695pub(crate) fn write_revisions(758pub(crate) fn write_revisions(
696 source: &[u8],759 source: &[u8],
697 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,760 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,
761) -> Result<Vec<u8>> {
762 write_revisions_with_payloads(source, &[], edit)
763}
764
765/// `write_revisions` that also stores embedded payloads: each becomes a file-data store
766/// object referenced from the root file node list under its identity, as OneNote embeds
767/// pictures and attachments.
768pub(crate) fn write_revisions_with_payloads(
769 source: &[u8],
770 payloads: &[([u8; 16], &[u8])],
771 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,
698) -> Result<Vec<u8>> {772) -> Result<Vec<u8>> {
699 let store = Store::parse(source)?;773 let store = Store::parse(source)?;
700 let is_section = store.header.file_type == FileType::Section;774 let is_section = store.header.file_type == FileType::Section;
...@@ -780,16 +854,29 @@ pub(crate) fn write_revisions(...@@ -780,16 +854,29 @@ pub(crate) fn write_revisions(
780 message: "New objects require a section file",854 message: "New objects require a section file",
781 });855 });
782 }856 }
783 if replacement.jcid & 0x20000 == 0857 if replacement.global_ids.keys().any(|i| *i > 0xffffff) {
784 || replacement.global_ids.keys().any(|i| *i > 0xffffff)
785 {
786 return Err(Error {858 return Err(Error {
787 offset: 0,859 offset: 0,
788 message: "Invalid property object declaration",860 message: "Invalid property object declaration",
789 });861 });
790 }862 }
791 compact(*id, &replacement.global_ids)?;863 compact(*id, &replacement.global_ids)?;
792 PropertySets::parse(&replacement.bytes)?;864 if is_file_declaration(replacement.jcid) {
865 if !is_section {
866 return Err(Error {
867 offset: 0,
868 message: "File-data objects require a section file",
869 });
870 }
871 replacement.file_declaration()?;
872 } else if replacement.jcid & 0x20000 == 0 {
873 return Err(Error {
874 offset: 0,
875 message: "Invalid property object declaration",
876 });
877 } else {
878 PropertySets::parse(&replacement.bytes)?;
879 }
793 }880 }
794 // Native coalescing of duplicate readonly styles can leave dangling references.881 // Native coalescing of duplicate readonly styles can leave dangling references.
795 let mut aliases = BTreeMap::new();882 let mut aliases = BTreeMap::new();
...@@ -824,6 +911,9 @@ pub(crate) fn write_revisions(...@@ -824,6 +911,9 @@ pub(crate) fn write_revisions(
824 replacements.remove(id);911 replacements.remove(id);
825 }912 }
826 for object in replacements.values_mut() {913 for object in replacements.values_mut() {
914 if is_file_declaration(object.jcid) {
915 continue;
916 }
827 let mut remapped = Vec::new();917 let mut remapped = Vec::new();
828 for property in PropertySets::parse(&object.bytes)?.sets.iter().flatten() {918 for property in PropertySets::parse(&object.bytes)?.sets.iter().flatten() {
829 if let Value::References {919 if let Value::References {
...@@ -859,12 +949,21 @@ pub(crate) fn write_revisions(...@@ -859,12 +949,21 @@ pub(crate) fn write_revisions(
859 }949 }
860 let mut changed: BTreeSet<_> = replacements.keys().copied().collect();950 let mut changed: BTreeSet<_> = replacements.keys().copied().collect();
861 for (id, replacement) in &replacements {951 for (id, replacement) in &replacements {
952 let data = if is_file_declaration(replacement.jcid) {
953 let (reference, extension) = replacement.file_declaration()?;
954 ObjectData::File {
955 reference,
956 extension,
957 }
958 } else {
959 ObjectData::Properties(&replacement.bytes)
960 };
862 revision.objects.insert(961 revision.objects.insert(
863 *id,962 *id,
864 crate::Object {963 crate::Object {
865 jcid: replacement.jcid,964 jcid: replacement.jcid,
866 reference_count: 0,965 reference_count: 0,
867 data: ObjectData::Properties(&replacement.bytes),966 data,
868 global_ids: Arc::clone(&replacement.global_ids),967 global_ids: Arc::clone(&replacement.global_ids),
869 },968 },
870 );969 );
...@@ -1139,6 +1238,55 @@ pub(crate) fn write_revisions(...@@ -1139,6 +1238,55 @@ pub(crate) fn write_revisions(
1139 )?);1238 )?);
1140 }1239 }
1141 }1240 }
1241 let mut data_nodes = Vec::new();
1242 for (guid, payload) in payloads {
1243 if !is_section {
1244 return Err(Error {
1245 offset: 0,
1246 message: "Embedded payloads require a section file",
1247 });
1248 }
1249 let mut blob = vec![
1250 0xe7, 0x16, 0xe3, 0xbd, 0x65, 0x26, 0x11, 0x45, 0xa4, 0xc4, 0x8d, 0x4d, 0x0b, 0x7a,
1251 0x9e, 0xac,
1252 ];
1253 blob.extend_from_slice(&(payload.len() as u64).to_le_bytes());
1254 blob.extend_from_slice(&[0; 12]);
1255 blob.extend_from_slice(payload);
1256 blob.resize(blob.len().next_multiple_of(8), 0);
1257 blob.extend_from_slice(&[
1258 0x22, 0xa7, 0xfb, 0x71, 0x79, 0x0f, 0x0b, 0x4a, 0xbb, 0x13, 0x89, 0x92, 0x56, 0x42,
1259 0x6b, 0x24,
1260 ]);
1261 let chunk = append(&mut output, &blob)?;
1262 data_nodes.push(node(0x94, Some(Reference::Data(chunk)), guid)?);
1263 }
1264 if !data_nodes.is_empty() {
1265 // Payload declarations live in the file-data store list the root list references.
1266 let root = store.list(store.header.root)?;
1267 match root.nodes.iter().find(|node| node.id == 0x90) {
1268 Some(reference) => {
1269 let Some(Reference::NodeList(chunk)) = reference.reference else {
1270 return Err(Error {
1271 offset: reference.offset,
1272 message: "File-data store reference lacks a list",
1273 });
1274 };
1275 counts.push(append_fragment(
1276 source,
1277 &mut output,
1278 store.list(chunk)?,
1279 &data_nodes,
1280 )?);
1281 }
1282 None => {
1283 let list_id = allocate_list()?;
1284 let chunk = append_list(&mut output, list_id, &data_nodes)?;
1285 counts.push((list_id, data_nodes.len()));
1286 root_nodes.push(node(0x90, Some(Reference::NodeList(chunk)), &[])?);
1287 }
1288 }
1289 }
1142 if !root_nodes.is_empty() {1290 if !root_nodes.is_empty() {
1143 counts.push(append_fragment(1291 counts.push(append_fragment(
1144 source,1292 source,
crates/onestore/tests/page_attachments.rs created+203
...@@ -0,0 +1,203 @@
1use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
3 document::{Document, Format},
4 page::{
5 Attachment, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,
6 },
7};
8use std::sync::Arc;
9
10const NATIVE: &[u8] =
11 include_bytes!("../../../corpus/native/cold-05-06-attachment/notebook/synthetic.one");
12const AUTHOR: &str = "Attachment author";
13const PAYLOAD: &[u8] = b"Rust wrote this attachment.\n";
14
15fn first_page(bytes: &[u8]) -> (ExGuid, Page) {
16 let store = Store::parse(bytes).unwrap();
17 assert!(store.checksum_mismatches.is_empty());
18 let index = RevisionIndex::parse(&store).unwrap();
19 let document = Document::parse(&index).unwrap();
20 let (space, _) = document.pages().unwrap()[0];
21 (space, Page::from_space(&document, space).unwrap())
22}
23
24fn page_in(bytes: &[u8], space: ExGuid) -> Page {
25 let store = Store::parse(bytes).unwrap();
26 assert!(store.checksum_mismatches.is_empty());
27 let index = RevisionIndex::parse(&store).unwrap();
28 Page::from_space(&Document::parse(&index).unwrap(), space).unwrap()
29}
30
31fn body_paragraphs(page: &mut Page) -> &mut Vec<PageParagraph> {
32 page.objects
33 .iter_mut()
34 .find_map(|object| match object {
35 PageObject::Outline(outline) if !outline.title => Some(&mut outline.paragraphs),
36 _ => None,
37 })
38 .unwrap()
39}
40
41fn attachments(page: &Page) -> Vec<&Attachment> {
42 page.objects
43 .iter()
44 .filter_map(|object| match object {
45 PageObject::Outline(outline) => Some(outline),
46 _ => None,
47 })
48 .flat_map(|outline| outline.paragraphs.iter())
49 .filter_map(|p| match &p.content {
50 ParagraphContent::Attachment(attachment) => Some(attachment),
51 _ => None,
52 })
53 .collect()
54}
55
56fn plain_paragraph(template: &PageParagraph, text: &str) -> PageParagraph {
57 let mut paragraph = template.clone();
58 paragraph.id = new_id().unwrap();
59 paragraph.parent = None;
60 paragraph.level = 1;
61 paragraph.lists.clear();
62 paragraph.tags.clear();
63 paragraph.style = None;
64 paragraph.collapsed = false;
65 paragraph.content = ParagraphContent::Text(TextObject {
66 id: new_id().unwrap(),
67 date_field: None,
68 text: onestore::page::Paragraph::new(
69 text.into(),
70 Format {
71 font: Some("Calibri".into()),
72 font_size: Some(11.0),
73 language: Some(1033),
74 ..Default::default()
75 },
76 ),
77 tags: Vec::new(),
78 });
79 paragraph
80}
81
82#[test]
83fn a_native_attachment_reads_as_paragraph_content_with_payload_and_preview() {
84 let (_, page) = first_page(NATIVE);
85 let found = attachments(&page);
86 let [attachment] = found.as_slice() else {
87 panic!("{page:?}");
88 };
89 assert_eq!(attachment.filename, "fictitious-attachment.txt");
90 assert!(
91 attachment
92 .source_path
93 .as_deref()
94 .unwrap()
95 .ends_with("fictitious-attachment.txt")
96 );
97 assert_eq!(attachment.bytes.as_ref().map(|b| b.len()), Some(43));
98 assert_eq!(attachment.preview.as_ref().map(|b| b.len()), Some(724));
99 assert_eq!(attachment.size, Some([24.0, 24.0]));
100}
101
102fn insert_attachment(section: &str, preview: Option<Arc<[u8]>>) -> Vec<u8> {
103 let source = onestore::create_section(section, "Before the file", "Author").unwrap();
104 let (space, before) = first_page(&source);
105 let mut after = before.clone();
106 let template = body_paragraphs(&mut after)[0].clone();
107 let mut holder = plain_paragraph(&template, "");
108 holder.format = Format::default();
109 holder.content = ParagraphContent::Attachment(Attachment {
110 id: new_id().unwrap(),
111 filename: "notes 🦀.txt".into(),
112 source_path: None,
113 size: Some([24.0, 24.0]),
114 bytes: Some(Arc::from(PAYLOAD)),
115 preview: preview.clone(),
116 });
117 body_paragraphs(&mut after).push(holder.clone());
118 body_paragraphs(&mut after).push(plain_paragraph(&template, "After the file"));
119 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
120 let stored = page_in(written.as_bytes(), space);
121 let mut expected = after.clone();
122 expected.title = stored.title.clone();
123 assert_eq!(stored, expected);
124 let found = attachments(&stored);
125 let [stored_attachment] = found.as_slice() else {
126 panic!()
127 };
128 assert_eq!(stored_attachment.bytes.as_deref(), Some(PAYLOAD));
129 assert_eq!(stored_attachment.preview, preview);
130 assert_eq!(
131 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
132 .unwrap()
133 .as_bytes(),
134 written.as_bytes()
135 );
136 let mut removed = stored.clone();
137 body_paragraphs(&mut removed).retain(|p| p.id != holder.id);
138 let again = PreparedEdit::page(written.as_bytes(), space, &removed, AUTHOR).unwrap();
139 assert!(attachments(&page_in(again.as_bytes(), space)).is_empty());
140 written.as_bytes().to_vec()
141}
142
143/// `ONESTORE_ATTACHMENT_EXPORT` names a directory receiving both candidates for a cold reopen:
144/// `plain/` without an icon preview and `icon/` with the native fixture's icon.
145#[test]
146fn an_attachment_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {
147 let (_, native) = first_page(NATIVE);
148 let icon = attachments(&native)[0].preview.clone().unwrap();
149 let plain = insert_attachment("files.one", None);
150 let with_icon = insert_attachment("icon.one", Some(icon));
151 if let Some(directory) = std::env::var_os("ONESTORE_ATTACHMENT_EXPORT") {
152 let directory = std::path::PathBuf::from(directory);
153 for (name, written) in [("plain", &plain), ("icon", &with_icon)] {
154 let directory = directory.join(name);
155 std::fs::create_dir_all(&directory).unwrap();
156 std::fs::write(directory.join("files.one"), written).unwrap();
157 let file_id = Store::parse(written).unwrap().header.file_id;
158 std::fs::write(
159 directory.join("Open Notebook.onetoc2"),
160 onestore::create_table_of_contents(
161 "Open Notebook.onetoc2",
162 &[("files.one", file_id)],
163 )
164 .unwrap(),
165 )
166 .unwrap();
167 }
168 }
169}
170
171#[test]
172fn attachments_need_a_file_name_and_stored_ones_stay_fixed() {
173 let source = onestore::create_section("files.one", "Text", "Author").unwrap();
174 let (space, before) = first_page(&source);
175 let mut after = before.clone();
176 let template = body_paragraphs(&mut after)[0].clone();
177 let mut holder = plain_paragraph(&template, "");
178 holder.content = ParagraphContent::Attachment(Attachment {
179 id: new_id().unwrap(),
180 filename: "sub/dir.txt".into(),
181 source_path: None,
182 size: None,
183 bytes: Some(Arc::from(PAYLOAD)),
184 preview: None,
185 });
186 body_paragraphs(&mut after).push(holder);
187 assert!(PreparedEdit::page(&source, space, &after, AUTHOR).is_err());
188 let (space, native) = first_page(NATIVE);
189 let mut renamed = native.clone();
190 let mut found = 0;
191 for object in &mut renamed.objects {
192 if let PageObject::Outline(outline) = object {
193 for paragraph in &mut outline.paragraphs {
194 if let ParagraphContent::Attachment(attachment) = &mut paragraph.content {
195 attachment.filename = "renamed.txt".into();
196 found += 1;
197 }
198 }
199 }
200 }
201 assert_eq!(found, 1);
202 assert!(PreparedEdit::page(NATIVE, space, &renamed, AUTHOR).is_err());
203}
crates/onestore/tests/page_images.rs created+190
...@@ -0,0 +1,190 @@
1use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
3 document::{Document, Format},
4 page::{Image, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},
5};
6use std::sync::Arc;
7
8const NATIVE: &[u8] =
9 include_bytes!("../../../corpus/native/cold-05-05-image/notebook/synthetic.one");
10const AUTHOR: &str = "Picture author";
11/// A one-pixel PNG.
12const PNG: &[u8] = &[
13 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52,
14 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x08, 0x02, 0x00, 0x00, 0x00, 0x90, 0x77, 0x53,
15 0xde, 0x00, 0x00, 0x00, 0x0c, 0x49, 0x44, 0x41, 0x54, 0x08, 0xd7, 0x63, 0xf8, 0xcf, 0xc0, 0x00,
16 0x00, 0x03, 0x01, 0x01, 0x00, 0x18, 0xdd, 0x8d, 0xb0, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e,
17 0x44, 0xae, 0x42, 0x60, 0x82,
18];
19
20fn first_page(bytes: &[u8]) -> (ExGuid, Page) {
21 let store = Store::parse(bytes).unwrap();
22 assert!(store.checksum_mismatches.is_empty());
23 let index = RevisionIndex::parse(&store).unwrap();
24 let document = Document::parse(&index).unwrap();
25 let (space, _) = document.pages().unwrap()[0];
26 (space, Page::from_space(&document, space).unwrap())
27}
28
29fn page_in(bytes: &[u8], space: ExGuid) -> Page {
30 let store = Store::parse(bytes).unwrap();
31 assert!(store.checksum_mismatches.is_empty());
32 let index = RevisionIndex::parse(&store).unwrap();
33 Page::from_space(&Document::parse(&index).unwrap(), space).unwrap()
34}
35
36fn body_paragraphs(page: &mut Page) -> &mut Vec<PageParagraph> {
37 page.objects
38 .iter_mut()
39 .find_map(|object| match object {
40 PageObject::Outline(outline) if !outline.title => Some(&mut outline.paragraphs),
41 _ => None,
42 })
43 .unwrap()
44}
45
46fn pictures(page: &Page) -> Vec<&Image> {
47 page.objects
48 .iter()
49 .filter_map(|object| match object {
50 PageObject::Outline(outline) => Some(outline),
51 _ => None,
52 })
53 .flat_map(|outline| outline.paragraphs.iter())
54 .filter_map(|p| match &p.content {
55 ParagraphContent::Image(image) => Some(image),
56 _ => None,
57 })
58 .collect()
59}
60
61fn plain_paragraph(template: &PageParagraph, text: &str) -> PageParagraph {
62 let mut paragraph = template.clone();
63 paragraph.id = new_id().unwrap();
64 paragraph.parent = None;
65 paragraph.level = 1;
66 paragraph.lists.clear();
67 paragraph.tags.clear();
68 paragraph.style = None;
69 paragraph.collapsed = false;
70 paragraph.content = ParagraphContent::Text(TextObject {
71 id: new_id().unwrap(),
72 date_field: None,
73 text: onestore::page::Paragraph::new(
74 text.into(),
75 Format {
76 font: Some("Calibri".into()),
77 font_size: Some(11.0),
78 language: Some(1033),
79 ..Default::default()
80 },
81 ),
82 tags: Vec::new(),
83 });
84 paragraph
85}
86
87#[test]
88fn a_native_picture_reads_as_paragraph_content_with_its_payload() {
89 let (_, page) = first_page(NATIVE);
90 let found = pictures(&page);
91 let [picture] = found.as_slice() else {
92 panic!("{page:?}");
93 };
94 assert_eq!(picture.bytes.as_ref().map(|b| b.len()), Some(68));
95 assert!(picture.bytes.as_ref().unwrap().starts_with(&PNG[..8]));
96 assert_eq!(picture.size, Some([0.75, 0.75]));
97}
98
99/// `ONESTORE_IMAGE_EXPORT` names a new directory receiving the candidate for a cold reopen.
100#[test]
101fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {
102 let source = onestore::create_section("pictures.one", "Before the picture", "Author").unwrap();
103 let (space, before) = first_page(&source);
104 let mut after = before.clone();
105 let template = body_paragraphs(&mut after)[0].clone();
106 let mut holder = plain_paragraph(&template, "");
107 holder.format = Format::default();
108 let image = Image {
109 id: new_id().unwrap(),
110 layout: Default::default(),
111 size: Some([0.75, 0.75]),
112 bytes: Some(Arc::from(PNG)),
113 alt: None,
114 background: false,
115 };
116 holder.content = ParagraphContent::Image(image.clone());
117 body_paragraphs(&mut after).push(holder.clone());
118 body_paragraphs(&mut after).push(plain_paragraph(&template, "After the picture"));
119 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
120 let stored = page_in(written.as_bytes(), space);
121 let mut expected = after.clone();
122 expected.title = stored.title.clone();
123 assert_eq!(stored, expected);
124 let found = pictures(&stored);
125 let [picture] = found.as_slice() else {
126 panic!()
127 };
128 assert_eq!(picture.bytes.as_deref(), Some(PNG));
129 assert_eq!(
130 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
131 .unwrap()
132 .as_bytes(),
133 written.as_bytes()
134 );
135 if let Some(directory) = std::env::var_os("ONESTORE_IMAGE_EXPORT") {
136 let directory = std::path::PathBuf::from(directory);
137 std::fs::create_dir(&directory).unwrap();
138 std::fs::write(directory.join("pictures.one"), written.as_bytes()).unwrap();
139 let written_store = Store::parse(written.as_bytes()).unwrap();
140 std::fs::write(
141 directory.join("Open Notebook.onetoc2"),
142 onestore::create_table_of_contents(
143 "Open Notebook.onetoc2",
144 &[("pictures.one", written_store.header.file_id)],
145 )
146 .unwrap(),
147 )
148 .unwrap();
149 }
150 let mut removed = stored.clone();
151 body_paragraphs(&mut removed).retain(|p| p.id != holder.id);
152 let again = PreparedEdit::page(written.as_bytes(), space, &removed, AUTHOR).unwrap();
153 let stored = page_in(again.as_bytes(), space);
154 assert!(pictures(&stored).is_empty());
155 assert_eq!(body_paragraphs(&mut stored.clone()).len(), 2);
156}
157
158#[test]
159fn pictures_need_a_recognised_payload_and_stored_ones_stay_fixed() {
160 let source = onestore::create_section("pictures.one", "Text", "Author").unwrap();
161 let (space, before) = first_page(&source);
162 let mut after = before.clone();
163 let template = body_paragraphs(&mut after)[0].clone();
164 let mut holder = plain_paragraph(&template, "");
165 holder.content = ParagraphContent::Image(Image {
166 id: new_id().unwrap(),
167 layout: Default::default(),
168 size: None,
169 bytes: Some(Arc::from(b"not a picture".as_slice())),
170 alt: None,
171 background: false,
172 });
173 body_paragraphs(&mut after).push(holder);
174 assert!(PreparedEdit::page(&source, space, &after, AUTHOR).is_err());
175 let (space, native) = first_page(NATIVE);
176 let mut resized = native.clone();
177 let mut found = 0;
178 for object in &mut resized.objects {
179 if let PageObject::Outline(outline) = object {
180 for paragraph in &mut outline.paragraphs {
181 if let ParagraphContent::Image(image) = &mut paragraph.content {
182 image.size = Some([10.0, 10.0]);
183 found += 1;
184 }
185 }
186 }
187 }
188 assert_eq!(found, 1);
189 assert!(PreparedEdit::page(NATIVE, space, &resized, AUTHOR).is_err());
190}
crates/onestore/tests/page_model.rs+2
...@@ -38,6 +38,8 @@ fn identities(page: &Page) -> Vec<onestore::ExGuid> {...@@ -38,6 +38,8 @@ fn identities(page: &Page) -> Vec<onestore::ExGuid> {
38 }38 }
39 }39 }
40 }40 }
41 ParagraphContent::Image(image) => ids.push(image.id),
42 ParagraphContent::Attachment(attachment) => ids.push(attachment.id),
41 ParagraphContent::Unsupported(unsupported) => ids.push(unsupported.id),43 ParagraphContent::Unsupported(unsupported) => ids.push(unsupported.id),
42 }44 }
43 }45 }
crates/onestore/tests/page_tables.rs+3-1
...@@ -72,7 +72,9 @@ fn referenced(page: &Page) -> std::collections::BTreeSet<ExGuid> {...@@ -72,7 +72,9 @@ fn referenced(page: &Page) -> std::collections::BTreeSet<ExGuid> {
72 }72 }
73 }73 }
74 }74 }
75 ParagraphContent::Unsupported(_) => {}75 ParagraphContent::Image(_)
76 | ParagraphContent::Attachment(_)
77 | ParagraphContent::Unsupported(_) => {}
76 }78 }
77 }79 }
78 }80 }
crates/onestore/tests/support/page_edits.rs+6
...@@ -123,6 +123,12 @@ fn projection(page: &Page) -> String {...@@ -123,6 +123,12 @@ fn projection(page: &Page) -> String {
123 ParagraphContent::Table(table) => {123 ParagraphContent::Table(table) => {
124 out.push_str(&format!(" table {}", table.id))124 out.push_str(&format!(" table {}", table.id))
125 }125 }
126 ParagraphContent::Image(image) => {
127 out.push_str(&format!(" image {}", image.id))
128 }
129 ParagraphContent::Attachment(attachment) => {
130 out.push_str(&format!(" attachment {}", attachment.id))
131 }
126 ParagraphContent::Unsupported(u) => {132 ParagraphContent::Unsupported(u) => {
127 out.push_str(&format!(" unsupported {}", u.id))133 out.push_str(&format!(" unsupported {}", u.id))
128 }134 }
tools/test_attachment_edit.py created+36
...@@ -0,0 +1,36 @@
1from pathlib import Path
2import re
3import runpy
4import shutil
5from tempfile import TemporaryDirectory
6import unittest
7import xml.etree.ElementTree as ET
8
9from native_xml import ns
10
11ROOT = Path(__file__).resolve().parent.parent
12FIXTURE = ROOT / 'corpus/attachment-edit'
13compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
14
15
16class AttachmentEditTest(unittest.TestCase):
17 def check(self, variant):
18 with TemporaryDirectory() as temporary:
19 read = Path(temporary) / 'read'
20 shutil.copytree(FIXTURE / variant / 'cold/read', read)
21 compare(FIXTURE / variant / 'candidate', read)
22 page, = (ET.parse(path).getroot() for path in sorted((FIXTURE / variant / 'cold/read').glob('page-*.xml')))
23 inserted, = page.iter('{%s}InsertedFile' % ns['one'])
24 self.assertEqual(inserted.get('preferredName'), 'notes 🦀.txt')
25 texts = [re.sub(r'<[^>]*>', '', t.text or '') for t in page.findall('.//one:OE/one:T', ns)]
26 self.assertEqual(texts, ['Before the file', 'After the file'])
27
28 def test_a_rust_inserted_attachment_renders_natively_with_its_payload(self):
29 self.check('plain')
30
31 def test_a_rust_inserted_attachment_with_an_icon_preview_renders_natively(self):
32 self.check('icon')
33
34
35if __name__ == '__main__':
36 unittest.main()
tools/test_picture_edit.py created+34
...@@ -0,0 +1,34 @@
1import base64
2from pathlib import Path
3import re
4import runpy
5import shutil
6from tempfile import TemporaryDirectory
7import unittest
8import xml.etree.ElementTree as ET
9
10from native_xml import ns
11
12ROOT = Path(__file__).resolve().parent.parent
13FIXTURE = ROOT / 'corpus/picture-edit'
14compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
15
16
17class PictureEditTest(unittest.TestCase):
18 def test_a_rust_inserted_picture_renders_natively_with_its_payload(self):
19 with TemporaryDirectory() as temporary:
20 read = Path(temporary) / 'read'
21 shutil.copytree(FIXTURE / 'cold/read', read)
22 compare(FIXTURE / 'candidate', read)
23 page, = (ET.parse(path).getroot() for path in sorted((FIXTURE / 'cold/read').glob('page-*.xml')))
24 image, = page.iter('{%s}Image' % ns['one'])
25 self.assertEqual(image.get('format'), 'png')
26 payload = base64.b64decode(image.find('one:Data', ns).text)
27 self.assertEqual(payload[:8], b'\x89PNG\r\n\x1a\n')
28 self.assertEqual(len(payload), 69)
29 texts = [re.sub(r'<[^>]*>', '', t.text or '') for t in page.findall('.//one:OE/one:T', ns)]
30 self.assertEqual(texts, ['Before the picture', 'After the picture'])
31
32
33if __name__ == '__main__':
34 unittest.main()