authorgravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-12 02:17:29-07:00
committergravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-12 17:00:32-07:00
log8a86e3ac0f817d8d27104a0a58e865b17e94eb35
tree00fdc935cec8969af55baee1f12356e7e14042e6
parentb6b6e903d65a5b876c5c009ba390df25d9842040
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

feat: write bullets and numbering through the page model

A paragraph's list references and the list definitions of the page model are differences the page writer now lowers. A definition new to the section becomes a native list node (jcid 0x60012) carrying the format string, list font, restart, bullet index and the definition's text formatting; a changed definition is rewritten in place; a definition another paragraph already owns is copied, since native list nodes belong to one paragraph; a dropped reference leaves the node unreferenced. Bullet definitions name their glyph and font and carry the cleared flag every native bullet node has. List nodes are allocated like other new objects and renamed to the model's identities in squash: a node written directly under a model identity had the null compact identity and failed OneNote's integrity check on open. The three-way merge carries definitions the local side added or changed, so a local bullet merges with a remote edit elsewhere. The model mutation driver toggles bullets, and its round-trip projection coalesces runs that differ only in attributes the writer leaves as stored, such as the language tag. corpus/list-edit is the native gate: a Rust-authored page with a bullet, two numbered paragraphs, a nested numbered child, a restart at three and plain paragraphs, cold-read by OneNote 2010 with the expected glyph and numbering text; tools/test_list_edit.py checks it without a VM. Assisted-by: claude-fable-5.1

24 files changed, 943 insertions(+), 21 deletions(-)

corpus/list-edit/README.md created+23
...@@ -0,0 +1,23 @@
1# Rust list authoring
2
3`candidate/` is the page writer's output for
4`bullets_numbering_nesting_and_restarts_publish_on_a_fresh_page` in
5`crates/onestore/tests/page_lists.rs`: on a section created in Rust, the first
6paragraph gains a bullet (glyph U+25CB in Courier New, bullet index 4, 11 pt),
7then two numbered paragraphs, a nested numbered child, a numbered paragraph
8restarting at three, a nested plain paragraph and a trailing plain paragraph.
9The numbering definition is the upper-roman `##.` node copied from
10`corpus/outline-edit/tree`. Every list node is written with the model's
11identity after squash, so the reread model equals the saved one.
12
13`cold/` is a fresh OneNote 2010 read: the bullet renders (reported as bullet
14index 3 at 11 pt), the numbered paragraphs read `I.` and `II.`, the nested
15child restarts at `I.`, the restarted paragraph reads `III.`, and the plain
16paragraphs carry no list. `tools/test_list_edit.py` checks this without a VM.
17
18A list node whose compact identity would be the null identity (sequence zero
19at table index zero) fails OneNote's integrity check on open; the writer
20allocates list nodes like other new objects and renames them to the model's
21identities in squash. Regenerate with `ONESTORE_LIST_EXPORT` set to a new
22absolute directory while running the test, then cold-open it with
23`tools/native_runner.py OUTPUT COLD --expected-pages 1 --collect-notebook`.
corpus/list-edit/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/list-edit/candidate/Open Notebook.onetoc2 differ
corpus/list-edit/candidate/lists.one created
Binary files /dev/null and b/corpus/list-edit/candidate/lists.one differ
corpus/list-edit/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-17B0A046 -ExpectedPages 1", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/list-edit/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-17b0a046", "hostname": "ONE-M6-17B0A046"}
corpus/list-edit/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/list-edit/cold/notebook/Open Notebook.onetoc2 differ
corpus/list-edit/cold/notebook/lists.one created
Binary files /dev/null and b/corpus/list-edit/cold/notebook/lists.one differ
corpus/list-edit/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-17B0A046",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/list-edit/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{1AC2A9B8-51B4-4638-9851-F754A13A05D2}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-12T09:14:51.000Z" color="#B49EDE"><one:Section name="lists" ID="{A3DA852C-B677-0102-2D84-7B4443EA2B27}{1}{B0}" path="C:\one-tests\runs\capture\notebook\lists.one" lastModifiedTime="2026-09-12T09:14:51.000Z" color="#8AA8E4"><one:Page ID="{CCC37BFA-FA16-02EB-03E1-4FBBE655489B}{1}{B0}" name="Bullet item" dateTime="2026-09-12T09:13:26.000Z" lastModifiedTime="2026-09-12T09:14:51.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/list-edit/cold/read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{CCC37BFA-FA16-02EB-03E1-4FBBE655489B}{1}{B0}" name="Bullet item" dateTime="2026-09-12T09:13:26.000Z" lastModifiedTime="2026-09-12T09:14:51.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-12T09:13:26.000Z" objectID="{A3DA852C-B677-0102-2D84-7B4443EA2B27}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="174.7353515625" height="93.99401092529296"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:13:26.000Z" lastModifiedTime="2026-09-12T09:13:26.000Z" objectID="{A3DA852C-B677-0102-2D84-7B4443EA2B27}{24}{B0}" alignment="left"><one:List><one:Bullet bullet="3" fontSize="11.0"/></one:List><one:T><![CDATA[Bullet item]]></one:T></one:OE><one:OE author="List author" lastModifiedBy="List author" creationTime="2026-09-12T09:13:26.000Z" lastModifiedTime="2026-09-12T09:13:26.000Z" objectID="{39E883AF-FB5F-0191-13DD-9964EEE54ADE}{0}{B0}" alignment="left"><one:List><one:Number numberSequence="1" numberFormat="##." fontSize="11.0" font="Calibri" text="I."/></one:List><one:T><![CDATA[First numbered]]></one:T></one:OE><one:OE author="List author" lastModifiedBy="List author" creationTime="2026-09-12T09:13:26.000Z" lastModifiedTime="2026-09-12T09:13:26.000Z" objectID="{5AAA1C5F-4E81-070A-01B4-590656D2919E}{0}{B0}" alignment="left"><one:List><one:Number numberSequence="1" numberFormat="##." fontSize="11.0" font="Calibri" text="II."/></one:List><one:T><![CDATA[Second numbered]]></one:T><one:OEChildren><one:OE author="List author" lastModifiedBy="List author" creationTime="2026-09-12T09:13:26.000Z" lastModifiedTime="2026-09-12T09:13:26.000Z" objectID="{6EEF8349-4BA3-08CB-1949-3CAE9C929F70}{0}{B0}" alignment="left"><one:List><one:Number numberSequence="1" numberFormat="##." fontSize="11.0" font="Calibri" text="I."/></one:List><one:T><![CDATA[Nested numbered]]></one:T></one:OE></one:OEChildren></one:OE><one:OE author="List author" lastModifiedBy="List author" creationTime="2026-09-12T09:13:26.000Z" lastModifiedTime="2026-09-12T09:13:26.000Z" objectID="{DDC3F5A0-347D-0DA0-093F-D3B15B2149AA}{0}{B0}" alignment="left"><one:List><one:Number numberSequence="1" numberFormat="##." fontSize="11.0" font="Calibri" text="III."/></one:List><one:T><![CDATA[Restarted at three]]></one:T><one:OEChildren><one:OE author="List author" lastModifiedBy="List author" creationTime="2026-09-12T09:13:26.000Z" lastModifiedTime="2026-09-12T09:13:26.000Z" objectID="{7F22E94B-5DED-0B3E-2C06-A3771BE9A013}{0}{B0}" alignment="left"><one:T><![CDATA[Nested plain]]></one:T></one:OE></one:OEChildren></one:OE><one:OE author="List author" lastModifiedBy="List author" creationTime="2026-09-12T09:13:26.000Z" lastModifiedTime="2026-09-12T09:13:26.000Z" objectID="{8FB9D23C-5BC2-0E0B-0A92-46745CD77274}{0}{B0}" alignment="left"><one:T><![CDATA[Plain again]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/list-edit/cold/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/list-edit/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/m4/lists",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/list-edit/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/list-edit/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/list-edit/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 1604,
5 "sha256": "25824736beb7e6c953922576b8d8309ec64568d08d1c850e71efa80efe2ee069",
6 "mtime_ns": 1789204406442521201
7 },
8 {
9 "path": "lists.one",
10 "bytes": 11328,
11 "sha256": "8ee832fe46e233c22995fd73ed9df9cb9ff345752570a250979aaa26c8f1ef15",
12 "mtime_ns": 1789204406442128119
13 }
14]
corpus/list-edit/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
crates/notebook/README.md+3-2
...@@ -14,8 +14,9 @@ diffs the supplied model against the page stored in the supplied local snapshot,...@@ -14,8 +14,9 @@ diffs the supplied model against the page stored in the supplied local snapshot,
14writes the difference into the working image and queues one14writes the difference into the working image and queues one
15`Operation::Page(PageIntent { before, after, author })`; `before` is the page the15`Operation::Page(PageIntent { before, after, author })`; `before` is the page the
16edit started from and is the precondition reconciliation checks. Text, styles,16edit started from and is the precondition reconciliation checks. Text, styles,
17paragraph structure, outline layout, insertions and deletions are all differences17paragraph formatting, bullets and numbering, paragraph structure, outline
18between `before` and `after`; the library never sees editor operations.18layout, insertions and deletions are all differences between `before` and
19`after`; the library never sees editor operations.
1920
20```no_run21```no_run
21use onestore::{ExGuid, page::Page};22use onestore::{ExGuid, page::Page};
crates/notebook/src/merge.rs+18-1
...@@ -97,12 +97,29 @@ pub(crate) fn merge(base: &Page, ours: &Page, theirs: &Page) -> Option<Page> {...@@ -97,12 +97,29 @@ pub(crate) fn merge(base: &Page, ours: &Page, theirs: &Page) -> Option<Page> {
97 }97 }
98 result = reorder(result, &our_order);98 result = reorder(result, &our_order);
99 }99 }
100 // Definitions we added or changed travel with our paragraphs; theirs win elsewhere.
101 let mut definitions = theirs.definitions.clone();
102 for (id, definition) in &ours.definitions {
103 match (base.definitions.get(id), theirs.definitions.get(id)) {
104 (Some(before), Some(after)) if before != definition => {
105 if after != before && after != definition {
106 return None;
107 }
108 definitions.insert(*id, definition.clone());
109 }
110 (None, None) => {
111 definitions.insert(*id, definition.clone());
112 }
113 (None, Some(after)) if after != definition => return None,
114 _ => {}
115 }
116 }
100 Some(Page {117 Some(Page {
101 title: theirs.title.clone(),118 title: theirs.title.clone(),
102 created: theirs.created,119 created: theirs.created,
103 margin_origin: theirs.margin_origin,120 margin_origin: theirs.margin_origin,
104 objects: result,121 objects: result,
105 definitions: theirs.definitions.clone(),122 definitions,
106 })123 })
107}124}
108125
crates/notebook/tests/sync_model.rs+60
...@@ -532,3 +532,63 @@ fn outline_moves_merge_with_remote_text_edits_but_not_with_remote_moves() {...@@ -532,3 +532,63 @@ fn outline_moves_merge_with_remote_text_edits_but_not_with_remote_moves() {
532 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))532 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
533 );533 );
534}534}
535
536#[test]
537fn a_local_bullet_merges_with_a_remote_text_edit_and_publishes_its_definition() {
538 let (space, before) = page_titled(OUTLINES, "Move leaf down");
539 let directory = tempfile::tempdir().unwrap();
540 let cache = Replica::create(directory.path().join("cache.sqlite"), OUTLINES).unwrap();
541 let mut after = before.clone();
542 let bullet = onestore::page::text::new_id().unwrap();
543 after.definitions.insert(
544 bullet,
545 onestore::page::Definition {
546 kind: onestore::document::Kind::List {
547 font: Some("Courier New".into()),
548 format: Some("\u{25cb}".into()),
549 restart: None,
550 bullet: Some(4),
551 },
552 format: onestore::document::Format {
553 font_size: Some(11.0),
554 color: Some(0xff000000),
555 ..Default::default()
556 },
557 },
558 );
559 let outline = after
560 .objects
561 .iter_mut()
562 .find_map(|object| match object {
563 PageObject::Outline(outline) if !outline.title => Some(outline),
564 _ => None,
565 })
566 .unwrap();
567 outline.paragraphs[0].lists = vec![bullet];
568 let id = cache
569 .save(OUTLINES, space, &after, "Model author")
570 .unwrap()
571 .unwrap();
572 let mut server = remote_with(space, |page| edit_paragraph(page, 1, 0..0, "Remote "));
573 assert!(
574 matches!(cache.sync_once(&mut server).unwrap(), Some((n, EditStatus::Published { .. })) if n == id)
575 );
576 let published = page_of(&server.durable, space);
577 let outline = published
578 .objects
579 .iter()
580 .find_map(|object| match object {
581 PageObject::Outline(outline) if !outline.title => Some(outline),
582 _ => None,
583 })
584 .unwrap();
585 assert_eq!(outline.paragraphs[0].lists, vec![bullet]);
586 assert!(matches!(
587 published.definitions[&bullet].kind,
588 onestore::document::Kind::List {
589 bullet: Some(4),
590 ..
591 }
592 ));
593 assert!(texts(&published)[1].starts_with("Remote "));
594}
crates/onestore/README.md+1-1
...@@ -57,7 +57,7 @@ harness also accepts `--client-profile release`....@@ -57,7 +57,7 @@ harness also accepts `--client-profile release`.
57| `ParagraphJoin`, `PreparedEdit::join` | Join adjacent ordinary text while preserving inherited character styles and native text-identity rules |57| `ParagraphJoin`, `PreparedEdit::join` | Join adjacent ordinary text while preserving inherited character styles and native text-identity rules |
58| `TextAttribute`, `PreparedEdit::format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs |58| `TextAttribute`, `PreparedEdit::format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs |
59| `OutlineEdit`, `PreparedEdit::outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content |59| `OutlineEdit`, `PreparedEdit::outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content |
60| `PreparedEdit::page` | Publish an edited `page::Page` as one revision per changed space: text, character formatting, paragraph insertion/split/join/move/deletion, outline insertion/deletion/position/width and saved collapse state, lowered onto the typed writers with the model's identities |60| `PreparedEdit::page` | Publish an edited `page::Page` as one revision per changed space: text, character and paragraph formatting, bullets and numbering, paragraph insertion/split/join/move/deletion, outline insertion/deletion/position/width and saved collapse state, lowered onto the typed writers with the model's identities |
61| `TreeEdit`, `PreparedEdit::tree` | Move or delete a subtree on one page, normalize surviving containers, and replace an emptied table cell's paragraph atomically |61| `TreeEdit`, `PreparedEdit::tree` | Move or delete a subtree on one page, normalize surviving containers, and replace an emptied table cell's paragraph atomically |
62| `PreparedEdit::commit`, `PreparedEdit::commit_file` | Publish the exact prepared image under caller-held exclusion or the conservative filesystem adapter |62| `PreparedEdit::commit`, `PreparedEdit::commit_file` | Publish the exact prepared image under caller-held exclusion or the conservative filesystem adapter |
63| `read_file` | Read a snapshot under whole-file exclusion |63| `read_file` | Read a snapshot under whole-file exclusion |
crates/onestore/src/page/write.rs+191-12
...@@ -5,7 +5,7 @@ use super::{Outline, Page, PageObject, PageParagraph, ParagraphContent};...@@ -5,7 +5,7 @@ use super::{Outline, Page, PageObject, PageParagraph, ParagraphContent};
5use crate::{5use crate::{
6 Error, ExGuid, Insertion, ObjectData, OutlineEdit, ParagraphJoin, ParagraphSplit, PropertySets,6 Error, ExGuid, Insertion, ObjectData, OutlineEdit, ParagraphJoin, ParagraphSplit, PropertySets,
7 RevisionIndex, Store, TextAttribute, TreeEdit, Value,7 RevisionIndex, Store, TextAttribute, TreeEdit, Value,
8 document::{Document, Format},8 document::{Document, Format, Kind},
9 write::{PropertyObject, RevisionEdit, write_revisions},9 write::{PropertyObject, RevisionEdit, write_revisions},
10};10};
11use std::{11use std::{
...@@ -173,8 +173,10 @@ impl Lowering<'_> {...@@ -173,8 +173,10 @@ impl Lowering<'_> {
173 return Err(invalid("Page creation time and margins cannot be edited"));173 return Err(invalid("Page creation time and margins cannot be edited"));
174 }174 }
175 for (id, definition) in &after.definitions {175 for (id, definition) in &after.definitions {
176 if before.definitions.get(id) != Some(definition) {176 if before.definitions.get(id) != Some(definition)
177 return Err(invalid("List, tag and style definitions cannot be edited"));177 && !matches!(definition.kind, Kind::List { .. })
178 {
179 return Err(invalid("Tag and style definitions cannot be edited"));
178 }180 }
179 }181 }
180 let old = View::new(before)?;182 let old = View::new(before)?;
...@@ -206,6 +208,7 @@ impl Lowering<'_> {...@@ -206,6 +208,7 @@ impl Lowering<'_> {
206 self.place(&old, &new, &placed, &page_order)?;208 self.place(&old, &new, &placed, &page_order)?;
207 self.delete(&old, &new, &consumed)?;209 self.delete(&old, &new, &consumed)?;
208 self.edit_text(&new)?;210 self.edit_text(&new)?;
211 self.edit_lists(after, &new)?;
209 self.edit_paragraph_formatting(&new)?;212 self.edit_paragraph_formatting(&new)?;
210 self.edit_formatting(&new)?;213 self.edit_formatting(&new)?;
211 self.edit_layout(&old, &new)?;214 self.edit_layout(&old, &new)?;
...@@ -272,13 +275,12 @@ impl Lowering<'_> {...@@ -272,13 +275,12 @@ impl Lowering<'_> {
272 let Some(previous) = old.paragraphs.get(id) else {275 let Some(previous) = old.paragraphs.get(id) else {
273 continue;276 continue;
274 };277 };
275 let same = paragraph.lists == previous.lists278 let same = paragraph.tags == previous.tags
276 && paragraph.tags == previous.tags
277 && paragraph.style == previous.style279 && paragraph.style == previous.style
278 && paragraph.format == previous.format;280 && paragraph.format == previous.format;
279 if !same {281 if !same {
280 return Err(invalid(282 return Err(invalid(
281 "Paragraph lists, tags, styles and paragraph formatting cannot be edited",283 "Paragraph tags, styles and paragraph formatting cannot be edited",
282 ));284 ));
283 }285 }
284 match (&paragraph.content, &previous.content) {286 match (&paragraph.content, &previous.content) {
...@@ -557,13 +559,11 @@ impl Lowering<'_> {...@@ -557,13 +559,11 @@ impl Lowering<'_> {
557 let text = paragraph559 let text = paragraph
558 .text()560 .text()
559 .filter(|text| text.date_field.is_none() && text.tags.is_empty())561 .filter(|text| text.date_field.is_none() && text.tags.is_empty())
560 .filter(|_| {562 .filter(|_| paragraph.tags.is_empty() && paragraph.style.is_none())
561 paragraph.lists.is_empty()
562 && paragraph.tags.is_empty()
563 && paragraph.style.is_none()
564 })
565 .ok_or_else(|| {563 .ok_or_else(|| {
566 invalid("New paragraphs contain plain text without lists, tags, fields or styles")564 invalid(
565 "New paragraphs contain plain text without tags, fields or styles",
566 )
567 })?;567 })?;
568 let insertion = Insertion::paragraph(568 let insertion = Insertion::paragraph(
569 self.id(container),569 self.id(container),
...@@ -659,6 +659,185 @@ impl Lowering<'_> {...@@ -659,6 +659,185 @@ impl Lowering<'_> {
659 Ok(())659 Ok(())
660 }660 }
661661
662 /// Gives each paragraph the list nodes its model references: a definition new to the
663 /// section becomes a list node carrying the model's identity, a changed one is rewritten
664 /// in place, a definition another paragraph already owns is copied because native list
665 /// nodes belong to one paragraph, and a dropped reference leaves the node unreferenced.
666 fn edit_lists(&mut self, after: &Page, new: &View<'_>) -> Result<(), Error> {
667 let current = self.current()?;
668 let stored = View::new(&current)?;
669 let mut owners: BTreeMap<ExGuid, ExGuid> = BTreeMap::new();
670 for (id, paragraph) in &stored.paragraphs {
671 for list in &paragraph.lists {
672 owners.insert(*list, *id);
673 }
674 }
675 for (id, paragraph) in &new.paragraphs {
676 let image_id = self.id(*id);
677 let previous = stored
678 .paragraphs
679 .get(&image_id)
680 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
681 let same = paragraph.lists.len() == previous.lists.len()
682 && paragraph
683 .lists
684 .iter()
685 .zip(&previous.lists)
686 .all(|(model, node)| {
687 after.definitions.get(model) == current.definitions.get(node)
688 });
689 if same {
690 continue;
691 }
692 let mut nodes = Vec::new();
693 for list in &paragraph.lists {
694 // Writer-allocated identities keep a nonzero sequence number so the node's
695 // compact identity is never the null identity; squash renames them to the
696 // model's. A copy for a second owner keeps its allocated identity.
697 let node_id = match owners.get(list) {
698 Some(owner) if *owner == image_id => *list,
699 Some(_) => ExGuid {
700 guid: crate::write::fresh_guid()?,
701 n: 1,
702 },
703 None => {
704 let written = ExGuid {
705 guid: crate::write::fresh_guid()?,
706 n: 1,
707 };
708 self.alias.insert(*list, written);
709 written
710 }
711 };
712 let definition = after
713 .definitions
714 .get(list)
715 .ok_or_else(|| invalid("A paragraph references a missing list definition"))?;
716 let Kind::List {
717 font,
718 format,
719 restart,
720 bullet,
721 } = &definition.kind
722 else {
723 return Err(invalid("A paragraph list must reference a list definition"));
724 };
725 if bullet.is_some() && (format.is_none() || font.is_none()) {
726 return Err(invalid("A bullet definition names its glyph and font"));
727 }
728 let mut values: Vec<(u32, Vec<u8>)> = Vec::new();
729 if let Some(format) = format {
730 let units: Vec<u16> = format.encode_utf16().collect();
731 let count = u16::try_from(units.len())
732 .map_err(|_| invalid("List format exceeds the document range"))?;
733 let mut bytes = count.to_le_bytes().to_vec();
734 bytes.extend(units.iter().flat_map(|unit| unit.to_le_bytes()));
735 values.push((0x1c001c1a, bytes));
736 }
737 if let Some(font) = font {
738 values.push((0x1c001c52, crate::create::string(font)));
739 }
740 if let Some(restart) = restart {
741 values.push((0x14001cb7, restart.to_le_bytes().to_vec()));
742 }
743 if let Some(bullet) = bullet {
744 values.push((0x10001d0e, bullet.to_le_bytes().to_vec()));
745 // Every native bullet node carries this cleared flag alongside its index.
746 values.push((0x0c001cc0, vec![0]));
747 }
748 let style = &definition.format;
749 if let Some(font) = &style.font {
750 values.push((0x1c001c0a, crate::create::string(font)));
751 }
752 if let Some(size) = style.font_size {
753 let half = (size * 2.0).round();
754 if !(0.0..=f32::from(u16::MAX)).contains(&half) {
755 return Err(invalid("List font size is outside the document range"));
756 }
757 values.push((0x10001c0b, (half as u16).to_le_bytes().to_vec()));
758 }
759 if let Some(color) = style.color {
760 values.push((0x14001c0c, color.to_le_bytes().to_vec()));
761 }
762 if let Some(language) = style.language {
763 values.push((0x14001c3b, language.to_le_bytes().to_vec()));
764 }
765 for (flag, id) in [(style.bold, 0x08001c04), (style.italic, 0x08001c05)] {
766 if let Some(flag) = flag {
767 values.push((id | (u32::from(flag) << 31), Vec::new()));
768 }
769 }
770 nodes.push((node_id, values));
771 }
772 let (space, object) = (self.space, self.id(*id));
773 self.apply(|image| {
774 let store = Store::parse(image)?;
775 let index = RevisionIndex::parse(&store)?;
776 let document = Document::parse(&index)?;
777 let parents = crate::edit::editable_parents(
778 document.active(space)?,
779 &document.pages_in(space)?,
780 object,
781 )?;
782 let modified = crate::create::current_timestamps()?.0.to_le_bytes();
783 crate::write::write_revision(image, space, |raw| {
784 let mut changed = BTreeMap::new();
785 let mut target = PropertyObject::from_object(&raw.objects[&object])?;
786 let mut references = Vec::new();
787 for (list, values) in &nodes {
788 let mut node = match raw.objects.get(list) {
789 Some(existing) => {
790 if existing.jcid != 0x60012 {
791 return Err(invalid(
792 "A list definition identity belongs to another object",
793 ));
794 }
795 let mut node = PropertyObject::from_object(existing)?;
796 node.remove(&[
797 0x1c001c1a, 0x1c001c52, 0x14001cb7, 0x10001d0e, 0x0c001cc0,
798 0x1c001c0a, 0x10001c0b, 0x14001c0c, 0x14001c3b, 0x08001c04,
799 0x08001c05,
800 ])?;
801 node
802 }
803 None => PropertyObject {
804 jcid: 0x60012,
805 bytes: crate::create::properties(&[])?,
806 global_ids: std::sync::Arc::new(BTreeMap::from([(0, list.guid)])),
807 },
808 };
809 node.set(
810 &values
811 .iter()
812 .map(|(id, bytes)| (*id, bytes.as_slice()))
813 .collect::<Vec<_>>(),
814 )?;
815 node.set(&[(0x14001d7a, &modified)])?;
816 node.reference(*list)?;
817 references.extend_from_slice(&target.reference(*list)?);
818 changed.insert(*list, node);
819 }
820 if references.is_empty() {
821 target.remove(&[0x24001c26])?;
822 } else {
823 target.set(&[(0x24001c26, &references)])?;
824 }
825 target.set(&[(0x14001d7a, &modified)])?;
826 changed.insert(object, target);
827 crate::formatting::touch_ancestors(
828 raw,
829 &parents,
830 object,
831 &modified,
832 &mut changed,
833 )?;
834 Ok(changed)
835 })
836 })?;
837 }
838 Ok(())
839 }
840
662 fn edit_paragraph_formatting(&mut self, new: &View<'_>) -> Result<(), Error> {841 fn edit_paragraph_formatting(&mut self, new: &View<'_>) -> Result<(), Error> {
663 let current = self.current()?;842 let current = self.current()?;
664 let current = View::new(&current)?;843 let current = View::new(&current)?;
crates/onestore/tests/page_lists.rs created+322
...@@ -0,0 +1,322 @@
1use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
3 document::{Document, Format, Kind},
4 page::{
5 Definition, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,
6 },
7};
8use std::collections::BTreeMap;
9
10const TREES: &[u8] =
11 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");
12const AUTHOR: &str = "List author";
13
14fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {
15 let store = Store::parse(bytes).unwrap();
16 let index = RevisionIndex::parse(&store).unwrap();
17 let document = Document::parse(&index).unwrap();
18 document
19 .pages()
20 .unwrap()
21 .into_iter()
22 .find_map(|(space, _)| {
23 let page = Page::from_space(&document, space).unwrap();
24 (page.title == title).then_some((space, page))
25 })
26 .unwrap()
27}
28
29fn page_in(bytes: &[u8], space: ExGuid) -> Page {
30 let store = Store::parse(bytes).unwrap();
31 let index = RevisionIndex::parse(&store).unwrap();
32 Page::from_space(&Document::parse(&index).unwrap(), space).unwrap()
33}
34
35fn body_paragraphs(page: &mut Page) -> &mut Vec<PageParagraph> {
36 page.objects
37 .iter_mut()
38 .find_map(|object| match object {
39 PageObject::Outline(outline) if !outline.title => Some(&mut outline.paragraphs),
40 _ => None,
41 })
42 .unwrap()
43}
44
45fn list_ids(page: &Page) -> Vec<Vec<ExGuid>> {
46 page.objects
47 .iter()
48 .filter_map(|object| match object {
49 PageObject::Outline(outline) if !outline.title => {
50 Some(outline.paragraphs.iter().map(|p| p.lists.clone()).collect())
51 }
52 _ => None,
53 })
54 .next()
55 .unwrap()
56}
57
58fn raw_object(bytes: &[u8], space: ExGuid, id: ExGuid) -> String {
59 let store = Store::parse(bytes).unwrap();
60 let index = RevisionIndex::parse(&store).unwrap();
61 format!(
62 "{:?}",
63 index.resolve_active(space).unwrap().objects[&id].data
64 )
65}
66
67fn bullet_definition() -> Definition {
68 Definition {
69 kind: Kind::List {
70 font: Some("Courier New".into()),
71 format: Some("\u{25cb}".into()),
72 restart: None,
73 bullet: Some(4),
74 },
75 format: Format {
76 font_size: Some(11.0),
77 color: Some(0xff000000),
78 ..Default::default()
79 },
80 }
81}
82
83fn plain_paragraph(template: &PageParagraph, text: &str) -> PageParagraph {
84 let mut paragraph = template.clone();
85 paragraph.id = new_id().unwrap();
86 paragraph.parent = None;
87 paragraph.level = 1;
88 paragraph.lists.clear();
89 paragraph.tags.clear();
90 paragraph.style = None;
91 paragraph.collapsed = false;
92 paragraph.content = ParagraphContent::Text(TextObject {
93 id: new_id().unwrap(),
94 date_field: None,
95 text: onestore::page::Paragraph::new(
96 text.into(),
97 Format {
98 font: Some("Calibri".into()),
99 font_size: Some(11.0),
100 language: Some(1033),
101 ..Default::default()
102 },
103 ),
104 tags: Vec::new(),
105 });
106 paragraph
107}
108
109#[test]
110fn a_numbered_paragraph_becomes_plain_while_its_neighbours_keep_their_nodes() {
111 let (space, before) = page_by_title(TREES, "Move numbered subtree down");
112 let lists = list_ids(&before);
113 let numbered: Vec<usize> = (0..lists.len()).filter(|i| !lists[*i].is_empty()).collect();
114 assert!(numbered.len() >= 2, "{lists:?}");
115 let mut after = before.clone();
116 body_paragraphs(&mut after)[numbered[0]].lists.clear();
117 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
118 let stored = page_in(written.as_bytes(), space);
119 let mut expected = after.clone();
120 expected.title = stored.title.clone();
121 expected.definitions.remove(&lists[numbered[0]][0]);
122 assert_eq!(stored, expected);
123 for other in &numbered[1..] {
124 for node in &lists[*other] {
125 assert_eq!(
126 raw_object(written.as_bytes(), space, *node),
127 raw_object(TREES, space, *node)
128 );
129 }
130 }
131 assert_eq!(
132 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
133 .unwrap()
134 .as_bytes(),
135 written.as_bytes()
136 );
137}
138
139#[test]
140fn bullets_and_numbering_write_from_definitions_and_read_back() {
141 let (space, before) = page_by_title(TREES, "Move numbered subtree down");
142 let lists = list_ids(&before);
143 let numbered = (0..lists.len()).find(|i| !lists[*i].is_empty()).unwrap();
144 let plain = (0..lists.len()).find(|i| lists[*i].is_empty()).unwrap();
145 let mut after = before.clone();
146 let bullet = new_id().unwrap();
147 after.definitions.insert(bullet, bullet_definition());
148 let template = body_paragraphs(&mut after)[plain].clone();
149 body_paragraphs(&mut after)[plain].lists = vec![bullet];
150 let number = new_id().unwrap();
151 let mut numbering = after.definitions[&lists[numbered][0]].clone();
152 let Kind::List { restart, .. } = &mut numbering.kind else {
153 panic!()
154 };
155 *restart = Some(3);
156 after.definitions.insert(number, numbering);
157 let mut fresh = plain_paragraph(&template, "Numbered from three");
158 fresh.lists = vec![number];
159 body_paragraphs(&mut after).push(fresh);
160 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
161 let stored = page_in(written.as_bytes(), space);
162 let mut expected = after.clone();
163 expected.title = stored.title.clone();
164 assert_eq!(stored, expected);
165 assert_eq!(
166 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
167 .unwrap()
168 .as_bytes(),
169 written.as_bytes()
170 );
171}
172
173#[test]
174fn a_list_definition_changes_in_place() {
175 let (space, before) = page_by_title(TREES, "Move numbered subtree down");
176 let lists = list_ids(&before);
177 let numbered = (0..lists.len()).find(|i| !lists[*i].is_empty()).unwrap();
178 let node = lists[numbered][0];
179 let mut after = before.clone();
180 let definition = after.definitions.get_mut(&node).unwrap();
181 let Kind::List { restart, .. } = &mut definition.kind else {
182 panic!()
183 };
184 *restart = Some(7);
185 definition.format.bold = Some(true);
186 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
187 let stored = page_in(written.as_bytes(), space);
188 let mut expected = after.clone();
189 expected.title = stored.title.clone();
190 assert_eq!(stored, expected);
191 assert_eq!(list_ids(&stored)[numbered], vec![node]);
192}
193
194#[test]
195fn a_definition_shared_between_paragraphs_becomes_a_node_per_paragraph() {
196 let (space, before) = page_by_title(TREES, "Move numbered subtree down");
197 let lists = list_ids(&before);
198 let numbered = (0..lists.len()).find(|i| !lists[*i].is_empty()).unwrap();
199 let plain = (0..lists.len()).find(|i| lists[*i].is_empty()).unwrap();
200 let mut shared = before.clone();
201 body_paragraphs(&mut shared)[plain].lists = lists[numbered].clone();
202 let written = PreparedEdit::page(TREES, space, &shared, AUTHOR).unwrap();
203 let stored = page_in(written.as_bytes(), space);
204 let after = list_ids(&stored);
205 assert_eq!(after[numbered], lists[numbered]);
206 assert_eq!(after[plain].len(), 1);
207 assert_ne!(after[plain], lists[numbered]);
208 assert_eq!(
209 stored.definitions[&after[plain][0]],
210 before.definitions[&lists[numbered][0]]
211 );
212}
213
214#[test]
215fn missing_and_foreign_list_definitions_are_refused() {
216 let (space, before) = page_by_title(TREES, "Move numbered subtree down");
217 let lists = list_ids(&before);
218 let plain = (0..lists.len()).find(|i| lists[*i].is_empty()).unwrap();
219 let mut missing = before.clone();
220 body_paragraphs(&mut missing)[plain].lists = vec![new_id().unwrap()];
221 assert!(PreparedEdit::page(TREES, space, &missing, AUTHOR).is_err());
222 let mut foreign = before.clone();
223 let paragraph = body_paragraphs(&mut foreign)[plain].id;
224 foreign.definitions.insert(paragraph, bullet_definition());
225 body_paragraphs(&mut foreign)[plain].lists = vec![paragraph];
226 assert!(PreparedEdit::page(TREES, space, &foreign, AUTHOR).is_err());
227 let mut style: BTreeMap<ExGuid, Definition> = before.definitions.clone();
228 if let Some((id, definition)) = style
229 .iter_mut()
230 .find(|(_, d)| matches!(d.kind, Kind::Style { .. }))
231 {
232 definition.format.bold = Some(true);
233 let mut edited = before.clone();
234 edited.definitions.insert(*id, definition.clone());
235 assert!(PreparedEdit::page(TREES, space, &edited, AUTHOR).is_err());
236 }
237}
238
239/// A fresh page gains a bullet, two numbered paragraphs with a nested numbered child, a
240/// numbering restart and a trailing plain paragraph; `ONESTORE_LIST_EXPORT` names a new
241/// directory receiving the candidate for a cold native reopen.
242#[test]
243fn bullets_numbering_nesting_and_restarts_publish_on_a_fresh_page() {
244 let source = onestore::create_section("lists.one", "Bullet item", "Author").unwrap();
245 let store = Store::parse(&source).unwrap();
246 let index = RevisionIndex::parse(&store).unwrap();
247 let document = Document::parse(&index).unwrap();
248 let (space, _) = document.pages().unwrap()[0];
249 let before = Page::from_space(&document, space).unwrap();
250 let (_, trees) = page_by_title(TREES, "Move numbered subtree down");
251 let numbering = trees
252 .definitions
253 .values()
254 .find(|d| {
255 matches!(
256 d.kind,
257 Kind::List {
258 format: Some(_),
259 ..
260 }
261 )
262 })
263 .unwrap()
264 .clone();
265 let scenario = std::env::var("ONESTORE_LIST_SCENARIO").unwrap_or_else(|_| "all".into());
266 let wants = |name: &str| scenario == "all" || scenario == name;
267 let mut after = before.clone();
268 let template = body_paragraphs(&mut after)[0].clone();
269 if wants("bullet") {
270 let bullet = new_id().unwrap();
271 after.definitions.insert(bullet, bullet_definition());
272 body_paragraphs(&mut after)[0].lists = vec![bullet];
273 }
274 for (text, level, restart, name) in [
275 ("First numbered", 1, None, "numbered"),
276 ("Second numbered", 1, None, "numbered"),
277 ("Nested numbered", 2, None, "all"),
278 ("Restarted at three", 1, Some(3), "restart"),
279 ("Nested plain", 2, None, "nested"),
280 ] {
281 if !wants(name) {
282 continue;
283 }
284 let mut paragraph = plain_paragraph(&template, text);
285 if name != "nested" {
286 let id = new_id().unwrap();
287 let mut definition = numbering.clone();
288 let Kind::List { restart: value, .. } = &mut definition.kind else {
289 panic!()
290 };
291 *value = restart;
292 after.definitions.insert(id, definition);
293 paragraph.lists = vec![id];
294 }
295 paragraph.level = level;
296 if level == 2 {
297 paragraph.parent = Some(body_paragraphs(&mut after).last().unwrap().id);
298 }
299 body_paragraphs(&mut after).push(paragraph);
300 }
301 body_paragraphs(&mut after).push(plain_paragraph(&template, "Plain again"));
302 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
303 let stored = page_in(written.as_bytes(), space);
304 let mut expected = after.clone();
305 expected.title = stored.title.clone();
306 assert_eq!(stored, expected);
307 if let Some(directory) = std::env::var_os("ONESTORE_LIST_EXPORT") {
308 let directory = std::path::PathBuf::from(directory);
309 std::fs::create_dir(&directory).unwrap();
310 std::fs::write(directory.join("lists.one"), written.as_bytes()).unwrap();
311 let written_store = Store::parse(written.as_bytes()).unwrap();
312 std::fs::write(
313 directory.join("Open Notebook.onetoc2"),
314 onestore::create_table_of_contents(
315 "Open Notebook.onetoc2",
316 &[("lists.one", written_store.header.file_id)],
317 )
318 .unwrap(),
319 )
320 .unwrap();
321 }
322}
crates/onestore/tests/support/page_edits.rs+53-5
...@@ -91,11 +91,13 @@ fn projection(page: &Page) -> String {...@@ -91,11 +91,13 @@ fn projection(page: &Page) -> String {
91 match &paragraph.content {91 match &paragraph.content {
92 ParagraphContent::Text(text) => {92 ParagraphContent::Text(text) => {
93 out.push_str(&format!(" text {} {:?}", text.id, text.text.text()));93 out.push_str(&format!(" text {} {:?}", text.id, text.text.text()));
94 // Runs that differ only in attributes the writer leaves as stored,
95 // such as the language tag, project as one span.
96 let mut runs: Vec<(usize, String)> = Vec::new();
94 for span in text.text.spans() {97 for span in text.text.spans() {
95 let f = &span.format;98 let f = &span.format;
96 out.push_str(&format!(99 let attributes = format!(
97 " [{} {:?} {:?} {:?} {:?} {:?} {:?} {:?} {:?} {:?} {:?}]",100 "{:?} {:?} {:?} {:?} {:?} {:?} {:?} {:?} {:?} {:?}",
98 span.end,
99 f.bold.unwrap_or(false),101 f.bold.unwrap_or(false),
100 f.italic.unwrap_or(false),102 f.italic.unwrap_or(false),
101 f.underline.unwrap_or(false),103 f.underline.unwrap_or(false),
...@@ -106,7 +108,16 @@ fn projection(page: &Page) -> String {...@@ -106,7 +108,16 @@ fn projection(page: &Page) -> String {
106 f.font_size,108 f.font_size,
107 f.color,109 f.color,
108 f.highlight110 f.highlight
109 ));111 );
112 match runs.last_mut() {
113 Some((end, previous)) if *previous == attributes => {
114 *end = span.end
115 }
116 _ => runs.push((span.end, attributes)),
117 }
118 }
119 for (end, attributes) in runs {
120 out.push_str(&format!(" [{end} {attributes}]"));
110 }121 }
111 }122 }
112 ParagraphContent::Table(table) => {123 ParagraphContent::Table(table) => {
...@@ -188,7 +199,44 @@ fn mutate(page: &mut Page, bytes: &mut Bytes<'_>) {...@@ -188,7 +199,44 @@ fn mutate(page: &mut Page, bytes: &mut Bytes<'_>) {
188 .filter(|(_, o)| matches!(o, PageObject::Outline(_)))199 .filter(|(_, o)| matches!(o, PageObject::Outline(_)))
189 .map(|(i, _)| i)200 .map(|(i, _)| i)
190 .collect();201 .collect();
191 match kind % 9 {202 match kind % 10 {
203 8 => {
204 let Some(outline) = outlines.first().map(|i| match &mut page.objects[*i] {
205 PageObject::Outline(outline) => outline,
206 _ => unreachable!(),
207 }) else {
208 continue;
209 };
210 let at = usize::from(bytes.next().unwrap_or(0)) % outline.paragraphs.len().max(1);
211 let Some(paragraph) = outline.paragraphs.get_mut(at) else {
212 continue;
213 };
214 if paragraph.text().is_none() {
215 continue;
216 }
217 if paragraph.lists.is_empty() {
218 let id = new_id().unwrap();
219 page.definitions.insert(
220 id,
221 onestore::page::Definition {
222 kind: onestore::document::Kind::List {
223 font: Some("Courier New".into()),
224 format: Some("\u{25cb}".into()),
225 restart: None,
226 bullet: Some(4),
227 },
228 format: onestore::document::Format {
229 font_size: Some(11.0),
230 color: Some(0xff000000),
231 ..Default::default()
232 },
233 },
234 );
235 paragraph.lists = vec![id];
236 } else {
237 paragraph.lists.clear();
238 }
239 }
192 0 | 1 => {240 0 | 1 => {
193 let Some(o) = bytes.pick(outlines.len()) else {241 let Some(o) = bytes.pick(outlines.len()) else {
194 continue;242 continue;
tools/test_list_edit.py created+52
...@@ -0,0 +1,52 @@
1from pathlib import Path
2import re
3import runpy
4import shutil
5from tempfile import TemporaryDirectory
6import unittest
7import xml.etree.ElementTree as ET
8
9from native_xml import ns
10
11ROOT = Path(__file__).resolve().parent.parent
12FIXTURE = ROOT / 'corpus/list-edit'
13compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
14
15
16def items(read):
17 page, = (ET.parse(path).getroot() for path in sorted(read.glob('page-*.xml')))
18 outline, = page.findall('one:Outline', ns)
19
20 def walk(element, depth):
21 marker = element.find('one:List', ns)
22 kind = None
23 if marker is not None:
24 child, = list(marker)
25 kind = (child.tag.split('}')[1], child.get('text'))
26 text = re.sub(r'<[^>]*>', '', element.find('one:T', ns).text or '')
27 yield depth, kind, text
28 for nested in element.findall('one:OEChildren/one:OE', ns):
29 yield from walk(nested, depth + 1)
30
31 return [item for element in outline.findall('one:OEChildren/one:OE', ns) for item in walk(element, 0)]
32
33
34class ListEditTest(unittest.TestCase):
35 def test_rust_bullets_numbering_and_restarts_render_natively(self):
36 with TemporaryDirectory() as temporary:
37 read = Path(temporary) / 'read'
38 shutil.copytree(FIXTURE / 'cold/read', read)
39 compare(FIXTURE / 'candidate', read)
40 self.assertEqual(items(FIXTURE / 'cold/read'), [
41 (0, ('Bullet', None), 'Bullet item'),
42 (0, ('Number', 'I.'), 'First numbered'),
43 (0, ('Number', 'II.'), 'Second numbered'),
44 (1, ('Number', 'I.'), 'Nested numbered'),
45 (0, ('Number', 'III.'), 'Restarted at three'),
46 (1, None, 'Nested plain'),
47 (0, None, 'Plain again'),
48 ])
49
50
51if __name__ == '__main__':
52 unittest.main()